-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Misc]: How to build on Ascend NPU #1
Comments
目前只支持离线推理吗?openai服务接口运行报错: |
@beardog6 当前还在开发阶段,这些特性还没调试,欢迎进行合作开发,开发分支为npu_support |
@beardog6 这个是你测试的场景吗
|
是的,启动参数有所不同 @wangshuai09 |
我上面的测试通过了,你可以拉取最新的代码,看看你的参数可以跑通吗 |
Qwen2和Qwen2.5单卡测试通过 |
This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you! |
This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you! |
docker pull ascendai/pytorch:2.1.0-ubuntu22.04
docker run -p 2022:22 --name test-vllm --device /dev/davinci0 --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info -itd ascendai/pytorch:2.1.0-ubuntu22.04 bash
yum install git pip uninstall torch_npu git clone https://github.com/wangshuai09/vllm cd vllm git chekcout npu_support
VLLM_TARGET_DEVICE=npu pip install -e .
python examples/offline_inference_npu.py
Wechat Group
Scan the QR code via WeChat and join the

vLLM-NPU 交流群
.Before submitting a new issue...
The text was updated successfully, but these errors were encountered: