Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qwen-VL-7B model +vllm assert len(indices) == len(inputs) AssertionError #542

Open
sssunXw opened this issue Nov 18, 2024 · 0 comments
Open

Comments

@sssunXw
Copy link

sssunXw commented Nov 18, 2024

Using webui or vllm api will result in an error. The startup command is as follows:

CUDA_VISIBLE_DEVICES=0,1,2,3 API_PORT=8000 llamafactory-cli api
--model_name_or_path /mnt/workspace/models/qwen2_vl_7b
--template qwen2_vl
--infer_backend vllm
--vllm_enforce_eager

or llamafactory-cli webui

curl http://localhost:8000/v1/chat/completions
-H "Content-Type: application/json"
-d '{
"model": "Qwen2-VL-7B-Instruct",
"messages": [
{"role": "system", "content": "你是资深分析师."},
{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": "xxx"}},
{"type": "text", "text": "描述用户文化程度水平分布"}
]}
]
}'

The error is as follows:
image

@sssunXw sssunXw changed the title Qwen-VL-7B model +vllm推理图片内容报错 Qwen-VL-7B model +vllm assert len(indices) == len(inputs) AssertionError Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant