New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LLaVA-Video-7B-Qwen2 int4 quantization enabling on ARC #12482

Open

zhangcong2019 opened this issue Dec 3, 2024 · 0 comments

Assignees

Labels

zhangcong2019 commented Dec 3, 2024

From this leaderboard Open VLM Video Leaderboard - a Hugging Face Space by opencompass, llava-video is the leading model in video LLM.

LLaVA-Video-7B-Qwen2 fp16 cannot run on ARC A770 16GB, need 4bit quantization to run this model on A770.

glorysdj assigned Oscilloscope98

Oscilloscope98 assigned MeouSker77 and unassigned Oscilloscope98

qiuxin2012 added the user issue label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment