-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add qwen2-vl HF GPU example #12606
base: main
Are you sure you want to change the base?
Add qwen2-vl HF GPU example #12606
Conversation
pip install "trl<0.12.0" | ||
|
||
# [optional] only needed if you would like to use ModelScope as model hub | ||
pip install modelscope==1.21.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Refer to https://github.com/modelscope/modelscope/blob/master/requirements/datasets.txt, we may have a try on
pip install modelscope[datasets]==1.21.0
, as it includes addict simplejson python-dateutil sortedcontainers
-------------------- Input Image -------------------- | ||
http://farm6.staticflickr.com/5268/5602445367_3504763978_z.jpg | ||
-------------------- Prompt -------------------- | ||
请描述这幅图片 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's use What is in the image?
and 图片里有什么?
as the sample prompts to make it consistent with other examples. And move Chinese prompt to the 1st sample output.
low_cpu_mem_usage=True, | ||
use_cache=True, | ||
model_hub=model_hub) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
model = Qwen2VLForConditionalGeneration.from_pretrained(model_path,
load_in_4bit=True,
optimize_model=True,
trust_remote_code=True,
modules_to_not_convert=["vision"],
use_cache=True,
model_hub=model_hub)
ipex_llm.transformers.Qwen2VLForConditionalGeneration
from_pretrained