Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add qwen2-vl HF GPU example #12606

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

ATMxsp01
Copy link
Contributor

@ATMxsp01 ATMxsp01 commented Dec 24, 2024

  • Add support for ipex_llm.transformers.Qwen2VLForConditionalGeneration
  • Add Qwen2-VL GPU examples with from_pretrained

@ATMxsp01 ATMxsp01 marked this pull request as ready for review December 30, 2024 09:16
@Oscilloscope98 Oscilloscope98 self-requested a review December 30, 2024 09:20
pip install "trl<0.12.0"

# [optional] only needed if you would like to use ModelScope as model hub
pip install modelscope==1.21.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refer to https://github.com/modelscope/modelscope/blob/master/requirements/datasets.txt, we may have a try on
pip install modelscope[datasets]==1.21.0, as it includes addict simplejson python-dateutil sortedcontainers

-------------------- Input Image --------------------
http://farm6.staticflickr.com/5268/5602445367_3504763978_z.jpg
-------------------- Prompt --------------------
请描述这幅图片
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use What is in the image? and 图片里有什么? as the sample prompts to make it consistent with other examples. And move Chinese prompt to the 1st sample output.

low_cpu_mem_usage=True,
use_cache=True,
model_hub=model_hub)

Copy link
Contributor

@Oscilloscope98 Oscilloscope98 Jan 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    model = Qwen2VLForConditionalGeneration.from_pretrained(model_path,
                                                 load_in_4bit=True,
                                                 optimize_model=True,
                                                 trust_remote_code=True,
                                                 modules_to_not_convert=["vision"],
                                                 use_cache=True,
                                                 model_hub=model_hub)

@Oscilloscope98 Oscilloscope98 changed the title Add qwen2-vl example Add qwen2-vl HF GPU example Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants