Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for smaller text encoders #3

Open
vladmandic opened this issue May 16, 2024 · 2 comments
Open

Support for smaller text encoders #3

vladmandic opened this issue May 16, 2024 · 2 comments

Comments

@vladmandic
Copy link
Contributor

Currently MuLan internally uses OpenGVLab/InternVL-14B-224px as default text encoder
While its possible to pass path to any downloadable encoder, which ones did you test?

Note that InternVL-14B-224px is a massive model at 27GB in size and requires ~17GB of VRAM to execute in FP16 context which prohibits usage of this library on any normal consumer GPU

@Zeqiang-Lai
Copy link
Contributor

Great suggestion! Making MuLan available for every one is our ultimate goal and we will make some attempts on the smaller one.

@zengjie617789
Copy link

zengjie617789 commented May 24, 2024

I used Mini-InternVL-Chat-2B-V1-5 as text_encoder, but it loaded so slow and want to enter yes to trust_remote_code.
what is the problem?

now, it worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants