Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

是否有int版本的权重? #216

Open
qiangruoyu opened this issue Jan 3, 2025 · 1 comment
Open

是否有int版本的权重? #216

qiangruoyu opened this issue Jan 3, 2025 · 1 comment

Comments

@qiangruoyu
Copy link

Is your feature request related to a problem? Please describe.
是否有int版本的权重,希望能够通过单节点8*80G H800来部署,是否有int版本的效果表现。

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@haswelliris
Copy link
Contributor

参考trt-llm转换int4/int8权重 https://github.com/NVIDIA/TensorRT-LLM/tree/deepseek/examples/deepseek_v3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants