Skip to content

Issues: runpod-workers/worker-vllm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

22 Open 54 Closed
22 Open 54 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Chat completion (template) not working with VLLM 0.6.3 + Serverless
#129 opened Nov 12, 2024 by xingyaoww
5
Are visual LLMs supported?
#122 opened Oct 9, 2024 by iamkhalidbashir
3
How can I make an update package , like "pip install -U transformers"
#116 opened Sep 27, 2024 by dat-lequoc
An error occurred: The checkpoint you are trying to load has model type qwen2_vl but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
#114 opened Sep 23, 2024 by kicks66
1
Function calling auto-tool-choice
#113 opened Sep 22, 2024 by RRaphaell
1
Randomly the machine get stuck on loading model
#112 opened Sep 19, 2024 by Sapessii
2
Very slow cold starts even with flashboot
#111 opened Sep 18, 2024 by avacaondata
3
Correct way to implement RAG with vllm
#103 opened Aug 22, 2024 by Hel1zor
Documentation incorrect regarding boolean
#101 opened Aug 21, 2024 by scriptcoded
MODEL_REVISION & TOKENIZER_REVISION: Both are needed to configure the revision
#100 opened Aug 20, 2024 by TimPietrusky
Bitsandbytes support
#99 opened Aug 19, 2024 by ilyalasy
Support GGUF models
#98 opened Aug 12, 2024 by vladfaust
ValueError: rope_scaling must be a dictionary with two fields, type and factor
#89 opened Aug 1, 2024 by omar93939
2
Unable to deploy mistralai/Mistral-Nemo-Instruct-2407
#88 opened Jul 30, 2024 by TheMindExpansionNetwork
5
[feat] ability to set max_num_seqs
#87 opened Jul 30, 2024 by kalocide
1
Support for tools / tool_choice="auto" in OpenAI-compatible API
#85 opened Jul 19, 2024 by TimPietrusky
1 of 2 tasks
23
Issue: Update VLLM to Version .5.0++, and a few suggestions
#83 opened Jul 3, 2024 by nerdylive123
13
Gemma-2 is not available in this docker image.
#81 opened Jul 1, 2024 by codingchild2424
Update documentation to note support for extra parameters
#69 opened May 13, 2024 by bryankruman
1
Multi-LoRA
#60 opened Mar 27, 2024 by joaomsimoes
OpenAI API: API errors have wrong HTTP code
#57 opened Mar 13, 2024 by lucasavila00
3
Errors cause the instance to run indefinitely
#29 opened Dec 27, 2023 by gabewillen
23
ProTip! Updated in the last three days: updated:>2024-11-21.