Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sync with upstream @ v0.6.5 #265

Merged
merged 398 commits into from
Dec 19, 2024
Merged

Sync with upstream @ v0.6.5 #265

merged 398 commits into from
Dec 19, 2024

Conversation

dtrifiro
Copy link

SageMoore and others added 30 commits November 25, 2024 22:00
@SageMoore @youkaichao
custom allreduce + torch.compile (vllm-project#10121)
9a88f89 
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@DarkLight1337
[Misc] Remove outdated init protocols (vllm-project#10655)
9406353 
Signed-off-by: DarkLight1337 <[email protected]>
@youkaichao
[ci] add vllm_test_utils (vllm-project#10659)
334d64d 
Signed-off-by: youkaichao <[email protected]>
@jikunshang
[V1] Enable profile for LLMEngine (vllm-project#10665)
1f6584e
@andoorve @sroy745
[Bugfix] Fix for Spec model TP + Chunked Prefill (vllm-project#10232)
db66e01 
Signed-off-by: andoorve <[email protected]>
Signed-off-by: Sourashis Roy <[email protected]>
Co-authored-by: Sourashis Roy <[email protected]>
@conroy-cheers
[Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (vllm-project#9735)
f5792c7 
Signed-off-by: Conroy Cheers <[email protected]>
@mgoin
[Bugfix] Fix using -O[0,3] with LLM entrypoint (vllm-project#10677)
9a99273 
Signed-off-by: mgoin <[email protected]>
@mgoin
[Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (vllm-project#…
7576cd3 
…10642)
@ywang96
[V1] Refactor model executable interface for multimodal models (vllm-…
2f0a0a1 
…project#10570)

Signed-off-by: Roger Wang <[email protected]>
@xuechendi
Remove hard-dependencies of Speculative decode to CUDA workers (vllm-…
0a71900 
…project#10587)

Signed-off-by: Chendi Xue <[email protected]>
@ywang96
[V1] Update interface for idefics3 (vllm-project#10680)
0a4d968 
Signed-off-by: Roger Wang <[email protected]>
@jeongin601
[Bugfix][SpecDecode] apply sampling parameters to target probabilitie…
1bf905d 
…s for consistency in rejection sampling. (vllm-project#10198)

Signed-off-by: jeongin601 <[email protected]>
Signed-off-by: jeong_in.bae <[email protected]>
@yansh97
[bugfix] fix the default value of llm_int8_threshold in BitsAndBytesC…
cfb3bf2 
…onfig (vllm-project#10657)
@jikunshang
[Hardware][Gaudi]add get_name method for HPUAttentionBackend (vllm-pr…
e85250b 
…oject#10667)

Signed-off-by: Kunshang Ji <[email protected]>
@jeejeelee
[Misc]Further reduce BNB static variable (vllm-project#10597)
15cc2a9 
Signed-off-by: Jee Jee Li <[email protected]>
@tlrmchlsmth
[Kernel] Remove if-else with identical branches in marlin 2:4 (vllm-p…
e225110 
…roject#10687)

Signed-off-by: Tyler Michael Smith <[email protected]>
@shunxing12345 @Isotr0py
[Model] Support telechat2 (vllm-project#10311)
1209261 
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: xiangw2 <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
@bigPYJ1151
[Bugfix][Hardware][CPU] Fix intel-omp version to avoid segfault (vllm…
418cb3b 
…-project#10700)

Signed-off-by: jiang1.li <[email protected]>
@ywang96
[V1] Update interface for mistral-format Pixtral (vllm-project#10703)
9e0a147 
Signed-off-by: Roger Wang <[email protected]>
@youkaichao
[ci] fix slow tests (vllm-project#10698)
308cc5e 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] fix shape specialization (vllm-project#10722)
c411def 
Signed-off-by: youkaichao <[email protected]>
@Isotr0py
[Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (vllm-pr…
b98c62b 
…oject#10675)

Signed-off-by: Isotr0py <[email protected]>
@mzusman
[Bugfix][Mamba] Fix Multistep on Mamba-like models (vllm-project#10705)
197b448 
Signed-off-by: mzusman <[email protected]>
@DarkLight1337
[Bugfix] Ignore lm_head when loading embedding models (vllm-project…
9b4b150 
…#10719)

Signed-off-by: DarkLight1337 <[email protected]>
@tomeras91
[Frontend] don't block event loop in tokenization (preprocess) in Ope…
395b1c7 
…nAI compatible server (vllm-project#10635)

Signed-off-by: Tomer Asida <[email protected]>
@youkaichao
[misc] upgrade filelock version (vllm-project#10731)
cb4e1c3 
Signed-off-by: youkaichao <[email protected]>
@zixuanzhang226
[Model] support bitsandbytes quantization with minicpm3 model (vllm-p…
70dc14f 
…roject#10682)

Signed-off-by: Ubuntu <[email protected]>
@spacewander
[Doc] Update model in arch_overview.rst to match comment (vllm-projec…
278be67 
…t#10701)

Signed-off-by: spacewander <[email protected]>
@rickyyx
[Bug][CLI] Allow users to disable prefix caching explicitly (vllm-pro…
d9b4b3f 
…ject#10724)

Signed-off-by: rickyx <[email protected]>
@WoosukKwon
[V1] Do not allocate beyond the max_model_len (vllm-project#10730)
a79b122 
Signed-off-by: Woosuk Kwon <[email protected]>
janimo and others added 18 commits December 16, 2024 09:56
@janimo
[Model] Support Cohere2ForCausalLM (Cohere R7B) (vllm-project#11203)
bddbbcb
@Isotr0py @DarkLight1337
[Model] Refactor Ultravox to use merged input processor (vllm-project…
d927dbc 
…#11198)

Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@Isotr0py
[Doc] Reorder vision language examples in alphabet order (vllm-projec…
2ca830d 
…t#11228)

Signed-off-by: Isotr0py <[email protected]>
@varun-sundar-rabindranath
[misc] Layerwise profile updates (vllm-project#10242)
efbce85 
Signed-off-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
@youkaichao
[core] overhaul memory profiling and fix backward compatibility (vllm…
551603f 
…-project#10511)

Signed-off-by: youkaichao <[email protected]>
@bk-TurbaAI @mgoin
[Docs] hint to enable use of GPU performance counters in profiling to…
35ffa68 
…ols for multi-node distributed serving (vllm-project#11235)

Co-authored-by: Michael Goin <[email protected]>
@youkaichao
[ci][tests] add gh200 tests (vllm-project#11244)
c301616 
Signed-off-by: youkaichao <[email protected]>
@youkaichao @tlrmchlsmth
[torch.compile] fast inductor (vllm-project#11108)
88a412e 
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: Tyler Michael Smith <[email protected]>
@youkaichao
fix gh200 tests on main (vllm-project#11246)
35bae11 
Signed-off-by: youkaichao <[email protected]>
@mgoin
[CI] Add test case with JSON schema using references + use xgrammar b…
0064f69 
…y default with OpenAI parse (vllm-project#10935)

Signed-off-by: mgoin <[email protected]>
@kylehh @DarkLight1337
[Frontend] Add OpenAI API support for input_audio (vllm-project#11027)
66d4b16 
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@ywang96
[V1][VLM] Proper memory profiling for image language models (vllm-pro…
59c9b6e 
…ject#11210)

Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: ywang96 <[email protected]>
@wangxiyuan
[Platform] platform agnostic for EngineArgs initialization (vllm-proj…
e88db68 
…ect#11225)

Signed-off-by: wangxiyuan <[email protected]>
@tlrmchlsmth
[V1][Core] Use weakref.finalize instead of atexit (vllm-project#11242)
2bfdbf2 
Signed-off-by: Tyler Michael Smith <[email protected]>
@ywang96 @BBuf
[Misc] Kernel Benchmark for RMSNorm (vllm-project#11241)
02222a0 
Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Xiaoyu Zhang <[email protected]>
@Isotr0py
[Misc] Allow passing logits_soft_cap for xformers backend (vllm-proje…
f9ecbb1 
…ct#11252)

Signed-off-by: Isotr0py <[email protected]>
@joerunde
[Bugfix] Fix request cancellation without polling (vllm-project#11190)
2d1b9ba
@dtrifiro
Sync with upstream @ v0.6.5
438ea3c 
Release notes: https://github.com/vllm-project/vllm/releases/tag/v0.6.5
@dtrifiro dtrifiro requested a review from njhill as a code owner December 18, 2024 14:50
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 18, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dtrifiro

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dtrifiro dtrifiro added the lgtm label Dec 18, 2024
@NickLucche
Copy link

/lgtm <.<

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 18, 2024

@dtrifiro: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/smoke-test 438ea3c link true /test smoke-test
ci/prow/images 438ea3c link true /test images

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@dtrifiro dtrifiro merged commit 412cbfb into opendatahub-io:main Dec 19, 2024
8 of 11 checks passed
@dtrifiro dtrifiro deleted the sync-with-0.6.5 branch December 19, 2024 11:53
@dtrifiro dtrifiro mentioned this pull request Dec 19, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@njhill njhill Awaiting requested review from njhill njhill is a code owner

@NickLucche NickLucche Awaiting requested review from NickLucche

@RH-steve-grubb RH-steve-grubb Awaiting requested review from RH-steve-grubb

Assignees
No one assigned
Labels
approved lgtm ok-to-test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.