Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[do-not-merge] Ibm 20241217 #264

Closed
wants to merge 393 commits into from
Closed

[do-not-merge] Ibm 20241217 #264

wants to merge 393 commits into from

Conversation

fialhocoelho
Copy link

build image.

2015aroras and others added 30 commits November 25, 2024 17:26
@2015aroras
[Model] Add OLMo November 2024 model (vllm-project#10503)
9db713a
@youkaichao
[misc] do not read HOST_IP (vllm-project#10644)
6e9ff05 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[bugfix] fix aria model and add torch.compile (vllm-project#10645)
45ac4ff 
Signed-off-by: youkaichao <[email protected]>
@sanketkaleoss @mgoin
[Feature] vLLM ARM Enablement for AARCH64 CPUs (vllm-project#9228)
a6760f6 
Signed-off-by: Sanket Kale <[email protected]>
Co-authored-by: Sanket Kale <[email protected]>
Co-authored-by: mgoin <[email protected]>
@rickyyx
[v1] EngineArgs for better config handling for v1 (vllm-project#10382)
519e8e4 
Signed-off-by: rickyx <[email protected]>
@SageMoore @youkaichao
custom allreduce + torch.compile (vllm-project#10121)
9a88f89 
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@DarkLight1337
[Misc] Remove outdated init protocols (vllm-project#10655)
9406353 
Signed-off-by: DarkLight1337 <[email protected]>
@youkaichao
[ci] add vllm_test_utils (vllm-project#10659)
334d64d 
Signed-off-by: youkaichao <[email protected]>
@jikunshang
[V1] Enable profile for LLMEngine (vllm-project#10665)
1f6584e
@andoorve @sroy745
[Bugfix] Fix for Spec model TP + Chunked Prefill (vllm-project#10232)
db66e01 
Signed-off-by: andoorve <[email protected]>
Signed-off-by: Sourashis Roy <[email protected]>
Co-authored-by: Sourashis Roy <[email protected]>
@conroy-cheers
[Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (vllm-project#9735)
f5792c7 
Signed-off-by: Conroy Cheers <[email protected]>
@mgoin
[Bugfix] Fix using -O[0,3] with LLM entrypoint (vllm-project#10677)
9a99273 
Signed-off-by: mgoin <[email protected]>
@mgoin
[Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (vllm-project#…
7576cd3 
…10642)
@ywang96
[V1] Refactor model executable interface for multimodal models (vllm-…
2f0a0a1 
…project#10570)

Signed-off-by: Roger Wang <[email protected]>
@xuechendi
Remove hard-dependencies of Speculative decode to CUDA workers (vllm-…
0a71900 
…project#10587)

Signed-off-by: Chendi Xue <[email protected]>
@ywang96
[V1] Update interface for idefics3 (vllm-project#10680)
0a4d968 
Signed-off-by: Roger Wang <[email protected]>
@jeongin601
[Bugfix][SpecDecode] apply sampling parameters to target probabilitie…
1bf905d 
…s for consistency in rejection sampling. (vllm-project#10198)

Signed-off-by: jeongin601 <[email protected]>
Signed-off-by: jeong_in.bae <[email protected]>
@yansh97
[bugfix] fix the default value of llm_int8_threshold in BitsAndBytesC…
cfb3bf2 
…onfig (vllm-project#10657)
@jikunshang
[Hardware][Gaudi]add get_name method for HPUAttentionBackend (vllm-pr…
e85250b 
…oject#10667)

Signed-off-by: Kunshang Ji <[email protected]>
@jeejeelee
[Misc]Further reduce BNB static variable (vllm-project#10597)
15cc2a9 
Signed-off-by: Jee Jee Li <[email protected]>
@tlrmchlsmth
[Kernel] Remove if-else with identical branches in marlin 2:4 (vllm-p…
e225110 
…roject#10687)

Signed-off-by: Tyler Michael Smith <[email protected]>
@shunxing12345 @Isotr0py
[Model] Support telechat2 (vllm-project#10311)
1209261 
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: xiangw2 <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
@bigPYJ1151
[Bugfix][Hardware][CPU] Fix intel-omp version to avoid segfault (vllm…
418cb3b 
…-project#10700)

Signed-off-by: jiang1.li <[email protected]>
@ywang96
[V1] Update interface for mistral-format Pixtral (vllm-project#10703)
9e0a147 
Signed-off-by: Roger Wang <[email protected]>
@youkaichao
[ci] fix slow tests (vllm-project#10698)
308cc5e 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] fix shape specialization (vllm-project#10722)
c411def 
Signed-off-by: youkaichao <[email protected]>
@Isotr0py
[Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (vllm-pr…
b98c62b 
…oject#10675)

Signed-off-by: Isotr0py <[email protected]>
@mzusman
[Bugfix][Mamba] Fix Multistep on Mamba-like models (vllm-project#10705)
197b448 
Signed-off-by: mzusman <[email protected]>
@DarkLight1337
[Bugfix] Ignore lm_head when loading embedding models (vllm-project…
9b4b150 
…#10719)

Signed-off-by: DarkLight1337 <[email protected]>
@tomeras91
[Frontend] don't block event loop in tokenization (preprocess) in Ope…
395b1c7 
…nAI compatible server (vllm-project#10635)

Signed-off-by: Tomer Asida <[email protected]>
DarkLight1337 and others added 22 commits December 15, 2024 06:30
@DarkLight1337
[Misc] Clean up multi-modal processor (vllm-project#11207)
b10609e 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Bugfix] Fix error handling of unsupported sliding window (vllm-proje…
96d673e 
…ct#11213)

Signed-off-by: DarkLight1337 <[email protected]>
@KuntaiDu
[Doc] add documentation for disaggregated prefilling (vllm-project#11197
38e599d 
)

Signed-off-by: Kuntai Du <[email protected]>
@ShangmingCai
[Core] Support disaggregated prefill with Mooncake Transfer Engine (v…
d263bd9 
…llm-project#10884)

Signed-off-by: Shangming Cai <[email protected]>
@WoosukKwon
[V1][Minor] Cache np arange to reduce input preparation overhead (vll…
25ebed2 
…m-project#11214)

Signed-off-by: Woosuk Kwon <[email protected]>
@AlexHe99
Update deploying_with_k8s.rst (vllm-project#10922)
da6f409
@chenqianfzh
[Bugfix] Fix block size validation (vllm-project#10938)
69ba344
@yansh97
[Bugfix] Fix the default value for temperature in ChatCompletionReque…
17138af 
…st (vllm-project#11219)
@cennn @drikster80
WIP: [CI/Build] simplify Dockerfile build for ARM64 / GH200 (vllm-pro…
b3b1526 
…ject#11212)

Signed-off-by: drikster80 <[email protected]>
Co-authored-by: drikster80 <[email protected]>
@janimo
[Model] Support Cohere2ForCausalLM (Cohere R7B) (vllm-project#11203)
bddbbcb
@Isotr0py @DarkLight1337
[Model] Refactor Ultravox to use merged input processor (vllm-project…
d927dbc 
…#11198)

Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@Isotr0py
[Doc] Reorder vision language examples in alphabet order (vllm-projec…
2ca830d 
…t#11228)

Signed-off-by: Isotr0py <[email protected]>
@varun-sundar-rabindranath
[misc] Layerwise profile updates (vllm-project#10242)
efbce85 
Signed-off-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
@youkaichao
[core] overhaul memory profiling and fix backward compatibility (vllm…
551603f 
…-project#10511)

Signed-off-by: youkaichao <[email protected]>
@bk-TurbaAI @mgoin
[Docs] hint to enable use of GPU performance counters in profiling to…
35ffa68 
…ols for multi-node distributed serving (vllm-project#11235)

Co-authored-by: Michael Goin <[email protected]>
@youkaichao
[ci][tests] add gh200 tests (vllm-project#11244)
c301616 
Signed-off-by: youkaichao <[email protected]>
@youkaichao @tlrmchlsmth
[torch.compile] fast inductor (vllm-project#11108)
88a412e 
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: Tyler Michael Smith <[email protected]>
@youkaichao
fix gh200 tests on main (vllm-project#11246)
35bae11 
Signed-off-by: youkaichao <[email protected]>
@fialhocoelho
Merge branch 'main' of https://github.com/vllm-project/vllm into ibm-…
e08d963 
…20241217

Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho
Squash 6357
aefef9a 
Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho
Squash 10235
6ef1de1 
Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho
install numactl to enable fastsafetensors
ac7676e 
Signed-off-by: Jefferson Fialho <[email protected]>
@fialhocoelho fialhocoelho self-assigned this Dec 18, 2024
@fialhocoelho fialhocoelho requested a review from njhill as a code owner December 18, 2024 12:23
@openshift-ci openshift-ci bot requested review from rpancham and tarukumar December 18, 2024 12:23
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 18, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: fialhocoelho
Once this PR has been reviewed and has the lgtm label, please assign tarukumar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 18, 2024

@fialhocoelho: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/smoke-test ac7676e link true /test smoke-test
ci/prow/images ac7676e link true /test images

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@njhill njhill Awaiting requested review from njhill njhill is a code owner

@rpancham rpancham Awaiting requested review from rpancham

@tarukumar tarukumar Awaiting requested review from tarukumar

Assignees

@fialhocoelho fialhocoelho

Labels
do-not-merge/hold
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.