Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Main w8a8 fp8 #2

Open
wants to merge 259 commits into
base: yych
Choose a base branch
from
Open

Main w8a8 fp8 #2

wants to merge 259 commits into from

Conversation

HandH1998
Copy link
Owner

Motivation

Modifications

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

XiaotongJiang and others added 30 commits January 9, 2025 13:43
@XiaotongJiang
[Doc]: Deepseek reference docs (sgl-project#2787)
11fffbc
@zhaochenyang20
Doc: add block-wise FP8 in dpsk model reference (sgl-project#2830)
5cc1170
@merrymercy @heiner
Update README.md (sgl-project#2833)
2db03a0 
Co-authored-by: Heiner <[email protected]>
@Mutinifni
Add more metrics to serving benchmark. (sgl-project#2819)
8f15789
@CatherineSue
[Bugfix] Fix embedding model hangs with --enable-metrics (sgl-proje…
f290bd4 
…ct#2822)
@Muqi1029
[Bugfix] Fix bug in fork logic caused by null text_ (sgl-project#2835)
5413ec2
@gty111
Fix port number overflow (sgl-project#2826)
b170646
@coolhok
[Eagle2] Fix multiple concurrent request crashes (sgl-project#2730)
a47bf39
@xiezhq-hermann
Cache controller for hierarchical caching (sgl-project#2804)
5d6e946
@merrymercy
Update threshold in test_nightly_gsm8k_eval.py (sgl-project#2836)
f176958
@BBuf
[HotFix] fix fp8 scale load failed in tp>1 (sgl-project#2837)
f0e15dc
@zhyncs
chore: bump v0.4.1.post5 (sgl-project#2840)
f624901
@zhyncs
docs: update README (sgl-project#2841)
197cbf9
@shuaills
Improve: Token-In Token-Out Usage for RLHF (sgl-project#2843)
c4f9707
@BBuf
add sampling_scaling_penalties kernel (sgl-project#2846)
e2b16c4
@zhyncs
fix sgl-kernel build (sgl-project#2850)
a879c2f
@ispobock
Add int8 quant kernel (sgl-project#2848)
85b2e05
@bjmsong @mdattack
Support FP8 E4M3 KV Cache (sgl-project#2786)
0bb0f76 
Co-authored-by: root <[email protected]>
@sogalin @HaiShaw
Update base image for ROCm (sgl-project#2852)
a18ab81 
Co-authored-by: HAI <[email protected]>
@kkHuang-amd @sogalin
Integrate ROCm ater package for ck moe function feasibility (sgl-proj…
e808c1d 
…ect#2854)

Co-authored-by: wunhuang <[email protected]>
Co-authored-by: Lin, Soga <[email protected]>
@coolhok
[Fix]eagle2 health_generate is first request,apiserver will core (sgl…
4093aa4 
…-project#2853)
@merrymercy @rkooo567
Fix linear.py and improve weight loading (sgl-project#2851)
72c7776 
Co-authored-by: SangBin Cho <[email protected]>
@kkHuang-amd @sogalin
Unify sglang coding style (sgl-project#2856)
42f3909 
Co-authored-by: Lin, Soga <[email protected]>
@zhyncs
fix: not delete CNAME (sgl-project#2860)
20a9f5d
@zhyncs
docs: update link (sgl-project#2857)
41d7e5b
@zhyncs
minor: use ubuntu-latest instead of self-hosted runner for amd build (s…
4536d72 
…gl-project#2861)
@merrymercy
Use only one GPU for MLA CI tests (sgl-project#2858)
67008f4
@merrymercy
Collect more metrics: num_requests_total (sgl-project#2859)
51ab3cc
@bjmsong @mdattack
Integration of TurboMind AWQ (sgl-project#2828)
17de02f 
Co-authored-by: root <[email protected]>
@ispobock
Fix quant kernel accuracy issue (sgl-project#2865)
f3516c2
HandH1998 and others added 20 commits January 24, 2025 09:47
@HandH1998
Merge remote-tracking branch 'origin/main' into main_w8a8_fp8
4a98c75
@HandH1998
fix bug
8c3dc13
@zhyncs
minor: sync flashinfer and add turbomind as 3rdparty (sgl-project#3105)
153b414
@trevor-m
Allow local cutlass directory to be used in sgl-kernel build (sgl-pro…
685a573 
…ject#3037)
@adarshxs
[Docs] minor update for phi-3 and phi-4 (sgl-project#3096)
4505a43
@zhyncs
minor: update sgl-kernel setup (sgl-project#3107)
04f0b4c
@ispobock
Add workflow for sgl-kernel cu118 release (sgl-project#3109)
a22f60a
@ispobock
Add step to update sgl-kernel whl index (sgl-project#3110)
665e5e8
@BBuf
support fp32 in sampling_scaling_penalties kernel (sgl-project#3121)
5d9d15e
@yizhang2077
mirror fix for custom allreduce (sgl-project#3124)
9852214
@zhyncs
chore: bump v0.0.2.post17 for sgl-kernel (sgl-project#3125)
14e754a
@zhyncs
speedup pr test for sgl-kernel (sgl-project#3126)
3cab5f7
@ispobock
Update tag name for whl release (sgl-project#3127)
67ad433
@ispobock
Update whl index path (sgl-project#3128)
c23d570
@zhyncs
update installation doc for sgl-kernel (sgl-project#3129)
896c074
@yinfan98 @zhyncs
feat: refactor sgl-kernel and use TORCH_LIBRARY instead of PYBIND11_M…
9286740 
…ODULE for custom ops (sgl-project#3130)

Co-authored-by: yinfan.1024 <[email protected]>
Co-authored-by: yinfan98 <[email protected]>
Co-authored-by: Yineng Zhang <[email protected]>
@merrymercy
Fix CI tests (sgl-project#3132)
da6f808
@merrymercy
Use torch.compile for scaling penalty (sgl-project#3133)
27acf63
@hliuca
enable kv_scale for Gemma2 (sgl-project#3113)
8e48ca8
@HandH1998
Merge remote-tracking branch 'origin/main' into main_w8a8_fp8
a1b582e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.