Main w8a8 fp8#2

Open

HandH1998 wants to merge 259 commits intoyychfrom main_w8a8_fp8

+17,447-3,959

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 9, 2025

[Doc]: Deepseek reference docs (sgl-project#2787 )
XiaotongJiang
authored

Commits on Jan 13, 2025

Commits on Jan 15, 2025

Commits on Jan 16, 2025

[kernel] MiniMax-Text-01 prefill lightning_attn with triton (sgl-project#2911 )
BBuf
authored
feat: patch linear base (sgl-project#2915 )
zhyncs
authored
fix setup for sgl kernel (sgl-project#2917 )
zhyncs
authored
minor: use bear for compilation database (sgl-project#2919 )
zhyncs
authored
Improve benchmark scripts and error message printing (sgl-project#2922 )
merrymercy
authored
fixed lm_head.weight error for quantized qwen (sgl-project#2910 )
RinRin-32
authored
add profiling to bench_one_batch script (sgl-project#2821 )
yundai424
authored
Simplify the process launch code in server.py (sgl-project#2923 )
merrymercy
authored
Add CI for sgl-kernel (sgl-project#2924 )
ispobock
authored
Support multi-node DP attention (sgl-project#2925 )

merrymercy
and
dhou-xai
authored
Update release-docker-amd.yml to run on amd docker runner. (sgl-project#2927 )
saienduri
authored
Improve type annotation and styles (sgl-project#2926 )
merrymercy
authored
[kernel] MiniMax-Text-01 decode lightning_attn with triton (sgl-project#2920 )
BBuf
authored
Docs: Update pull_request_template.md (sgl-project#2928 )
zhaochenyang20
authored
Fix zmq binding (sgl-project#2930 )

merrymercy
and
chunyuan-w
authored
[Frontend] Fix request length check and add option to disallow auto truncation in scheduler (sgl-project#2876 )
CatherineSue
authored

Commits on Jan 17, 2025

Commits on Jan 18, 2025

Commits on Jan 19, 2025

Commits on Jan 20, 2025

Commits on Jan 21, 2025

[Doc] Update doc of custom logit processor (sgl-project#3021 )
hongpeng-guo
authored
Fix flaky tests in test_programs.py (sgl-project#3022 )
merrymercy
authored
[EAGLE] Fix some boundary situation when retract reqs and req's max token = 1 (sgl-project#2939 )

josephydu
and
josephyou
authored
Enable Cohere2 Models (sgl-project#3018 )
hliuca
authored
Misc fixes for eagle (flush_cache, CPU overhead) (sgl-project#3014 )
merrymercy
committed
minor: update Makefile for sgl-kernel (sgl-project#3025 )
zhyncs
authored
upgrade torch version for sgl-kernel (sgl-project#3026 )
zhyncs
authored
fp8 dispatch change

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
clean code

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
fix

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
clean code

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
Add accuracy and latency tests of eagle into CI (sgl-project#3027 )
merrymercy
authored
feat: add flashinfer as 3rdparty and use rmsnorm as example (sgl-project#3033 )
zhyncs
authored
Support sm90 Int8 gemm (sgl-project#3035 )
ispobock
authored
fix pr-test-sgl-kernel (sgl-project#3036 )
zhyncs
authored

Commits on Jan 22, 2025

Add performance and accuracy test code for FP8 GEMM operations

yych0745
authored and
HandH1998
committed
support w8a8 fp8
HandH1998
committed
Use int64 as indices for set_kv_buffer (sgl-project#3039 )
merrymercy
authored
support bias
HandH1998
committed
fix compilation
HandH1998
committed
clean code

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
clean code

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
Merge pull request #6 from HandH1998/tmptmp
HandH1998
authored
Merge remote-tracking branch 'origin/main' into main_w8a8_fp8
HandH1998
committed
format
HandH1998
committed
format
HandH1998
committed
Fix sgl-kernel compile for sm80 (sgl-project#3046 )
ispobock
authored
Merge branch 'main' into main_w8a8_fp8
zhyncs
authored
upd
zhyncs
authored
update norm cu (sgl-project#3048 )
zhyncs
authored
sync the upstream updates of flashinfer (sgl-project#3051 )
zhyncs
authored
Merge branch 'main' into main_w8a8_fp8
zhyncs
authored
feat: integrate norm kernels into sgl-kernel (sgl-project#3052 )
zhyncs
authored
feat: integrate activation kernels into sgl-kernel (sgl-project#3053 )
zhyncs
authored
minor: update header and use pytest (sgl-project#3054 )
zhyncs
authored
feat: integrate bmm_fp8 kernel into sgl-kernel (sgl-project#3056 )
zhyncs
authored
fix rotary_embedding rope_scaling for phi (sgl-project#3055 )
sudo-root-ns
authored
add notice about flashinfer in sgl-kernel (sgl-project#3057 )
zhyncs
authored
disable custom allreduce on HIP (sgl-project#3058 )
hliuca
authored
[Doc] Update doc of profiling with PyTorch Profiler (sgl-project#3038 )
Fridge003
authored
Fix the FP8 E4M3 parsing offline scales failure bug (sgl-project#3045 )
sleepcoo
authored
Add some flags to allow sync token ids across TP ranks (sgl-project#3060 )
merrymercy
authored

Commits on Jan 23, 2025

Commits on Jan 24, 2025

Commits on Jan 25, 2025

Commits on Jan 26, 2025