sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] Capture routed experts across MoE top-k paths
- [Feat] unified_memory: Support mxfp8 KV cache on the unified memory pool
- [Bug] --speculative-token-map crashes at TP>1 (device-side assert in init_lm_head): global hot-token ids index a vocab-sharded lm_head
- Fix: [Bug] expert-distribution endpoints terminate the scheduler when expert_distribution_recorder_mode is unset
- [Qwen4-Exp] Skip the PLE backing-file rewrite when the on-disk table already matches
- [Fix] Sync the grammar before mixed steps under speculative decoding
- [Fix] Triton SWA decode: read the full sliding window
- [Perf] Skip cached-prefix queries in torch-native attention
- [cake_kernels] opt-in model routes: Qwen3.5 GDN prefill/decode, FP8 grouped MoE, NVFP4 warp-decode MoE (SGLANG_CAKE_ROUTES) + Qwen3.5 e2e report
- [cake_kernels] opt-in Kimi-K3 routes: KDA recurrent decode, FP8 projections, Cake MLA (SGLANG_CAKE_ROUTES) + Kimi-K3 e2e report
- Docs
- Python not yet supported