sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] disaggregation: Let PD decode servers prefill requests marked disagg_local_prefill
- [Perf] mem_cache: Evict a running request's prompt-end SWA window last (opt-in)
- [Fix] disaggregation: Transfer the draft's DSA indexer cache for hybrid DSA models
- [Fix] true-on-policy: Cast Qwen2/Qwen3 dense inputs to params_dtype, not the quantized weight dtype
- [Fix] [AMD] moe: Support SiLU-gated MXFP8 MoE on gfx950
- [Fix] [AMD] imports: Skip import-time device probes when no GPU is visible
- [Perf] prefill-cuda-graph: Replay the prefill graph for 1-token extends so full prefix-cache hits are no longer slower than cold requests
- [Perf] [NVIDIA] minimax-m3: Score EAGLE verify rows in one pass over the index K
- [Perf] minimax-m3: Split the score-only prefill indexer over the KV range
- [Fix] openai: Support long prompt encoding with fastokens
- Docs
- Python not yet supported