sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] DSv4.1: Size the encoder-replay SWA pool by the DSpark draft's stages, and keep more prefix tails
- [DSv4.1] Support encoder and decoder SWA bounded replay under DP attention on CUDA
- [Fix] Merge request stop with a str or tuple template stop_str
- [Fix] Preserve NVFP4 writer scales for shared KV layers
- [Fix] Load compressed-tensors NVFP4 KV calibration with matched cache format
- [Feat] sgl-router: Add --dp-rank-policy round_robin for --dp-aware
- [CPU] Add mamba_chunk_scan_combined and selective_state_update kernel on Xeon
- [Bug] DFlash + Qwen3.8-27B: speculative decoding accept rate drops to 0 during serving
- [ROCm][Perf] Qwen3.8 mono decode: two persistent FlyDSL launches per layer on MI355X
- [NPU] Add sampling-mask support for the Ascend sampling backend
- Docs
- Python not yet supported