sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] deterministic: replay hybrid Mamba prefix instead of restoring donated state
- [Feat] hicache: Share request-end Mamba backups
- [Feat] diffusion: Add --model-checksum to verify model files before loading
- [Doc] cookbook: Refresh Qwen3.8-Flash-Next GB300 AgentX points
- Importing sglang replaces `triton.next_power_of_2` with a host-only function, breaking other Triton kernels in the process
- [Feat] observability: Add time to first answer token histogram
- [Fix] detokenizer: Freeze GC in the output router
- [RFC] scheduler: UniBoost, a starvation-bounded schedule policy (#48076-#48079)
- [Feat] disaggregation: Support Qwen3.5 and Qwen3.8 MTP with NIXL CPU buffers
- [Fix] kda: Allocate FlashInfer prefill-graph checkpoints in the SSM state dtype
- Docs
- Python not yet supported