sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] mem_cache: Move FP4 MLA block scales with their KV in `move_kv_cache`
- [Fix] deterministic: Validate the per-phase attention backends
- Add DoSA prefill indexing and GLM-5.2 TTFT reproduction
- [Test] wan2.2-s2v: Add the 2-GPU server accuracy case
- [Feat] [NPU] GLM-4.1V: Add ModelSlim W8A8S support with multimodal serving optimizations
- [Feat] plugins: Add SGLANG_REQUIRED_PLUGINS for fail-fast loading
- [Report] DeepSeek-V4.1-Flash on 4x SM120 (83 GiB): 1M context + DSpark works on v0.5.21, setup notes and numbers
- [Perf] hisparse: Preload the PD decode device buffer in one all-layer launch
- [Feat] KVShard: Enable KV cache sharding for Qwen3.8 Flash Next
- [Fix] [NVIDIA] allreduce: Stage VMM graph inputs in CustomAllreduce instead of IPC-registering them
- Docs
- Python not yet supported