sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [NPU DSA prefill sharding 1/2] Indexer: shard prefill queries across attention TP
- [NPU DSA prefill sharding 2/2] Token shard: every head for this rank's slice of the tokens
- [MPS][MLX] Add batched prefill to the exported region
- [Bugfix] Let the GLM47 tool grammar follow always-thinking templates (GLM-5.3)
- [DeepSeek-V4.1] Restore the wide-prefill mHC stats overlap with the TP all-reduce
- [Test] rust: Add manual API v1 frontend E2E coverage
- [DeepSeek-V4.1] Run decoder SWA bounded replay under the breakable prefill CUDA graph
- [LoRA 01/21] Shared hooks for the LoRA MoE and dense engines
- Parallelize EAGLE spec-decode KV index building over token blocks (flashinfer draft decode; verify/draft-extend for flashinfer, flashinfer_mla, aiter)
- [Perf] Native MLA prefill under BCG for TRT-LLM and Tokenspeed
- Docs
- Python not yet supported