sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] Compare bearer tokens as bytes so non-ASCII keys don't 500
- feat: support GLM-5 KDA prefill with breakable CUDA graphs
- [Bug] NGRAM cached requests miss newly inserted longer contexts
- Fix NGRAM cached misses after trie growth
- [KV cache] Make KV-copy tiling configurable by subclasses
- Insert launched extend batches into the radix cache before the next batch under overlap
- [Diffusion][Perf] Reduce indexing overhead in Ulysses QKV packing
- [Feature] Skip local vision encoder init for qwen3 in language-only EPD
- [Perf] Skip TP all-reduce of sampled token ids when world size is 1
- [Feature] Share target embed/lm_head with the Qwen3.5 MTP draft
- Docs
- Python not yet supported