sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] spec: Select only tokens with probability in chain rejection sampling's final draw
- [Perf] radix-tree: Cap Rust tree-core match keys with a memoryview instead of a slice
- [Feat] EPLB: Speed up EPLB balanced packing with a bounded min-heap
- [Feat] mem_cache: Support INT8 Mamba checkpoints in HiCache L1/L2
- [Perf] rerank: Cache compiled Jinja templates
- [Feat] [NPU] dsa: complete A2 packed INT8 KV integration
- [Bug] DeepSeek V4.1: natural EOS with unclosed reasoning produces a successful empty answer
- [Feat] [NPU] dsa: add graph-safe A2 INT8 indexer support
- [Fix] benchmark: Correct single-turn token counts and native stream failures
- [Feat] EPLB: Build only the local rank EPLB static dispatch map
- Docs
- Python not yet supported