sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] scheduler: Add opt-in aging to the shortest-prefill-first policy
- [Perf] hicache: Keep the buffer load-back ownership check from waiting on the GPU at admission
- [Feat] scheduler: in-process output sink
- [Refactor] moe: Unify MoE router GEMM behind shared RouterGate layer
- [Fix] disaggregation: Keep the KV budget abort for PD prefill requests
- [Doc] diffusion: Fix cookbook commands, model IDs and catalog entries
- [Fix] processor: Preserve reasoning content in Rust renderers
- [Doc] diffusion: Fix feature guides that disagree with the code
- [Feature] Share GPU memory between the KV pool and the mamba/linear-attention state pool instead of a static split
- [Fix] diffusion: Bind disaggregation listeners before worker connections
- Docs
- Python not yet supported