sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] hicache: Evict pass-through Mamba host checkpoints before path tails
- [Fix] sampling: Validate n before parallel request expansion
- [Feat] [AMD] kda: Add a gfx950 TLX KDA backend
- Anmolgupt/embedding serving perf
- [Refactor] multimodal: Let token-space strategies declare media decoding and the video device
- [Distributed] Keep deprecated parallel_state getters traceable under torch.compile
- [DeepSeek] Use the out-of-place MLA value projection while torch.compile is tracing
- [Mem Cache] Free SWA pages with the Triton kernel on Triton-capable out-of-tree platforms
- [Model Executor] Return the input from ForwardBatch._pad_tensor_to_size when no padding is needed
- [Grammar] Emit compact JSON when --constrained-json-disable-any-whitespace is set (xgrammar)
- Docs
- Python not yet supported