sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- feat(moe): integrate FlashInfer BF16 MegaMoE for DeepSeek-V3
- fix(openai): honor named tool_choice in the Kimi K3 encode path
- [Fix] Parse Hunyuan tool arguments with JSON Schema type arrays
- [Feature] Breakable prefill CUDA graphs under decode context parallelism
- fix(kv-events): retry replay batches under backpressure
- [NPU][DSV4.1] Adapt NPU with DeepSeek V4.1
- [PD] Batch the decode-side KV preallocation and stage its lengths through pinned memory
- [PD] Harden deferred decode KV release: keep the drain ack, defer all failures, hold prefill source pages
- [Refactor] Separate V4.1 indexer capture options from candidate state
- [ROCm] Fix HiCache kernel backend GPU fault by using hipHostMalloc for host KV pool
- Docs
- Python not yet supported