sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Rust] Reduce output chunk allocations
- [Metrics] Count KV bootstrap and transfer timeout observations
- [HiCache] Use attention-TP ownership for bounded CP file caches
- [Fix] Reject prompts that cannot reserve paged KV headroom
- [Perf][CPU] Defer token allocator free-list copies until allocation pressure
- [Tracing] Close filtered thread spans at explicit completion
- [Bugfix] Honor no_logs when logging requests
- [Bugfix] Zero the cuda-graph padded rows of the GDN/KDA recurrent output
- [AMD] Enable aiter gluon flashkda for Kimi-K3 KDA prefill
- [WIP]DeepEP Support for Intel XPU in SGLang
- Docs
- Python not yet supported