sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- feat: expose per-request speculative decoding stats
- Fix integer created_at in Responses streaming events
- Confine WASM module loading to configured roots
- [AMD] Fix AITER page-size-1 graph metadata
- [Auto Tune] Add allreduce algorithm auto-tuner (#13363 item 4)
- [Auto Tune] Add speculative decoding parameter auto-tuner (#13363 item 7)
- [Auto Tune] Add CI workflow for auto-tune on 20 models x 5 platforms (#13363 item 3)
- [HiSparse] Fix copy_cache_planned_kernel grid sizing for prefetch replay
- Fix PD streaming load tracking before dispatch
- [Fix] Bound the incremental detokenization window on a U+FFFD stream
- Docs
- Python not yet supported