sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] macOS: shared-memory names can exceed the 31-char POSIX limit, deadlocking `in_the_same_node_as` with no error output
- feat: expose per-request speculative decoding stats
- Fix integer created_at in Responses streaming events
- Confine WASM module loading to configured roots
- [AMD] Fix AITER page-size-1 graph metadata
- [Auto Tune] Add allreduce algorithm auto-tuner (#13363 item 4)
- [Auto Tune] Add speculative decoding parameter auto-tuner (#13363 item 7)
- [Auto Tune] Add CI workflow for auto-tune on 20 models x 5 platforms (#13363 item 3)
- [MiniMax-M3] Triton sparse attention in full prefill CUDA Graph
- [PD] Restore the prefill-side bootstrap/alloc_wait split dropped in #26780
- Docs
- Python not yet supported