sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] qwen38flashnext cookbook image predates #38851: start-up crash in the QSA KV-extraction kernel once max_running_requests reaches ~4000 (e.g. with --disable-radix-cache)
- [AMD] [Fix] Handle unaligned QSA extend prefixes from chunk-cache tails
- [Refactor][1/2] Migrate FFN registration and checkpoint loading
- [sglang-miles] Cherry-pick #37740: keep deterministic inference off the Lean path
- [MoE] Add opt-in Hopper W4AFP8 block-scale DeepEP LL prototype
- [Diffusion][SenseNova] Add layerwise offload support
- [DSV4][CP] Support CP and multistream forward-prepare on DSV4
- [dLLM] Preserve prompt tails and literal mask tokens across prefix reuse
- [Kimi-K3] Drop tool markup without declared tools
- [Benchmark] Export client-side streaming timelines as Chrome traces
- Docs
- Python not yet supported