sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] [AMD] minimax: PD with a pipeline-parallel prefill and a different decode attention TP (Mooncake)
- [Perf] diffusion/cache: Hash CUDA conditioning tensors on the GPU with keyed NH
- [Fix] tokenizer_manager: Validate token_indices_to_pool at request entry
- [Fix] sampling: Parse the UOCR page-guard lookahead inside its error handling
- [Fix] server_args: Redact the HiCache storage backend config in server info
- [Fix] attention: Return zeros instead of NaN from merge_state_v2 when both states are empty
- [CI] [AMD] nightly: Pause the MI30x Qwen3.8-Flash-Next test until the gfx942 LDS fix
- [Feat] disaggregation: Add `mini_lb` to sglang so it runs without sglang-router
- [Feat] [AMD] minimax: DSpark on a pipeline-parallel PD prefill (PP4 x TP1 -> TP4)
- [Perf] kimi-k3.1: Route and quantize the SP tail's MoE input on each rank's rows
- Docs
- Python not yet supported