sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [PD] Send prefill KV chunks as soon as the batch result copy completes
- [DSV4.1] Support encoder SWA bounded replay with CUDA DP attention
- [NPU] Support host-pool backup for PD decode retraction
- [AMD] Use the AITER CK blockscale GEMM by default for gfx942 block-FP8 linears
- [Perf] Avoid dtype conversion in decode page count
- [Feat] [AMD] dsv4: Support interleave prefill CP with breakable CUDA graphs
- [RFC] Prefill sequence parallelism with replicated MoE for small-TP long-prompt prefill (Qwen3.5 MoE)
- [AMD] MoE sort patch: fuse the stage-1 MXFP8 quant at hidden 5120 and sort up to 385 experts
- [diffusion] VSA-H3: support sm80 (A100) with a raw-pointer Triton load path
- [NPU] Pin CPU-offloaded weights for host-to-device copies
- Docs
- Python not yet supported