sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] multimodal: Let a token-space strategy load its own media and choose its image decoder and RGB conversion
- [Perf] [NVIDIA] dsv41: run MegaAttention's wo_a on FlashInfer's SM107 batched MXFP8 GEMM
- [Feature] Share a prompt's KV among requests admitted in the same prefill batch
- [Bug] [AMD] --enable-deterministic-inference fails at startup on RDNA: batch-invariant matmul needs 99 KB LDS (64 KB available)
- [Fix] quantization: [sglang-miles] FP4-dequantized MoE experts report their 128x128 blocks to the runners
- [Fix] moe: [sglang-miles] DeepGEMM masked activation quant takes any group count with FP32 scales
- [Chore] dspark: [sglang-miles] Backport the per-architecture DSpark DP a2a allowlist from main (#48547)
- [Feat] deepseek_v4: [sglang-miles] DSpark over DeepEP under attention DP
- [Fix] disaggregation: Preserve unconsumed abort-ack poison across room reopen
- [Fix] io_struct: Preserve video_config when slicing requests
- Docs
- Python not yet supported