sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [NPU][dLLM] Stop materializing fp32 [tokens, vocab] logits in the denoise step
- [Bugfix] Fix DeepSeek-V4 non-EP TBO for attention TP > 1
- [Bug] DeepSeek-V4 (dsv4 backend + DSPARK) TP=8 on 8×H20: decode forward hangs indefinitely at ~245K context — all GPUs spin at 100% util / low power, watchdog kills server
- [AMD] [GLM5] Fuse the ROCm DSA indexer q/k prep into aiter's single kernel
- [Kernel] Add SM120 per-tensor FP8 GEMM for small-M (#33632)
- [BugFix][DP] Prevent health checks from perturbing user request routing
- Feat[DCP] Support decode context parallelism for DSA models with the flashmla_kv backend
- [AMD][PD] enable mori transfer for PD decode DCP
- [SenseNova-U1] Feature & Performance Tracking
- [HiSparse] Add speculative decoding support with multi-step KV cache swapping
- Docs
- Python not yet supported