sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- Kimi-K3.1 Issue Tracker
- [CI] ci: Stop the suite and exit when a timed-out test process cannot be killed
- [Bug] NextN/MTP drafts of ModelOpt NVFP4 checkpoints get no KV-cache scales, so the draft's FP8 KV cache runs at 1.0
- [Fix] attention: Order the fp8 paged MQA logits kernel's TMEM dealloc after every math warp's tcgen05.ld
- [Feat] [NVIDIA] speculative: add an optional recurrent checkpoint commit hook
- [Fix] test: Propagate deterministic prefix logprob failures
- [Fix] kda: Align unfused safe-gate diagonal normalization anchor
- [Bug] FlashInfer and Triton apply checkpoint KV-cache scales to a BF16 KV cache, so decode output is garbage (default flags for fp8/Quark FP8-KV checkpoints)
- [Fix] attention: do not apply KV-cache scales to a BF16 KV cache in FlashInfer and Triton
- [Fix] models: load FP8 KV-cache scales for GLM-4.5, Llama 4, HunYuan, HY3, EXAONE-MoE, LFM2 and the Qwen3 family
- Docs
- Python not yet supported