sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] deterministic: Size the MQA-logits chunks without free memory
- [Feat] [AMD] model: Add Kimi-K3.1 MI35x support
- [Fix] batch_invariant_ops: Cap the mean kernel's grid so large outputs launch on ROCm
- [Fix] [Intel] diffusion: Skip XPU component prefetch that would not fit
- [Perf] dp-attention: Skip the dead H2D copy of global_num_tokens_gpu
- `HybridLinearKVPool` ignores `v_head_dim` for the full-attention MHA pool (hybrid linear-attention models with V head dim != K head dim)
- Allow models outside sglang's architecture list to use the MLA KV cache (detect MLA from config, not architecture name)
- Feature request: overlap scheduling for externally registered linear-attention models using the mamba radix cache
- [Feature] Metric for time to first answer token (after reasoning)
- [Feat] kimi-k31: Support PCP KV sharding with HiCache and DFlash
- Docs
- Python not yet supported