sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] disaggregation: Send decode transfer destinations to prefill only after the in-flight forward
- [Fix] unified-memory: Refuse an MXFP8 KV cache at boot
- [Fix] mla: Attend each draft step's own tokens on flashinfer_mla, flashmla and trtllm_mla
- [Fix] unified-memory: Keep the host's state slot count when a draft's state fuses into it
- [Feat] unified-memory: Fuse sliding-window draft layers on fa3
- [Feat] unified-memory: Admit fa4 for EAGLE and DFLASH
- [Feat] unified-memory: Admit DSPARK on trtllm_mha
- [Feat] unified-memory: Fuse MLA drafts into MLA hosts
- [Feat] unified-memory: Fuse MLA drafts on flashmla, trtllm_mla and cutedsl_mla
- [Feat] unified-memory: Keep a fused draft through HiCache
- Docs
- Python not yet supported