sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bugfix] Fix repetition penalty updates under overlap scheduling
- [Metrics] Expose multi-detokenizer router CPU usage
- [Bug] EP standard dispatch: -1 sentinel aliases to last local expert, padded spec-verify rows overflow masked-GEMM per-expert m_max (OOB stores, engine crash)
- [unified-memory] Admit asymmetric K/V rows (MiMo-V2-Flash)
- [unified-memory] Fuse MTP drafts with sliding-window and recurrent-state layers
- fix(moe): keep the -1 drop sentinel out of EP local-expert mapping (masked-GEMM per-expert overflow, engine crash)
- [HiCache] Keep unified memory off the direct page_first_direct copies
- [AITER] Give pa_decode_gluon a page table when the KV layout is SHUFFLE 5D
- [diffusion] share the layerwise pinned weight stores across co-resident instances
- [CPU] enable grok-2 model on xeon
- Docs
- Python not yet supported