sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] profile: Allow profile_by_stage without num_steps
- [Fix] nemotron: Accept transformers 5 layer type names
- [MiniMax-M3] Keep the fused q/k/v + index GEMM writable by online weight updates
- [Fix] disaggregation: Send aux_item_lens next to aux_ptrs in PD registration
- [Perf] diffusion: keep GQA and MOSS-Image attention on cuDNN SDPA on Blackwell
- [Doc] diffusion: Set measured cookbook defaults for MiniMax-H3, FastH3 Omni, MOSS-Image and Qwen-Image-2.1
- [Refactor] hicache: agree storage hit queries on the scheduler thread
- [Perf] dspark: Refresh Mamba tracking once per verification step
- [Feat] [NPU] eplb: Enable dynamic expert load balancing on Ascend
- [Fix] diffusion: Fail fast on offline runtime failures
- Docs
- Python not yet supported