sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- fix(parser): prefer native token id for think-end markers
- [Feat] tbo: Add opt-in threshold-based admission policy
- [Feat] diffusion: add Physis H3 causal pipelines
- [Diffusion] Fuse SANA-Video depthwise convolution and GLU on SM120
- [Fix] unified_cache: Restore Inkling MTP draft state in the direct external linker
- [Bug] /start_profile with profile_by_stage=true and no num_steps crashes every scheduler (TypeError: NoneType += int)
- [Perf] cuda-graph: Upload each deduplicated executable on a private stream before its first launch
- [Fix] [AMD] diffusion: detect ROCm through torch when amdsmi is unavailable
- [Perf] scheduler: Add opt-in stop-string tail reuse and prefix pruning
- [Doc] [AMD] glm-5.2: PD disaggregation on MI355X with the Infera router
- Docs
- Python not yet supported