sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] Accept pack-quantized (WNA16) lm_head in should_apply_lm_head_quant_method so DFLASH does not crash
- Fuse MiMo verification RoPE and KV cache writes
- [ROCm] Isolate HiCache HIP host allocation
- [Bug] [model-gateway] Removed worker gauges are overwritten by in-flight health checks and requests
- [model-gateway] Fix removed worker gauges being overwritten by in-flight work
- [MPS][MLX] Capture shared logits processing in the exported region
- [Bugfix] Do not finish on grammar termination when ignore_eos is set
- [DSv4.1][SM120] Speed up decode/verify WO-A by reading an exact FP8 copy of the weight
- [AMD] [GLM5] Fuse mHC boundaries and expose FP8 prequant on gfx950
- [Bug] dots and Muse Glimmer detectors rewrite string-typed argument values (null becomes None, JSON-looking strings coerced)
- Docs
- Python not yet supported