sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Example] Demonstrate CPU prefix reuse with full-response metrics
- [UMBP] Support MiniMax-M3 sparse KV in the direct external linker
- [Fix] sgl-router: Send disagg_prefill_dp_rank to decode under --dp-aware
- [model-gateway] Send worker API key during initial HTTP health check
- [Fix] sgl-router: Enable the circuit breaker only with --cb-threshold
- [Fix] deterministic: Align chunked prefill to the Gated DeltaNet block size
- [Fix] sgl-router: Retry a send once when a pooled connection was closed
- [HiCache] Fix and document HF3FS benchmarks
- (draft) [Fix] [AMD] quark: Load FP8 shared experts into MXFP4 fused MoE weights
- [PD] Skip the cached-prefix early send in batches that start a HiCache load-back
- Docs
- Python not yet supported