sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] [NPU] memory-saver: Drain device before sleep unmap and preserve runtime tensor attrs
- [Perf] gemm: Opt-in table of pinned bit-identical cuBLASLt algorithms for BF16 linears
- [Perf] [NVIDIA] qsa: split-K sparse decode kernel for fp8 KV
- [Fix] [AMD] dsa: One decode graph variant on every attention-DP rank, one verify graph
- [Perf] memory: Expand single-GPU decode graph coverage
- [CI] diffusion: perf checks depend on which runner a job lands on, and in-job retries can't fix a slow host
- [Perf] logprobs: Pack readback and cache cumulative stream encoding
- [Feat] hicache: Add a direct_kernel IO backend on page_row for MLA/DSA pools
- [Fix] [XPU] gdn: Fix fork stride and make tests device-agnostic
- [Feat] qsa: host offload of the QSA K/V with a per-request GPU hot cache (1/N)
- Docs
- Python not yet supported