sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] Trim DP-padded query rows in the FlashInfer prefill path
- [SM120] Speed up TP4 logits and DSpark vocab all-gathers on PCIe-only hosts
- [DSv4.1] Speed up decode-size mHC mixing with a vectorized split-K reduce and 160 K slices on SM120
- [SM120] Speed up decode-sized MXFP8 linears with a Triton GEMM
- [Perf] Opt-in batch-invariant BF16 LM head GEMM for decode-sized batches
- [LoRA 02/21] Shared token routes and the LoRA workspace
- [LoRA 03/21] Grouped and per-row LoRA shrink and expand kernels
- [LoRA 04/21] Dense execution plans
- [LoRA 05/21] Shipped dense plan tables for SM90 and SM100
- [LoRA 06/21] Dense LoRA runner and the triton_v2 backend
- Docs
- Python not yet supported