sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Perf] flashinfer: release the shared workspace when no backend uses it
- [Fix] models: Pass HF sliding windows to RadixAttention without the query
- [Perf] qsa: scale the indexer prefill scores inside the relative scoring kernel
- [Fix] modelopt: Keep NVFP4 MoE experts intact across partial weight updates
- [CI] [AMD] ci: Register CUDA base/extra tests on AMD (parity experiment, do not merge)
- [Perf] [NVIDIA] fp8_fa_sm120: Run cross-attention when K/V length differs from Q
- [Fix] [XPU] compressed-tensors: Support W8A8 int8 on XPU and quantize Whisper MLP linears.
- [Feat] sail: support PLA recurrent linear attention fast paths [5/N]
- [Fix] scheduler/multimodal: Fix embedding over-length hang, cap CPU pool, fix ST6 reference
- [Feat] qsa: host offload of the MTP draft model's QSA K/V (3/N)
- Docs
- Python not yet supported