sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] cuda-graph: Keep deepstack media prefills eager
- [Feat] rust-server: add direct scheduler ingress (PoC)
- [Fix] mem_cache: Fix size_per_token to be per-token for DeepSeek V4 host pā¦
- [Perf] [NVIDIA] KDA: Extend the CuTe DSL MTP verify kernel to SM90
- [CI] tests: Give each NIXL storage test its own temp directory
- [Perf] NIXL: KV packing for TP-heterogeneous transfer (Muse Spark Open)
- [sglang-miles] LoRA/MoE/KDA: decode fast paths for colocated LoRA RL rollout
- [Feat] [XPU] quant: Add MXFP8 W8A8 MOE compressed-tensors
- [Perf] [AMD] GLM5: Add opt-in W8A16 KDA decode projections on gfx950
- [Fix] benchmark: Reject incomplete OpenAI streams
- Docs
- Python not yet supported