sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- Supports vdn-minimax-h3 on the NPU.
- [Feature] Validate empty and non-mapping diffusion config files
- [Fix] kt: pass the GPU expert mask that kt-kernel's inference API takes
- [Feat] model-loader: make the unbalanced load barrier timeout configurable
- [Fix] deepseek-v4: FP8 quantize-dequantize the attention output before wo_a for V4.1-Flash-1017
- [Bug] Streaming gpt-oss Responses report zero token usage
- [Fix] openai: Track usage for streaming Harmony Responses
- [Fix] kt: run with zero GPU experts (--kt-num-gpu-experts 0)
- [RFC]: Programmable KV Cache: Composable Policies for Agentic Serving
- [Perf] [AMD] dsv41: Emit wo_b's MXFP8 input from the TP4 wo_a GEMM for verify rows on gfx950
- Docs
- Python not yet supported