sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [AMD] feat: Add opt-in AITER FlashKDA prefill for Kimi-K3
- [Feature] SRT plugins: opt-in fail-fast for required plugins and hooks (`SGLANG_REQUIRED_PLUGINS`)
- [Mamba] fix: Keep chunked prefill cuts on the mamba chunk grid
- [DCP] perf: Slice gathered MHA prefixes with host offsets
- [Feature] Let a general plugin provide the weight offloader, with a post-load step that receives the loaded model
- [DCP] Validate DCP nesting at launch; reject non-bf16/fp16 KV under NPU DCP
- [Feature] Public, non-fatal way for a model or plugin to refuse online weight updates, also honoured by `begin_weight_update`
- [Bug] HermesDetector reorders streamed normal text across a tool call
- [diffusion] model: add native Ovis-Image support
- [Bug] Qwen3CoderDetector silently flips boolean parameters to `false` when the value has surrounding whitespace
- Docs
- Python not yet supported