sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [SMG] Backlog-aware prefill selection for cache_aware in PD mode
- [SMG] Clear cache-aware trees for workers flushed by /flush_cache
- [KV events] Publish opaque LoRA block IDs and adapter names
- [Feat] sgl-router: Separate LoRA block IDs from prompt lookup keys
- [DeepGEMM] Pre-compile FP8 einsum to avoid cold-start JIT stalls
- Add opt-in error for greedy fallback in speculative verification
- [Intel][XPU] Allow mamba extra_buffer radix cache strategy on XPU
- [gRPC] Support tokenized multimodal generation
- [cherrypick from #42889] [Perf] Serialize OpenAI chat/completion responses in one pass
- [AMD] Add an opt-in gfx1250 cooperative-cluster path for DeepSeek-V4 TopK v2
- Docs
- Python not yet supported