sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] DeepSeek-V4-Flash: speculative and ordinary decoding produce different greedy outputs
- [RFC] PD disaggregation: send the last KV chunk from later PP stages as soon as their forward finishes
- [Feat] step3-vl: Add Rust image preprocessing
- [Performance][DCP] Prefix-hit prefill pays 4*batch_size device syncs per layer when gathering KV for MHA extend
- [Feat] DSpark: Add opt-in online latency cost correction
- [Perf] dcp: Vectorize the per-request reorder in the MHA extend KV gather
- [Perf] [SM90] dsv4.1: Add fused FP8 prefill indexer scoring
- [Fix] sampling: Add upper bounds for top_k, top_logprobs_num and n
- [Perf] runtime: Tune FlashInfer MoE tactics on distinct dummy tokens
- [Perf] kimi-k3: Rank-local expert ids for the EP routed MoE
- Docs
- Python not yet supported