sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] srt: bound FanOutCommunicator queueing calls with a reply deadline
- [sglang-miles] [Diffusion] Return the negative cond from a rollout that runs no CFG
- [Fix] disaggregation: wait for the prefill forward before sending KV chunks under PP
- [Bug] DeepSeek-V4-Flash: speculative and ordinary decoding produce different greedy outputs
- [RFC] PD disaggregation: send the last KV chunk from later PP stages as soon as their forward finishes
- [Feat] step3-vl: Add Rust image preprocessing
- [Performance][DCP] Prefix-hit prefill pays 4*batch_size device syncs per layer when gathering KV for MHA extend
- [Feat] DSpark: Add opt-in online latency cost correction
- [Perf] dcp: Vectorize the per-request reorder in the MHA extend KV gather
- [Perf] [SM90] dsv4.1: Add fused FP8 prefill indexer scoring
- Docs
- Python not yet supported