sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [AMD] Add FlyDSL GDN prefill backend
- [Bug] Match OpenAI tool schema field order and omission semantics for `strict` and `defer_loading`
- [feat] host-staged all-reduce for two GPUs without peer access
- [Tracing] Merge async trace flushes across scheduler batches
- [Tracking] Pipeline parallelism x speculative decoding (EAGLE/MTP): converge prefill gate PRs, make the path model-agnostic, decide decode design
- fix(bench_serving): record flushed_cache state in benchmark results
- [Bug] Apple Metal (mps) reports no device memory capacity, so mem_fraction_static falls back to 0.95 and prefill OOMs
- DSV4.1 PD Road Map
- [BugFix] Fix Janus-Pro CLIPVisionTower Normalize shadowing and mean/std normalize
- [BugFix] Reset send_token_offset on input_embeds retract
- Docs
- Python not yet supported