sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- Pcp rebase
- [Perf] Avoid RadixKey suffix copies in cache walks
- [Fix][Profiler] Count target verify as decode stage
- [Feature] Support Custom OTLP Trace Service Name
- [docker] cu13 build fails with ResolutionImpossible installing runai-model-streamer: torch pins nvidia-nccl-cu13==2.29.7 but constraints.txt carries SGL_NCCL_VERSION (2.30.7)
- [Bug] Seeded sampling changes token when a batchmate enables top-k/top-p
- [Bug] `sharded_state` cannot save and load an MLA model
- [Feature] Support for same GPU native DP replicas with optional CUDA MPS
- [RFC][NPU] Ascend Cross-Generation Device Adaptation
- perf: cut device syncs and redundant launches from decode scheduling
- Docs
- Python not yet supported