sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] hicache: Disable the NIXL L3 cleaner in the NIXL storage tests
- [Bug] tokenizer post-load fixes ignore `--revision` (`revision` passed, `tokenizer_revision` read)
- [Perf] hicache: Reuse the host eviction heap across one write_back eviction walk
- [Fix] bench: Size the one-batch KV-capacity skip by the heaviest attention-DP rank for ragged batches
- [Refactor] cuda-graph: Register DSpark verify epilogue buffers as declared graph buffers
- [Feat] lora: Coordinate routing reuse through DP scheduler metadata
- [Perf] [NVIDIA] gdn: Narrower value tile for the packed GDN decode on SM90
- [Fix] anthropic: attach inline system messages in place to preserve prefix cache
- [CI] [AMD] aiter: Update ROCm AITER pin to 89657640
- [Fix] layernorm: Keep weightless RMSNorm's weight as a buffer so it survives memory release
- Docs
- Python not yet supported