sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] openai: keep explicit thinking aliases consistent
- [Fix] model_loader: Read each shard sequentially before the multi-thread loader maps it
- [CI] base-b: Move kernel and unit tests that pass on the 5090 to 1-gpu-small
- [Perf] mem_cache: Replace O(N) list.pop(0) with deque in StagingAllocator, FlexKVComm, and LMCacheUnifiedRadixCache
- [Feat] rust-server: Fan control requests out to every DP rank
- [Doc] serving: show optional shared admission for background workers
- [MLX] Add opt-in memory diagnostics and document the buffer cache cap
- [Feat] [NPU] eplb: enable dynamic expert load balancing on Ascend
- [Fix] disaggregation: Key decode-offloaded KV pages by bigrams under EAGLE
- [Perf] diffusion: Project Wan's cross-attention text K/V once per denoising loop
- Docs
- Python not yet supported