sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [NPU DSV4]Preserve DeepSeek-V4 SWA cap mode with model-default ratios
- [HiCache] Register layer-first host pools as whole-layer ranges
- [Perf][Multimodal] Bound Qwen-VL video decode batches by pixel budget
- [Fix] Handle overflowing TopK v2 threshold buckets exactly
- [MiniMax-M3] Fix EAGLE3 verification on CUDA
- [Kernel] Extend the SM100 swapab decode path to the V4.1 KV layouts
- [UMBP] Share one direct-linker key across CP ranks for DeepSeek-V4 pools
- Fix AutoTokenizer failure for granitemoe-family models with GPT2Tokenizer hub override
- [HiCache] Back up Mamba and SWA states on write_back eviction when full KV is already on host
- CI/CD Speed Up Optimizations
- Docs
- Python not yet supported