sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Refactor] custom-ar-v2: Share one symmetric workspace per group
- [Perf] [NPU] MiniMax-H3: optimize RMSNorm and SwiGLU
- [Fix] [NPU] attention: Run page_size<16 models on the reference attention path
- [Fix] docker: Exclude nested Python bytecode from Docker contexts
- [Feat] hicache: Add an independent draft host memory budget
- [Bug] Whisper: fc1/fc2 are built without the quantization config, so a quantized checkpoint transcribes nothing
- [Fix] whisper: Build fc1 and fc2 with the quantization config
- [Rust server] Tokenization runs on a single thread by default and becomes the tail-latency bottleneck for long prompts
- [Perf] rust-server: Size the tokenizer pool from the frontend cores
- [Perf] [NVIDIA] minimax-sparse: Gather the decode indexer CP queries and candidates with multimem on Hopper
- Docs
- Python not yet supported