sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] qwen2.5-vl: use per-frame cu_seqlens for video full attention
- [Fix] openai: Return required and named tool calls without a tool-call parser
- [Bug] Hybrid Mamba model + EAGLE: KV-cache sizing error says to raise --mem-fraction-static when the Mamba state cache is what fills memory
- [Feat] diffusion: support BRIA Fibo 2.0 generation and editing
- [Perf] R3: Add --routed-experts-dtype int16 to halve the routed-expert host cache
- [Fix] mem_cache: broadcast packed draft layers in the MLA dedup linker
- [Feat] [AMD] hicache: Add --hicache-mamba-size-gb to pin the HiCache Mamba host pool size
- [Perf] radix_cache: page-stride prefix matching and bigram boundary preservation
- [Fix] http_server: Start HTTP workers with lazy CUDA module loading
- [Fix] diffusion: Enable Cosmos 3 Edge and Nano fused QK-norm/RoPE on SM 121 (GB10)
- Docs
- Python not yet supported