sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] unified_cache: Support MHA / QSA + Mamba models with PLE slot states in the direct external linker
- [Perf] GGUF/MoE: llama.cpp's MMQ (MUL_MAT_ID path) behind a new ggml_moe_mmq op
- [Bug] FLUX.3 Image: TP/SP ranks pick packed or sequential CFG independently and can hang
- [Fix] diffusion/flux3: Make TP and SP ranks agree on sequential CFG
- [Fix] cuda_graph: Recapture target prefill graphs after disk weight updates
- [Feat] mem_cache: Optionally zero KV pages on free
- [Fix] [CPU] topk: Bound sigmoid vectors for small expert counts
- [Fix] diffusion: Fix rollout conditioning and scalar weight checksums
- [Bug] diffusion: Rollout conditioning misalignment and scalar weight checksum failure
- [Feat] KVShard: opt-in Rust tree backend with rotation-phase support
- Docs
- Python not yet supported