sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- Fix embedding overrides for single embedding inputs
- [Fix] Only log "Enable Aiter AllReduce Fusion" when the flag is actually set
- [mem_cache] Add per-request skip_cache_insert and stop deriving it from the PD fake bootstrap host
- Fix MLX shutdown test mock after request-ingestion refactor
- [Hardening] Log why the fused all-reduce + RMSNorm path falls back instead of discarding it
- feat(sgl-router): complete the DeepSeek-V4 chat encoder and forward input_ids for dsv4 traffic
- `DUAL_STREAM_TOKEN_THRESHOLD` makes the DSA indexer's dual-stream gate unsatisfiable on ROCm — the leftover #14337's TODO named
- [ROCm] Enable AITER ASM MLA decode with EAGLE + CUDA graphs
- Fix FlashInfer full-prefill SWA graph buffer lifetime
- Fix per-token top logprobs in multi-token chat streaming chunks
- Docs
- Python not yet supported