sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] scheduler: don't crash with --max-queued-requests 0 and priority scheduling
- [Perf] FP8 decode: Hopper small-M GEMV, fused KDA output quant, CUDA grouped top-k, and partial-block FP8 tails
- [Perf] diffusion: Avoid full GC per tensor update
- [Perf] dsa: pass tiled decode page_table_row_stride at runtime
- [Feat] deepseek-v4.1: the V4.1 KV cache layouts on SM120 / SM121
- [Chore][NVIDIA] Update to torch 2.15rc1 in cuda 13.4 container
- [CI] [AMD] test: Skip Mistral-7B and Mixtral-8x7B FP8 in nightly GSM8K eval
- [Feat] rust/grpc: Propagate the caller's trace context to scheduler spans
- [Perf] dsa: Run the packed prefill top-k on CUDA
- [Fix] scheduler: Make a Req's "__req__" self-reference weak
- Docs
- Python not yet supported