sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [HiCache] Fix stale host KV from write-through copies racing the in-flight forward
- [PD] Fix retraction restores racing the in-flight decode forward
- [Radix Cache] Optionally keep an SWA margin beyond the window in cached prompts
- [cake_kernels] SP all-gather matmul route: torch's default symmetric-memory backend, prepared launcher inside prefill CUDA-graph capture (stacked on #42532)
- [DSpark] Keep output sampling exact with folded draft sampling by storing corrected logits in FP32
- [DSpark] Produce correct SPS cost tables when profiling a server without a table
- [DSpark] Calibrate STS temperatures from the drafts each step actually accepted
- [Spec] Make DeepSeek-V4 compact-verify replays find their own tier's metadata under SGLANG_TEST_RAGGED_VERIFY_FORCE_UNIFORM_CAPTURE
- [SM120] Stop DeepSeek-V4 CUDA graphs from writing into freed FlashMLA page-split buffers
- [DSv4.1] Cut encoder SWA bounded-replay window copies from ~4.2 ms to ~0.1 ms per decode step
- Docs
- Python not yet supported