sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [DSV4.1] Add H20 block-FP8 tuned configs
- fix(bench): speed-bench counts 2 input tokens per prompt when the chat template returns BatchEncoding
- [Bugfix] Fix chunked-prefill infinite loop and KV leak for fake-prefi…
- [Scheduler] Fix PrefillDelayer all-gather buffer size under context parallelism
- [HiCache] Add capacity budget option to NIXL L3 cleaner
- [HiCache] Isolate L3 storage faults: keep storage threads alive, always ack, per-direction circuit breaker
- [DSv4][SM80] Support CP decode attention TP with FP8 Marlin linears
- [Feature] Qwen3.5forsequenceclassification model is supported
- Sampler: gather sampled-token logprobs before log
- Add tuned fused-MoE Triton config for E=128,N=768 bf16 on B300 SXM6 PC
- Docs
- Python not yet supported