sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] Mask unused ReplaySSM history before checkpoint arithmetic
- [Fix] Preserve BF16 NaNs in per-token FP8 quantization
- [Quantization] Skip MTP online NVFP4 quantization by default
- perf(kimi-k3): fold BS1 shared-expert all-reduce into the deferred-finalize push
- [Fix] Keep cancelled control calls from consuming later replies
- [Fix] hicache: give a rank-sharded L3 sidecar its own key on a replicated-KV model
- [Fix] hicache: clear the storage backend when a weight update flushes the cache
- [Cake] kimi_k3_fp8_projection: per-weight cached launcher in the MLA projection route
- [WIP][AMD] rocm10.1
- [HiCache] Fix SWA over-admission of host load-backs that crashes the scheduler
- Docs
- Python not yet supported