sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] kv-cache: Let platform plugins register KV cache dtypes
- [Fix] [AMD] mla: Bound absorb-BMM wave budget (follow-up to #46918)
- [Test] [NPU] npu_memcache: Add L3 hit-rate checks and MetaService bootstrap
- [Doc] hicache: Document buffer_only anchor-lock cap and DP-attention scaling of schedule conservativeness
- [Refactor] scheduler: Add the guards for aborting a request on receipt
- [Fix] lang: Handle empty options and NaN/None token logprobs in `ChoicesSamplingMethod`
- [NPU] adapt Qwen3.8-27B-w8a8 mtp.fc quantazition
- [Feat] [NVIDIA] moe: opt-in FlashInfer cuTile NVFP4 W4A4 runner for MiniMax-M3.1 on SM12x
- [Feat] [Intel] diffusion: Run ConvRot INT8 linears on native sgl-kernel-xpu ops
- [Fix] [Intel] docker/xpu: Build on Ubuntu 24.04 with the GPU stack and oneAPI from Intel OMIX 0.5.0
- Docs
- Python not yet supported