sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] [NPU] DeepSeek-V4: reduced-layer MTP drafts, C128 L3 residency, MemCache anchor
- [Fix] pp: Post the first rank's output recv after its own forward
- [Perf] cosmos3: Accelerate conditioning hashing with GPU BLAKE3
- [Feat] unified_cache: [WIP] Support PP prefill and DSA PD decode in the Mooncake external linker
- [Fix] rpd_utils: Use is not None in the start time filter
- [Feat] [AMD] mono kernels: Register specs by model type
- [Perf] attention: Enable tensor descriptors in the Triton extend-attention kernel
- [Fix] function_call: Normalize GLM non-streaming tool names
- [AMD] vision: Batch ViT across requests on ROCm when the DP encoder is enabled
- [Fix] [AMD] diffusion: materialize SDPA operands for long sequences
- Docs
- Python not yet supported