sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Doc] [NPU] ascend: Add a knowledge base for SGLang deployment on Ascend NPUs
- [Doc] diffusion: Correct Wan video requests and document padding tradeoffs
- [Bug] `/v1/responses` streaming returns HTTP 200 + `response.failed` for context-length-exceeded requests, while `/v1/chat/completions` returns HTTP 400
- [Fix] metrics: Add MLA/MoE-aware FLOPs and fix forward_overlap device gate in the MFU estimator
- [Perf] speculative: Support zero-copy QLEN_MASK in FlashAttentionBackend for NGRAMWorker
- [Bug] AttributeError: 'QwenImagePipelineConfig' object has no attribute 'sample_sigmas'。
- [Perf] [AMD] diffusion: Quantize MXFP4 activations with the AITER HIP kernel
- [Perf] [AMD] dsa: Route the DCP prefill by its wire bytes
- [CI] Xeon: SPR socket0 server tests OOM-killed during KV allocation on sglang-spr08/18/19
- [Perf] [NVIDIA] dsv41: Run the side-stream mHC mix in BF16x3 for Blackwell decode batches
- Docs
- Python not yet supported