sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] quantization: Apply each fused projection's own NVFP4 global scale in linear layers
- [Perf] mlp: Extend dense SwiGLU/block-FP8 quant fusion to SM90, prefill and TP>1
- [Bug] SM120 DeepSeek-V4.1: low-ratio indexer decode dies with schedule_meta=None (fp8_fp4_paged_mqa_logits arg #6) in dev-dsv41
- [Feat] dllm: Expose cumulative request-level forward metrics
- [Feat] fireredasr2: Add FireRedASR2-LLM speech recognition support
- [Feat] docker: Install AWS EFA in the main image; drop the EFA image variant
- [Feat] scheduler: Include per-request time stats in scheduler outputs without enable_metrics
- [Doc] diffusion/realtime: Document the session protocol shared by all realtime pipelines
- [Perf] [AMD] kimi-k3: Run KDA target verify through AITER's fused FlyDSL kernel
- [Fix] deepseek_v2/glm4_moe_lite/nextn: Handle checkpoint KV-cache scales for MLA attention and NVFP4 draft models
- Docs
- Python not yet supported