sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- Error deploy GPT OSS-120B with heterogeneous TP in a P/D
- [Bugfix] Defer DeepEP imports and honor the DeepGEMM disable flag
- [Playground] Verified cell: h100 / flash-vision / fp4 / balanced / single
- [Fix] Autotune the breakable prefill graph path before capture
- [MegaMoE] Add --megamoe-backend flashinfer_cutedsl (FlashInfer NVFP4 mega kernel, experimental)
- [Bug] compressed-tensors W4AFP8 MoE: cutlass_w4a8_moe passes a literal chunk_size=128, so a group_size≠128 checkpoint loads clean and serves silently wrong output
- Draft: [NVIDIA] Merge CUDA 13.4 container into main dockerfile
- [Quantization] Support per-channel Fp8 MoE with FlashInfer TRT-LLM
- [diffusion] Honor CPU offload on native transformers fallback
- fix: count re NOT_LITERAL as one character in stop_regex length bound
- Docs
- Python not yet supported