sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [AMD] Fuse small-M DSV4.1 shared-expert and router work horizontally
- [Perf] Scale fp8 KV writes once: drop triton's K/V clones and the wrappers' unit scale
- [Fix][NPU] Keep MoE routing weights in FP32 through AscendTPDispatcher
- [AMD] Use Triton unified attention and page_size=32 on gfx1250
- [Diffusion] Tune Kitchen INT8 row splitting for SM120
- [OpenAI] Encode chat prompts one message at a time and reuse the ids across turns
- [Bug] DSv4.1: text-only batches take `vision_topk()` on CUDA, skipping recorder and EPLB remap
- [DSv4.1][CUDA] Preserve standard TopK post-processing for VL routing
- Use tuned AITER BF16 GEMM for ROCm MLA query projections
- [Spec] Plan FlashInfer EAGLE verify and draft from host-known lengths (SGLANG_ENABLE_SYNC_FREE_SPEC_PLAN)
- Docs
- Python not yet supported