sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] [NVIDIA] blackwell: Keep SM120 off kernels that ship for SM100 only
- [Spec][XPU]Dspark compact mode with sps cost table support added
- [Fix] [NPU] mem_cache: Fall back to pageable KV backups on pinned host OOM
- [Perf] diffusion: load MiniMax H3 reference VAEs on demand
- [Refactor] hicache: Key DCP L3 pages by `logical_page_size`
- [Perf] [AMD] GLM5.3: Run KDA decode and HIP prefill on a BF16 state pool on gfx950
- [Perf] openai: serialize plain /v1/completions stream chunks without pydantic
- [Fix] linker: Offload Mamba branch checkpoints on stored nodes
- [Fix] attention: Fence SM120 attention K/V shared-memory reads before buffer reuse
- [Feat] benchmark: add synthetic audio input to serving benchmark
- Docs
- Python not yet supported