sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] `--tokenizer-backend fastokens` deadlocks the HTTP server when a multimodal request arrives while a text request is being tokenized
- [Fix] multimodal: Keep the processor's single text unpadded so the shared tokenizer is not reconfigured
- [CI] [NVIDIA] gb300: Remove the 4-gpu-gb300 CI suite
- [Refactor] [AMD] kimi-k3: Rename the ROCm K3 env vars and read them through Envs
- [Fix] hicache: Hold PD decode restores until in-flight load-backs land
- [Bug] Forward hooks never run when CUDA graphs are on (docs say they run on every forward pass)
- [Doc] forward-hooks: Say that hooks don't run under CUDA graphs
- [Perf] [AMD] kimi-k3/mla: Add an opt-in fp8-Q ASM route for short verify windows on gfx950
- [Feat] alice-ai: add model and checkpoint MTP support
- [Feature] Validate empty and non-mapping diffusion config files
- Docs
- Python not yet supported