sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [LoRA 17/21] CuTeDSL BF16 grouped GEMMs and provider
- [LoRA 18/21] FP8 MoE LoRA on the Triton vendor
- [LoRA 19/21] CuTeDSL FP8 grouped GEMMs and providers
- [LoRA 20/21] Shipped CuTeDSL FP8 base-GEMM tile tables
- [LoRA 21/21] NVFP4 MoE LoRA on the Marlin vendor
- [Feature] Return output top logprobs as flat base64 arrays on request
- [DSv4.1] Skip wasted layer 20 work on earlier prompt tokens under decoder bounded replay
- [Fix] Stop DeepSeek-V4.1 encoder bounded replay from crashing on the first prefill without speculation
- Keep tool calls possible with response_format under tool_choice=auto
- [Bug] trigger_cuda_user_coredump misses the default pipe: driver creates corepipe_<host>_<pid>, helper expects documented corepipe.cuda.<host>.<pid>
- Docs
- Python not yet supported