sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Unified Cache][UMBP] Support DeepSeek-V4.1 and packed draft layers in the UMBP direct linker
- [AMD] Select the fused AR+RMSNorm 1-stage kernel by token count and read volume
- [Scheduler] Stop the prefill delayer's queue trigger from ending its own delay
- [GLM-5.3-Flash] Use top-k v2 for the k-pool indexer
- [Fix] Escape control characters in Hunyuan tool-call streams
- [OpenAI] Let a named tool_choice keep every tool in the prompt
- models: add fusion integration for gemma models
- [Spec] Exact snapshot-free GDN verify for DFLASH and NEXTN (SGLANG_ENABLE_GDN_REPLAYSSM_FOLD)
- [WIP][sglang-miles] Add GPU delta decode and in-place weight updates
- [MoE] Skip block-padding rows in the TMA down activation and quant
- Docs
- Python not yet supported