sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] responses: Kick-start the stream generator so validation errors answer a real HTTP 400
- [Feat] speculative: Insert bounded prompt tail on extend in NGRAMWorker
- [Bug] Full prefill CUDA graph capture fails for hybrid linear attention models (IndexError in HybridLinearAttnBackend._replay_metadata)
- [Fix] flashattention: Resolve window KV pool for frozen-KV MTP draft (illegal memory access on Gemma-4)
- [Fix] pd: harden rebootstrap submission and count pending retries
- [Fix] dsa: align FlashMLA metadata with prefill graph query rows
- [Feat] hicache: report resident L2 hit origins
- [Bug] Decode HiCache returns HTTP 500 when a shorter prefix has no Mamba checkpoint
- [Feat] moe: Add platform-scoped expert executor providers
- [Fix] lora: Skip unmatched weights during non-strict buffer loading in LoRAMemoryPool
- Docs
- Python not yet supported