sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [perf] lossless final-layer prefill optimization for Qwen models
- [MiMo-VL] Avoid sorting vision window indices
- [mem_cache] Skip the empty-key insert in UnifiedRadixCache.cache_finished_req
- fix: load IQuestLoopCoder gate projection weights
- [Bug] [NPU] ascend attention backend silently corrupts output with page_size 16 (128 OK, 1 rejected per #25169)
- [NPU] Enable NGRAM speculative decoding on Ascend NPUs
- [function_call] gemma4: streaming tool_index follows call order, not tools-list position
- [fa4] pick nvvm.fmax signature by try/except instead of CUDA_VERSION gate
- [MLX] Add a dense-model e2e correctness test for Apple Silicon
- [Performance][PP/PD] Bootstrap admission waits ~7.8s for consensus under PP16 concurrent prefill
- Docs
- Python not yet supported