sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] Glm47MoeDetector emits only the first of multiple tool calls in one final streaming increment
- [Bug] GLM streaming parsers forward unknown tools when SGLANG_FORWARD_UNKNOWN_TOOLS is disabled
- Add flashinfer GVR top-k backend for DSv4 decode
- [Bug] FlashInfer backend drops attn_logit_softcapping: the cap is passed to the deprecated forward() instead of plan(), so gemma-2 and grok-1 run uncapped
- [AMD] Remove three per-layer copy/cast kernels from Kimi-K3 aiter MoE path
- [Fix] Prevent PP deadlock during weight update group initialization
- [feat] Support DeepSeek-V4-Flash-0731 on sm89
- fix: guard set_embed_and_head against tied word embeddings
- feat: load community EAGLE drafts without repackaging the checkpoint
- [DSA] Remove the device-to-host syncs on the decode DP-divergent branch
- Docs
- Python not yet supported