sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- BUG:skip when spech4 deepseekv4
- [docs] deterministic inference: add a coverage map, document the numerics env vars, refresh the backend list
- fix(sampler): abort requests with fully-NaN logits instead of sampling garbage (#33187)
- feat(scheduler): add lpm anti-starvation aging (#31954)
- feat(openai): support drop_thinking for DeepSeek chat encoding
- [CUDA Graph] Reduce memory by reusing a process-wide capture stream
- [Fix] Cohere Command4: strip echoed `<|START_THINKING|>` in streaming and flush the final text block
- fix(eplb): exclude fused shared slots from DeepEP counts
- Fix DSV4 indexer torch fallback allocating a num_heads-oversized intermediate
- Fix DSV4 torch paged-MQA-logits dispatch selecting a variant that asserts
- Docs
- Python not yet supported