sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Deterministic] SM120: route small-M bf16 GEMMs to a Triton tile that matches DeepGEMM bit for bit
- [Bug] DeepSeek-V4 on SM120: the default SGLANG_FP8_PAGED_MQA_LOGITS_TORCH=True also turns off the C4 indexer's row-chunk planner (+3.3–3.8 GiB at 128k)
- [AMD][CI] Skip redundant pip-cache chown walk and MORI rebuild in setup
- fix(hicache): handle finite prefetch I/O failures safely
- [Bugfix] Avoid mutating JSON schema response format
- [Bugfix] Preserve text in Ollama streams
- [Bug][MLX] min_new_tokens allows early EOS and stop-token termination
- Fix MLX min_new_tokens for EOS and stop tokens
- [FlashInfer][DSv4.1] Persist autotune measurement policies with cached tactics
- [DSV4.1] Reduce eager Engram extend preparation
- Docs
- Python not yet supported