sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Fix] Cap reasoning_tokens at the emitted length when a speculative run crosses max_new_tokens
- [Failure Tracker] PR Test (AMD)
- [Bug] Startup logs and server_info expose API keys in resolved server arguments
- Fix DP rendezvous port allocation outside ephemeral range
- [AMD] Add TLX backend for vectorized 5D paged decode
- [Fix] Preserve background flag in GenerateReqInput.__getitem__
- [mm] Release multimodal features at consumption (opt-in) to bound host memory
- [Bug] DFlash target_layer_ids fallback guesses wrong tap layers; count collision fails silently
- spec: require dflash_config.target_layer_ids instead of guessing tap layers
- [CPU][Spec] Prepack the DFlash draft context projection for AMX
- Docs
- Python not yet supported