sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feat] sgl-router: Add opt-in decode cache affinity
- [CI] [AMD] rust-ext: Build Rust extensions once in stage-a and reuse them in later stages
- [Feature] [RFC] Ownership boundaries for externally orchestrated checkpoint and restore
- [Feat] metrics: expose per-process CPU usage
- [Feat] linker: Support runtime attach and detach of the unified cache external linker
- [Fix] scheduler: Count all pending work in DP load snapshots
- [Fix] [AMD] diffusion: Install websockets with the ROCm diffusion extras
- [Bug] Out-of-vocab `stop_token_ids` crash the server in the min_new_tokens penalizer
- [CI] [AMD] spur: Run AMD CI suites on MI35x nodes scheduled through Spur
- [Fix] hicache: Budget deduplicated host pools by physical ownership
- Docs
- Python not yet supported