sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Bug] Disconnected OpenAI chat requests can continue decoding after client disconnect
- [Fix] tokenizer_manager: Skip watermark redaction in request dumps when watermarking is disabled
- [RFC] [XPU] paged-experts: Intel XPU support with Lookahead Expert Scheduling (LES) and an SSD tier for iGPU, single-GPU MoE offload
- Rainj me/rust cp apis migration
- [Fix] dsv4: Stabilize TopK v2 selection at tied scores
- [Fix] dsv4.1: [2/5] Support CP>1 - stabilize Hopper BF16x3 mHC
- [Fix] gdn: Keep a mixed batch's decode tail on the decode kernels under deterministic inference
- [Fix] [XPU] multimodal: add missing dependency ffmpeg
- [Feat] dsv4.1: [3/5] Support CP>1 - enable encoder replay with L1/L2 recovery
- [Fix] [AMD] dsv41: Fix wrong TP4 wo_a output from hipBLASLt's strided bmm on ROCm 7.2
- Docs
- Python not yet supported