sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Refactor] kda: Rename SGLANG_OPT_FUSED_KDA_VERIFY to SGLANG_OPT_KDA_TRITON_CHAIN_VERIFY
- [Perf] kda: Skip the per-layer target-verify row mask when PP spec is off
- [Perf] beam-search: Speed up trie-constrained beam search on SM120
- [Feat] linker: Describe the direct linker's Mooncake objects in the L3 layout
- [Feat] Dspark: Support PP decode with stage-local draft owners
- [Fix] model-gateway: Cancel paired PD requests promptly on failure
- [Profiling] Per-shape traces for prefill CUDA-graph capture
- [ROCm Product Quality Control]: Weekly Kimi Vendor Verifier (KVV) accuracy regression testing on AMD
- [Fix] disaggregation: Bound the connect phase of the retract rebootstrap POST
- [EFA Product Quality Control]: Nightly multi-node E2E functionality, accuracy & performance testing for EFA NIXL and EFA Mooncake Store, EFA DeepEP
- Docs
- Python not yet supported