sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Perf] Batch GPU->CPU sync for multi-item scoring embeddings
- [Bug] remote_instance weight loading fails with --pp-size 2 — TransferEngine info keyed by tp_rank
- [DSV4] Support CPU offload of retracted KV on the PD decode node
- [Kernel] Fix shared-memory reuse race in CuTe DSL DSV3 fused-A GEMM
- [Kernel] Fix cross-iteration shared-memory WAR races in Blackwell GDN prefill
- [Diffusion][LoRA] Rework linear and LoRA linear common part
- Prebuild fixup_zero_kv JIT module at TRTLLM-MLA backend init
- [DP] add routing_key load-balance method; carry routing_key through batched GenerateReqInput
- QSA prefill predicates: correctness claim withdrawn, pending maintainer review
- Fix shared-memory race in MoE alignment kernel
- Docs
- Python not yet supported