sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [cherrypick from #36389] [LoRA] Let DP-attention LoRA run idle ranks and eager prefill (partial)
- feat(router): add opt-in deterministic response cache
- fix(hicache): reject buffer-only write-back at runtime
- AttributeError: 'ComponentData' object has no attribute 'parent'
- [Fix] Report grammar-queue aborts as abort instead of 400 BadRequestError
- [Perf] Add packed BF16 prefill for DeepSeek-V4.1 on B200/B300
- [AMD] Resolve QSA packed-varlen decode to aiter on HIP
- [RFC / HiCache] Same-node peer L2 sharing across DP ranks via /dev/shm
- [MiMo-VL] Shard vision encoding across the DP encoder
- [Docker] Ship AWS EFA userspace alongside the system rdma-core
- Docs
- Python not yet supported