sglang
https://github.com/sgl-project/sglang
Python
SGLang is a high-performance serving framework for large language models and multimodal models.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported1 Subscribers
Add a CodeTriage badge to sglang
Help out
- Issues
- [Feature] SM90 single-token W8A16 GEMV for 128x128 block-FP8 linears
- [Perf] SM90 single-token W8A16 GEMV for 128x128 block-FP8 linears
- [Bug] bootstrap_room fan-out wraps in the Rust worker but not in Python (io_struct) — mixed PD legs can disagree on the rendezvous room near i64::MAX
- fix: Install numactl in runtime images - needed for SGLANG_NUMA_BIND_V2
- [Fix] Keep the server alive when speculative verify sampling ops fail to JIT (#43412)
- [Roadmap] Pipeline Parallelism Roadmap (2026 Q4)
- [gRPC] Forward caller mm_hashes and advertise multimodal inputs
- [VLM][EPD] Honor caller mm_hashes under zmq_to_scheduler and mooncake
- [Bug] sgl-model-gateway: usage/billing token accounting wraps (u32) and passes negative worker reports through unchecked
- [model-gateway] Fix usage accounting: clamp worker token counts and widen sums to u64
- Docs
- Python not yet supported