vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [ROCm][Perf][DSv4.1] Asm decode layers for 17-256-row decode steps (gfx950, TP4)
- [Bugfix][Structured Output] Allow integer multipleOf with xgrammar
- [Rust Frontend] Implement the scale-out render routes on the main server
- [Perf][MRV2][Mamba2] Fuse aligned metadata CUDA graph inputs
- [ROCm][MiniMax-M3] MXFP4 mode for the fused norm / RoPE / KV insert
- [Docs] Generate the quantization scheme table from source
- [Bugfix][NIXL] Reject NVFP4 KV cache when the prefill block size is smaller than decode's
- [Perf][Attention] Serve PrefixLM prefill with FlashAttention-2 in the Triton backend
- [Attention] Prefer 128-token KV blocks for FA4 on Blackwell
- [ROCm][Perf] Fuse in_proj_qkvz + in_proj_ba into AITER dual-splitk GEMM kernel (Qwen3.8-27B)
- Docs
- Python not yet supported