vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [ROCm][CI] Run the MI355 BF16 AR-RMS fusion E2E matrices nightly with a per-PR subset
- [Bugfix] Load FP8 KV-cache scales for Kimi-Linear's MLA and MiniMax-M3's sparse attention
- [Bug]: [ROCm] GLM-5.3 TP8 hangs with MTP and CPU KV offload
- [Feat][KV Offload] Add Hybrid KV Backend with bounded SSD streaming
- [Spec Decode][Watermarking] Allow watermarking with greedy drafts
- [RFC][HiSparse] Selective shared KV offloading for DeepSeek V4.1
- [WIP][HiSparse] Selective KV offload for DeepSeek V4.1
- [MoE][NVIDIA] Support SM107 in flashinfer_moe_ep_cutedsl for NVFP4 experts for FI 0.7.3
- [Bugfix][Attention] Use 4 warps when the 16-row prefill tile still misses shared memory
- [ROCm][DSv4.1][Perf] Run the fused mHC seams as AITER's single-launch HIP Mega-MHC
- Docs
- Python not yet supported