vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported62 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bug]: flashinfer_moe_ep_cutedsl fails at model load because vLLM CUDA images don't ship nvshmem4py (No module named 'nvshmem')
- [Test] Fix kernel tests on Blackwell GPUs
- [Bugfix] flashinfer_moe_ep_cutedsl: reject it without nvshmem4py and install nvshmem4py
- [Model][LoRA] Load Inkling PEFT adapters in the serving layout
- [Feature][DCP] --dcp-gather: gather-based DCP for MLA attention without native DCP (+ DSv4)
- [Bug]: prompt_embeds decode limit is per-part, allowing request-level memory amplification
- [DeepSeek-V4-P4] Align batch-invariant mHC and o-proj paths
- [Docs] Clarify Qwen4Exp PLE Engram configuration
- [Bugfix][DCP] Sort DCP group ranks so rank_in_group matches collective order
- [DeepSeek-V4-P5] Rewrite matmul_persistent with a split-K plan
- Docs
- Python not yet supported