vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bugfix][ROCm] Avoid unused index caches in MiniMax-M3 sharing layers
- [Bugfix][MoE] Preserve invalid expert IDs in DeepEP LL round-robin mapping
- [Quantization][MoE] Prefer Humming over Marlin for MXFP8 MoE on SM12x
- [Bugfix][NIXL] Don't fail a completed KV transfer when telemetry is unavailable
- [Bug]: DeepSeek-V4.1 with block FP8 MoE activations fails on Hopper at TP=4/8 (Humming partial input-scale group)
- [Spec Decode][DCP] DSpark: run sliding-window MLA drafts under DCP
- Optimize sliding window prefix cache hit search
- [Doc][Benchmark] Note synchronized waves in the max-concurrency load pattern
- [Bugfix] Don't bound beam search's search width by `max_logprobs`
- [Bugfix][NIXL] Serialize stale engine cleanup with the handshake lifecycle
- Docs
- Python not yet supported