vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bugfix][Rust] Preserve nullable string parameters in the Muse parser ( # 64546 )
- [Performance]: Triton and other eager attention backends re-derive K/V cache views on every call (same class as #59534)
- [ROCm][Feature][MiniMax-M3.1-Flash] Support aiter_sparse_pa on TP2
- [Bug]: DeepSeek-V4.1 decode logprobs drift after a prefix-cache hit with swa_bounded_replay (default)
- [Doc][Recipe] Ring-1T-FP8: replace removed use_inductor with backend
- [RFC]: Accept already-decoded images over NIXL in the vllm-rs gRPC API
- [Kernel] --linear-backend=flashinfer: use FI backend="auto" with default settings for linear GEMMs
- [CI] Enroll Language Models (Hybrid) and Granite Compatibility in automatic sharding (stacked on #58795)
- [Bugfix] Keep metadata of a delayed BlockStored when an OffloadKey is reused
- [Perf][GDN] Use FlashInfer's regular prefill route for short batches
- Docs
- Python not yet supported