vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported62 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Kernel] Extend LiteTopK decode to BF16 V4.1 top512 and next_n 1-6
- [Bugfix][Frontend] Keep qwen3/MiMo unclosed last tool parameter only when the call is closed
- [Bug]: Priority scheduling can un-schedule a request it already scheduled in the same step (encoder cache miss, prefix hits on unwritten KV, priority inversion)
- [Bugfix] Fix ModernBERT attention with tensor parallelism
- Fix Xid 31 MMU fault in EP16 CUDA-graph replay during reduce_scatterv
- [Config] Support in-memory Hugging Face config dictionaries
- [Bugfix][Attention] Check draft model support for FlashInfer large pages
- [CI] Automate dependency updates with hashes and a 7-day cooldown
- [Core][KVConnector] Don't wait on retained KV exports during cache-preserving keep pauses
- [Perf][Attention] DiffKV: split the 192-wide QK dot instead of padding it to 256
- Docs
- Python not yet supported