vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported62 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Test] Pin reasoning token counting for the response-template parser
- [Feature]: vllm run-batch --resume to continue an interrupted batch
- [Bug]: NIXL DCP config checks don't apply to NixlPullConnector
- [Bugfix][Reasoning] Strip <think> from multi-token streaming deltas in Ernie4.5 and HY4 parsers
- [Bug]: support_torch_compile's positional type check is off by one: each argument is checked against the previous parameter's annotation
- [MRV2][Spec Decode] Context Aware Sparse LM Head on quantized lm_heads
- [Bugfix] Reduce threaded PT loader peak checkpoint-tensor memory by 52-59%
- [Bugfix] Fix startup plan over-allocating 2.83 GiB KV after multimodal changes
- [Misc] Log enabled fusion passes only under VLLM_COMPILE
- [Docs] Clarify Qwen3 implicit reasoning end for tool calls
- Docs
- Python not yet supported