vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Core][Scheduler] Add opt-in aging to cache-aware admission
- [Perf][Bugfix][RL] Fix sparse NCCL update OOM and improve performance
- [Bugfix][EPLB][Hardware][Ascend] Set device_index for NPU in async EPLB worker
- [Bug]: Async EPLB worker asserts `device_index is not None` on Ascend NPU
- [Bugfix][KV Connector] Preserve parked load reservations across KV allocations
- [Bugfix][Rust Frontend] Don't reopen reasoning after the prompt closed it
- [Bugfix][Frontend] Collect all MCP tool pages in list_server_and_tools (#65074)
- [Bugfix][Frontend] Reject orphan tool results
- [Bugfix][Model] Avoid host synchronization in dots.ocr vision encoder
- [Bugfix][MTP] Add packed_modules_mapping and MTP quant alignment to DeepSeek-V4 NVIDIA/XPU MTP
- Docs
- Python not yet supported