vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bugfix][KV Connector] Keep same-host NIXL on NVLink with the extensible KV cache
- [ROCm][Perf][DeepSeek-V4.1] Fused mega-attention prefill kernel for MI355X
- [Feature][ROCm]: Configurable HIP host wait policy for WSL GPU workers
- [CI] Add check-docstring-style pre-commit hook and use single backticks in docstrings
- [WIP][Quantization] Triton kernels for ModelOpt IQ2_XXS / Q8_0 block weights on SM75+
- [Feature][Parser] Stream strict nested string arguments in Qwen XML
- [Bugfix][Parser] DeepSeek V4.1: treat DSML markers as structure only from the DSML token
- [Doc][ROCm] Add verified MI300X/MI325X entries to Qwen3.8-27B recipe
- [Doc] Use the dash-separated CLI flag names in scoring and tool-calling docs
- [Doc] Use the dash-separated CLI flag names in the remaining docs
- Docs
- Python not yet supported