vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [ROCm][MORI IO]Moriio handshake failure
- [Perf][Spec Decode] Fused logits gather for the DSpark draft head of opted-in models
- [Bug]: /v1/messages streaming reports a tool call cut short by EOS as stop_reason tool_use
- [Bugfix][Benchmark] Fix multichannel ASR audio duration filtering
- [Doc][ROCm] Qwen3.5-122B-A10B recipe: move AMD to v0.33.0, retire the UNIFIED_ATTN warning
- [ROCm][Perf] Use tuned AITER Triton `gemm_a8w8` for per-tensor FP8 decode GEMMs on gfx950
- [RFC]: A KV compatibility identity, enforced on NIXL and P2P transfers and readable by routers
- [Bugfix][CPU] Use the C++ GDN conv in speculative decoding on AVX-512BF16 CPUs
- [Bug]: NixlConnector drops kv_connector_extra_config.num_threads for non-UCX backends
- [MM] Support mm_device_do_normalize for Gemma 4 Unified
- Docs
- Python not yet supported