vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [MoE][FlashInfer] Opt-in transport (CFT/Lamport) selection for NVLink one-sided all-to-all
- [Model] Add transcription to the Transformers modeling backend
- [CI] Enable mypy for tests/v1/core and tests/v1/engine
- [ROCm][Perf][DSv4.1] Use the single-kernel DSv4 top-k on ROCm
- [ROCm][Perf][DSv4.1] Split-K BF16 router GEMM for gfx950 decode batches
- [CI/Build][NVIDIA] Move Rubin nightly images to CUDA 13.5
- [Bug]: CuMemAllocator.discard retains the CPU backup of asleep allocations
- [ROCm][Kimi-K3][Perf] Support BF16 MLA projections in specialized decode
- [Bug]: cyankiwi/Qwen3.8-27B-AWQ-INT4输出混乱
- [RFC]: Add a Tree Speculative Decoding Framework
- Docs
- Python not yet supported