vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bugfix][Spec Decode] Fix draft models using the target's pipeline stage under PP
- [XPU] Allow StatelessGroupCoordinator and elastic EPLB on XPU/xccl
- [ROCm][Quantization] Add tuned W8A8 block-FP8 Triton GEMM configs for MI355X
- [Bug]: Qwen3.5 vision RoPE angle precision differs from HF (public 9B reproduction, mixed FP32 workaround results)
- docs: fix broken FusedMoE imports in quantization guide
- [Multimodal] Add keyframe-aware sampling to PyNvVideoCodec backend
- [Bug]: MXFP4 linear layers on Hopper (default Humming kernel) silently round or zero weights whose E8M0 scale is 12+ binades below the fused tensor's largest scale
- [Kernel] Add GC-MoE backend with fused shared experts for DeepSeek-V4.1
- [Bugfix][Spec Decode] Fix non-deferred structured-output draft retrieval
- [Perf][ROCm][ROCM_ATTN] Run large fp8 decode batches on the AITER asm paged attention
- Docs
- Python not yet supported