vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported62 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Model][ROCm] qwen4_exp: serve the AMD path in fp16
- [Transformers backend] Add sequence parallelism and async TP
- [Bugfix][Parser] Align Kimi K3 response transitions
- [Bugfix][LoRA] Map adapter keys back through Transformers' reversed WeightRenamings
- [Bug]: `is_nvfp4_quantized` crashes on compressed-tensors configs whose `format` is a list (mixed NVFP4 + FP8 checkpoints)
- [Performance]: Default CUDA graph capture size (2 * max_num_seqs) runs short prefills eagerly at small max_num_seqs
- [Bugfix][Multimodal] Validate DeepStream probe result (fix #60212)
- [Core] Update PyTorch to 2.15.0, torchvision to 0.29.1, triton to 3.9.0 (test channel)
- [Bugfix] Enable MLA RoPE+KV-cache fusion when layer names are hoisted
- [Bugfix] Reject sweep benchmark names that collide after string conversion
- Docs
- Python not yet supported