vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bug]: cu134 nightly image has conflicting PyTorch/Triton dependency metadata (aarch64)
- [Bugfix][Kernel] Plan b12x NVFP4 W4A16 linear token counts at load time
- [Quantization] Load ModelOpt NVFP4 W4A16 token embeddings in mixed-precision checkpoints
- [Bugfix][Rust Frontend] Fill files missing from a cached Hub snapshot instead of treating them as absent
- [RFC]: Single Batch Overlap (down GEMM ↔ combine) for DeepEP LL
- [Bugfix][Spec Decode] Use the target's kernel block sizes for EAGLE draft attention
- [RFC]: DBO support for FlashInfer NVLink all-to-all backends (single-node H200/B200 first)
- [CI] Say which steps stay out of automatic sharding, and why
- [CI][Do not merge yet] Turn automatic sharding on by default
- [CI][Do not merge] Staging: flag combinations for default-on sharding
- Docs
- Python not yet supported