vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported61 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bug]: Tiered KV offload: a secondary-tier hit larger than the CPU tier loads nothing
- [Bug]: Qwen3-VL video pruning (EVS): M-RoPE recompute starts num_timestamp_tokens + 1 tokens too early when the prompt keeps the outer <|vision_start|> (transformers >= 5.10)
- [Perf][DSv4.1] Pack DSpark verify positions into one FlashMLA head tile on SM90
- [Doc] Document kv_load_failure_policy in the MooncakeStoreConnector guide
- [Bug]: nvidia/GLM-5.3-NVFP4: silent garbage output with expert parallelism + TP>1 + prefill context parallelism >1
- [Bugfix][Attention]Allow the one-time sync when building INT4 RHT signs
- [Bugfix][Model] Size RoPE cos/sin cache to max_model_len for unscaled RoPE
- [Perf][Model] Stack K2-Horizon MoVA value experts once after loading
- [Bugfix][MRV2] Coordinate final CUDA graph redispatch across DP ranks
- [Feature][Kernel] Triton autotune lora
- Docs
- Python not yet supported