vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Doc] Clarify --base-url must not include /v1 suffix in benchmark CLI
- [Feature][MRV2] Enable Qwen3.5 GDN ReplaySSM standard decode
- [Perf][MRV2] Trim the FULL graph output copy and the draft-token gather
- [XPU][UT] use fp32 when hf_runner on XPU
- [Kernel] Port fused_deepseek_v4_qnorm_rope_kv_insert to CuTe DSL
- [Rust Frontend] Bump the Rust toolchain to 1.100.0-beta.7
- [Rust Frontend] Replace assert!(matches!(..)) with std assert_matches!
- [Feature]: Add an NVIDIA RTX PRO 6000 Blackwell deployment recipe for DeepSeek-V4-Flash-Vision-Exp
- [Bugfix] Rebuild npcache when source checkpoint shards are updated
- [2/2][MRV2] Abort sibling microbatches when one fails
- Docs
- Python not yet supported