vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Perf] Enable PDL for Inductor-generated Triton kernels in 'interactivity' performance mode
- [Perf] Skinny GEMM: eager fast path and SM90 plans for the Qwen3.5 MoE BF16 projections
- [Perf][MoE] Grouped W8A16 skinny GEMM for block-FP8 routed experts at small token counts (SM90)
- [Misc] Skip other ranks' weights before weights transforms
- [Kernel] Let --linear-backend deep_gemm override the DeepGemm model a…
- [Bugfix][DSA] Skip sparse indexer prefill work for empty chunks
- Allow reset_prefix_cache to clear cached prefixes without preempting running requests
- [Bugfix][MoE] Avoid double router weighting in TRTLLM FP8 experts
- [Misc] Remove dead torch < 2.11 version checks and workarounds
- [KV Connector] Remove the in-tree LMCache integration package
- Docs
- Python not yet supported