vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported43 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bug]: vllm serve starts healthy but all requests fail with ModuleNotFoundError when using --prefix-caching-hash-algo xxhash or xxhash_cbor without xxhash installed
- [RFC]: Derender Endpoints to Provide Detokenization for Disaggregated Serving
- [Bugfix][LoRA] Dispatch fully-sharded fused MoE LoRA W13 expand per slice
- [Bug]: DeepSeek V4 model fails to load with transformers ≥ 4.57 — `compress_ratios` attribute removed
- [Bug]: Silent worker termination after long idle on dual-node `--no-ray` setup (likely `TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC` SIGABRT)
- [Bug]: rejection sampler skips argmax-invariant procs — patching the min_p/logit_bias 400 alone is insufficient
- [Bug]: `apply_top_k_top_p_pytorch` `scatter_` crashes under cudagraph capture on sm_121a (Blackwell GB10)
- [Bugfix][NVFP4] Expose batch-invariance for FlashInfer + CUTLASS FP4 MoE
- [Bug]: fully-sharded fused MoE W13 LoRA uses wrong slice offset after all-gather when local LoRA rank is 1
- [Bug] v0.21.0 release missing PR #42320 — DeepSeek-V4 MTP fails with `TypeError: missing required positional argument: post_mix`
- Docs
- Python not yet supported