vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported43 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bugfix][LoRA] Dispatch fully-sharded fused MoE LoRA W13 expand per slice
- [Bug]: DeepSeek V4 model fails to load with transformers ≥ 4.57 — `compress_ratios` attribute removed
- [Bug]: Silent worker termination after long idle on dual-node `--no-ray` setup (likely `TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC` SIGABRT)
- [Bug]: rejection sampler skips argmax-invariant procs — patching the min_p/logit_bias 400 alone is insufficient
- [Bug]: `apply_top_k_top_p_pytorch` `scatter_` crashes under cudagraph capture on sm_121a (Blackwell GB10)
- [Bugfix][NVFP4] Expose batch-invariance for FlashInfer + CUTLASS FP4 MoE
- [Bug]: fully-sharded fused MoE W13 LoRA uses wrong slice offset after all-gather when local LoRA rank is 1
- [Bug] v0.21.0 release missing PR #42320 — DeepSeek-V4 MTP fails with `TypeError: missing required positional argument: post_mix`
- [Bugfix] All pyNCCL copy-only operation to use int8 instead of fp8
- fix: add --api-key support and authentication warning to gRPC server
- Docs
- Python not yet supported