vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [MoE] Autotune FlashInfer MegaMoE live-token workspaces at startup
- [Tracking] Muse Spark Open Improvement
- [Bugfix][ROCm][DSv4.1] Inject Engram at a PP stage that starts at an Engram layer
- [Bug][ROCm] vLLM fails to detect AMD GPU in WSL when AMDSMI cannot initialize
- [Bug]: fuse_rope_kvcache breaks draft_model/EAGLE spec decode when the drafter's attention shape differs from the target's (ROCm)
- [Bug]: ROCM_AITER_FA crashes with draft_model/EAGLE drafters in extend_forward (None prefix in merge_attn_states; cp_mha_gather_cache illegal address)
- [CI] Release GPU and distributed state at every test-file boundary
- [Bug]: PyTorch `.bin` checkpoints can silently lose a distinct `lm_head` when embeddings are configured as tied
- [Bug]: `MessageQueue.shutdown()` cannot wake a reader blocked on an overflow payload
- [ROCm][UMBP] Support UMBP standalone mode for DeepSeek-V4.1-Flash
- Docs
- Python not yet supported