vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bug]: MCP tool discovery drops tools after the first tools/list page
- [Attention] Opt in to startup sizing of the FlashInfer workspace
- [Bugfix][MCP] Keep first server's tools on duplicate server name
- [Bug]: [ROCm] AITER MLA `paged_kv_indices` sized max_num_seqs x max_model_len (4 GiB per builder) pins a cached segment, Kimi-K3 DBO OOMs at startup
- [Bug]: [ROCm][Kimi-K3] DBO: MoE workspace locked smaller than the DP-gathered batch, first request fails
- [Bugfix][EC] Republish runner cache hits after CPU-tier eviction
- [Bugfix][DSv4.1] Preserve BF16 Engram checkpoint tables
- [Model Runner V2][Spec Decode] Draft token heads for block-diffusion speculators
- [ROCm][Perf] Build AITER MLA full-graph decode metadata in one kernel
- [Bugfix] shm_broadcast: unblock writer in acquire_write when shutdown
- Docs
- Python not yet supported