vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Performance][ROCm]: set HSA_SCRATCH_SINGLE_LIMIT in the ROCm image: per-dispatch scratch allocation costs ~130 us before large RCCL all-reduces on MI355X
- [Bug]: Responses API rejects `developer` role for Mistral Large 4 with --tokenizer-mode mistral
- [Feature]: Accept the global DP rank in X-data-parallel-rank under hybrid/external LB
- [Bugfix][Docker] Preserve source-built Triton in downstream uv installs
- [ROCm][Perf][GLM-5.3-Flash] mxfp4_mla: native gfx950 kernel for prefill and mixed steps (on #64323)
- [Feature]: Close the remaining DeepGEMM (and other kernel-library) runtime-JIT gaps in kernel warmup
- [Perf][DSv4] Add opt-in locality placement for MegaMoE weights
- [Bugfix][Core] Widen EAGLE periodic SWA tails so a diverging sibling keeps its prefix-cache hit
- [Bug] Sparse-attention indexer: native padded decode (`requires_padding`) gives wrong logits/top-k on ragged spec-decode steps
- [ROCm][Perf] gfx942 sparse MLA decode: load e8m0 scales once per 64 columns, BLOCK_K 16
- Docs
- Python not yet supported