vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported46 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- Log warning for scheduled token mismatch
- Clean up OMP and NUMA topology detection
- Enable grouped_topk kernel registration.Hook up the existing grouped_topk kernel to the kernel registry.
- [Bug]: vLLM attempts to download Hugging Face cache file during inference despite local model path (Gemma 4)
- [Bug]: Gemma 4 MoE (26B-A4B-it) crashes at startup — AssertionError: top_k is None in MoEMixin.recursive_replace
- [Bug]: Gemma 4 31B Structured Outputs weird behaviour / character output - might be a quick solve
- [Bug]: Gemma4 on vLLM + PI coding agent: Validation failed for tool "edit": - path: must have required property 'path'
- fix(reasoning): prevent streaming end-token desync in base and other parsers
- [Feature]: Speculative Prefill — Draft-Assisted Sparse Prefill for TTFT Reduction
- [Bug]: Certain Ranks Take a Look Time to Load Weights
- Docs
- Python not yet supported