vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Perf][Frontend] Reduce sampled-logprob return-path overhead
- [Perf][Frontend] Reduce chat SSE serialization and Harmony polling
- [Spec Decode] Measure the Dynamic SD table instead of writing it by hand
- [Model][CUDA] Gemma 4 native vision tower on H100: bound eager tower memory, fix Triton test
- [Bug]: /inference/v1/generate with "max_tokens": null bypasses default/override/platform output-token caps
- [Bugfix][Frontend] Handle non-string HTTPException details
- [Perf][HiSparse] Batch the coordinator's per-page residency bookkeeping
- [Bugfix] Serialize YAML list items that are mappings as JSON in --config
- [Core] SubGPU pipelining: run micro-batches on green-context SM pools (Model Runner V2)
- [Mamba] Add FlashInfer FP16 GDN ReplaySSM for Qwen3.5
- Docs
- Python not yet supported