vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported61 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bugfix][Tool Calling] Dots non-streaming parse drops post-tool-call text that streaming returns
- [Model] Load q/k/v scales for the quantized Kimi-K2.6
- [Bugfix][Compilation] Align speculative graph sizes with query and TP widths
- [Perf][GLM-5.3-Flash] Specialized decode for GB300 TP4
- [Perf][Attention] FA4 sparse MLA prefill with query groups for 16-64 heads per rank
- [CI/Build][KV Connector] Mooncake tests: patch the httpx module the connector uses
- [ROCm][Qwen3.8-Flash-Next] Specialized decode: MTP verify steps
- [Bugfix] Pin torchcodec to the CUDA-variant build matching the Docker image
- [RFC] Stream request-level prefill progress over gRPC
- [Bugfix][Spec Decode] Fail fast when a PP MTP drafter has no input embedding
- Docs
- Python not yet supported