vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported61 Subscribers
Add a CodeTriage badge to vllm
Help out
- Issues
- [Spec Decode][DeepSeek-V4.1] Support DSpark with pipeline parallelism
- [Model][LoRA] LoRA support for MiniMax M3
- [Bugfix][Core] Hash every overlapping multimodal item into prefix-cache block keys
- [Bugfix] Keep content after stray <tool_call> in hermes streaming
- Anthropic /v1/messages: inline system messages are merged into the leading system prompt, so every Claude Code turn misses the prefix cache after that point
- [RFC]: MoRIIO transfer contract: safe KV reclamation and bounded failure for P/D (READ and WRITE)
- [Core] Grow padded full attention blocks
- [Bugfix][Quantization] Thread input_quant into MXFP8 MoE dispatch
- [Bugfix][Parser] Repair tool argument types without breaking streamed prefixes
- [Perf][GLM-5.3] GLM-5.3 latency-specialized decode on B200 (CuTe DSL)
- Docs
- Python not yet supported