vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- docs: add --task embed to the RAG example launch commands
- [Bugfix] Do not initialize CUDA when importing vendored FLA
- fix(tool_parsers): stream content preceding <function_call> in Granite20bFCToolParser
- [CI] Bake torchao into the test image locks and re-enable the quantization jobs
- [Build][MiniMax-M3] Bump fmha_sm100 to 51f4f5c
- [Test] Restore PaddleOCR-VL generation coverage on Transformers v5
- [Bugfix][Gemma4] Preserve assistant content before tool calls
- [CI] Run the NIXL PD accuracy core sweep as pytest and shard it
- [Feature][Hybrid] K0 spec-window slot release for hybrid mamba models
- [Bugfix][Qwen2.5-Omni] Size the video encoder budget from the processor's video limits
- Docs
- Python not yet supported