vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- fix(reasoning): preserve text before <think> in BaseThinkingReasoningParser
- [Bugfix][Frontend] Validate speech-to-text stream options after form coercion
- fix(parser): don't leak buffered prefix when a PreLexedTerminal supersedes it
- [CI][ROCm] Restore the AMD mirror of Fault Tolerance E2E
- [Core] Add cache_salt_start_idx for divergence-point prefix-cache salting
- [Bugfix][Tool Parser] llama4_pythonic: render argument values as literals, not quoted strings
- [Bugfix] granite_20b_fc: render conversations that carry a system message
- [Bugfix][ROCm] Let the FP8 MoE oracle fall back when the global AITER flag can't serve the layer
- [Perf][Kimi-K3] Parse streamed reasoning and tool output in O(delta)
- [Tests] Make multi-GPU config tests independent of the host GPU count
- Docs
- Python not yet supported