vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported60 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bugfix][Frontend] Keep Muse tool call arguments valid JSON for non-fâŚ
- [Performance] Whisper: move log-mel extractor to GPU worker and batch conv stem
- [Bug]: EXIF-oriented images (orientation != 1) return 500 on mistral-tokenizer models: "Expected there to be 1 prompt placeholders ... found 0"
- [Doc][Helm] Use model-response probes in the Helm chart example
- [Bugfix][DeepSeek V4.1] Fix system-only prompt rendering
- [Perf][SM90] Score only candidate blocks in the DSv4.1 decode indexer
- [Bugfix][KV Connector] Pass hf3fs_metadata_server_url to Hf3fsMetadataClient
- [Frontend] systemone: let joint schema models take more options than labels
- [Frontend] systemone: reject unknown request fields
- [Bug][Structured Outputs] Parser._apply_structural_tag overrides a request's response_format when tools are present â schema not enforced (GLM-5.3)
- Docs
- Python not yet supported