vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported62 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Bug]: Tool-call argument coercion turns string values such as "0017" into integers for ["string", "integer"] parameters
- [Bugfix][Tool Parser] Preserve non-canonical numeric strings when schema allows string
- [Refactor][KV Connector][Mooncake] Move connector metadata and wire structs to their own modules
- [Docs] Add ROCm decode megakernel (MonoKernel) authoring guide
- [Tool Parser] Fix truncated/unclosed DSML parameters and add incremental DSML arg streaming
- [Bugfix][Tool Parser] Fix invalid streamed JSON for control characters in hy_v3 tool args
- [Bugfix][Structured Output][v0.31] Honor resolved backends and mask widths
- [Bugfix][Rust Frontend] Drop a DeepSeek DSML tool-call opener cut after its DSML token
- [Bugfix][Structured Output][v0.32] Honor resolved backends and mask widths
- [Bugfix][Tool Parser] Fix invalid streamed JSON for MiniCPM5 string args with leading or trailing spaces
- Docs
- Python not yet supported