vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Model] Muse Spark Open Cleanup
- [Perf][DSA] Rank BF16 prefill indexer logits with DeepSelect by default on SM100
- [Sleep Mode] Remove level-2 workarounds that #66010 makes unnecessary
- [Bug][KV Connector][Kimi-K3.1-Pro] Local hybrid hit can exceed draft-attention coverage
- [RFC]: Text sessions on /v1/realtime with streaming-input prefill
- [Bugfix][Parser] DeepSeek DSML: keep a closer inside a tool-argument value
- [Model] Remove preprocessing from simple `load_weights` methods
- [Bug][Prefix Cache][Kimi-K3.1-Pro] DSpark does not enable shared Mamba checkpoints
- [Perf][Kernel] Partition mHC input-norm stores on SM100
- [Bugfix][Qwen4Exp] Size the QSA decode logits workspace for the worst-case decode batch
- Docs
- Python not yet supported