vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Spec Decode][Qwen4Exp] Enable local argmax reduction for NVIDIA MTP
- [XPU][Kimi-K3] Run KDA metadata tests on XPU
- [Recipe] Kimi-K3: add MI355X Mooncake disaggregated prefill/decode guide
- [Recipe] Update Zhenwu Kimi-K3.1-Pro settings
- [Misc] Separate Mooncake PD and Store connectors into sibling packages
- [EPD] Support native gRPC workers in the Python proxy
- [Perf] Cut eager host overhead of collectives, MoE and router custom ops
- [Feature]: Add gc-moe as an option for moe-backend
- [Docs] Prepare PaddleOCR-VL recipe for the Ascend owner overlay
- [Perf][PP] Opt-in PP transport on CUDA copy engines (NVLink only)
- Docs
- Python not yet supported