vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported59 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Do not merge][Model] Kimi K3.1: minimal reference branch to run K3.1-Pro
- [KV Offload] Bound PUT staging and schedule aligned batches
- [Frontend] Accept `reasoning_content` and `join` in response templates
- [Frontend] Serve PelaAI/KnowLine-35B-A3B-Gen1.5 on /v1/systemone
- [ROCm][CI] Wait for DPX GPU lock to avoid collisions
- [CI] Fix B200 nightly gsm8k evals: ReplaySSM, Kimi-K3 DSpark, DeepEP LL
- [Perf][MoE] Shard replicated TP prefill tokens for SM90 MegaMoE
- [Perf][Attention] Tune single-request FP16 Triton decode on SM80
- [Security] Require native chat encoding in Kimi K3 mode
- [Perf] Whisper: torch.compile the encoder layer stack with compile_mm_encoder
- Docs
- Python not yet supported