vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported62 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Perf][DCP] Vectorize SM100 direct Output/LSE consumption
- [Bugfix] Preserve variadic overrides for positional-only names
- [Bugfix][Kernel] Fit Triton prefill attention tiles for 256-wide heads in shared memory (EmbeddingGemma 2 support)
- [Model][LoRA] Support LoRA for EmbeddingGemma2Model
- [KVConnector] Track MultiConnector completion owners independently
- [MRV2] Pass FULL replay context to synchronous KV loads
- feat: Add Omni support to the Transformers backend
- docs(benchmarks): document boolean sweep plot filters
- [Model][ROCm] qwen4_exp: serve the AMD path in fp16
- [Transformers backend] Add sequence parallelism and async TP
- Docs
- Python not yet supported