vllm
https://github.com/vllm-project/vllm
Python
A high-throughput and memory-efficient inference and serving engine for LLMs
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported43 Subscribers
View all SubscribersAdd a CodeTriage badge to vllm
Help out
- Issues
- [Model] Standardize `spec_step_idx`
- fix: honor CUDA graph memory profiling opt-out
- [Bugfix] Fix Gemma4 reasoning for batch chat completions
- add last_engine_iteration_timestamp_seconds in metric.
- Fix: Prevent RuntimeError in DeepGEMM warmup by deferring alignment check for non-FP8 modules
- [Bugfix][V1] Defensive guard for stale req_id in `_update_after_schedule`
- [Bugfix][Kernel] Fix mxfp8 scale swizzling after EP all-to-all
- [Kernel] Fuse dual RMSNorm + residual + scalar in Gemma4 MoE layers
- [Performance] Decode fast path for scheduler and model runner
- [Core] Added Feather as another waiting request queue to LLM inference scheduler
- Docs
- Python not yet supported