kthena
https://github.com/volcano-sh/kthena
Go
Lightweight, Modular, Kubernetes-native AI serving platform for scalable model serving.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Go not yet supported1 Subscribers
Add a CodeTriage badge to kthena
Help out
- Issues
- Separate gateway control plane and data plane
- Ability to specify the upgrade order of Roles
- Update prow bot and add cherrypicker
- does GPU pd disaggregation example work on nixl connector
- [Proposal] Native Kubernetes HPA Integration for ModelServing (Group-Level & Role-Level Scaling)
- Proposal: Centralized Scheduling Service for Kthena Router in Distributed Deployment
- How Modelbooster and ModelServing work with volcano queue？
- ModelRoute should reject invalid targetModels entries
- Mock backend for router-only benchmarking
- [Bug] Quick Start ModelBooster: demo-model-downloader CrashLoopBackOff due to hf-mirror.com HTTP 308 redirect and missing cacheURI leading slash
- Docs
- Go not yet supported