kthena
https://github.com/volcano-sh/kthena
Go
Lightweight, Modular, Kubernetes-native AI serving platform for scalable model serving.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Go not yet supported1 Subscribers
Add a CodeTriage badge to kthena
Help out
- Issues
- router: wait for the old server to release the port before reusing it
- router: changing a Gateway listener's hostname can leave its port dead (bind: address already in use)
- controller-manager: fail /healthz when an enabled controller stops
- model-serving: return from Run when the caches never sync
- perf(model-serving): reuse plugin chain per pass instead of recreating per role replica
- feat(router): support kv-cache-usage score plugin and vLLM kv_cache_usage_perc metric
- model-serving: fix flaky scale-down E2E caused by lost Role Events
- Honor recovery settings while waiting to replace failed Pods
- Fix rollout availability and replica health checks
- Resume unfinished Pod deletions after a controller restart
- Docs
- Go not yet supported