kthena
https://github.com/volcano-sh/kthena
Go
Lightweight, Modular, Kubernetes-native AI serving platform for scalable model serving.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Go not yet supported1 Subscribers
Add a CodeTriage badge to kthena
Help out
- Issues
- [Bug] ModelServing pods cannot be created after same-name re-creation: infinite re-enqueue loop caused by orphaned pods in manageRoleReplicasPerGroup
- Q: benchmark numbers in the Kthena introduction blog differ from our local reproduction — request for the detailed, repeatable test methodology (gateway/hardware specs, workload, metric definitions)
- trafficPolicy.retry is never read, so attempts and retryInterval have no effect
- bug(router): dequeue-time priority refresh breaks same-user FIFO in the fairness queue
- bug : controller-manager keeps reconciling after losing leader election
- ServingGroupRollingUpdate scales Roles in outdated ServingGroups before replacement
- fix: preserve Role configuration in outdated ServingGroups
- router: on-flight count goes negative after a pod is briefly NotReady, and traffic keeps leaning to that pod
- [RFC]: Agent-Aware KV Cache Affinity Scheduling and Lifecycle Management
- bug: ModelServing maxSurge rollouts finish with missing ordinal 0 and retain a temporary high ordinal
- Docs
- Go not yet supported