kthena
https://github.com/volcano-sh/kthena
Go
Lightweight, Modular, Kubernetes-native AI serving platform for scalable model serving.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Go not yet supported1 Subscribers
Add a CodeTriage badge to kthena
Help out
- Issues
- Include applicable ModelServing plugins in rollout revision identity
- bug: controller-manager healthz stays healthy after ModelServing controller exits
- feat(router): enable lora-affinity scheduler filter — datastore already tracks loaded adapters
- autoscaler: replicas are overscaled when ServingGroups contain more than one entry pod
- Use KV-cache usage as the engine-neutral scheduling signal
- feat(router): enable LoRA affinity filtering by default
- ModelServing: a late Pod delete event can delete its healthy replacement
- fix(autoscaler): pass ctx.Done() as stop channel to wait.Until in Run
- model-serving: back off repeated failure recovery
- perf(router): score each PD group's prefill set once during PD scheduling
- Docs
- Go not yet supported