kthena
https://github.com/volcano-sh/kthena
Go
Lightweight, Modular, Kubernetes-native AI serving platform for scalable model serving.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Go not yet supported1 Subscribers
Add a CodeTriage badge to kthena
Help out
- Issues
- ModelServing rolling-update partition misclassifies non-contiguous ServingGroup ordinals
- runtime: decode current vLLM map-shaped KV events
- fix(model-serving): clear graceMap when status update fails in handleErrorPod
- cleanup: use %w instead of %v for error wrapping in fmt.Errorf
- router: decrement on-flight count on the PodInfo counted at dispatch
- [Enhancement] Support pluggable endpoint discovery in Kthena Router
- feat(router): add AgentHints, ProgramAffinity plugin and SessionCachePinning manager
- cli: support -o yaml and -o json in kthena get
- test(e2e): add E2E tests for kvcache-aware plugin in memory index mode
- test(e2e): add kvcache-aware E2E tests for memory index mode
- Docs
- Go not yet supported