kthena
https://github.com/volcano-sh/kthena
Go
Lightweight, Modular, Kubernetes-native AI serving platform for scalable model serving.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Go not yet supported1 Subscribers
Add a CodeTriage badge to kthena
Help out
- Issues
- runtime: /v1/download_model and /v1/load_lora_adapter return 500 for a non-empty config object
- runtime: accept a config object in the download endpoints
- ModelInfer: Handle pod update event
- Use better SDK to decode request to and response from llm models
- Add best practice template for CLI
- [Feat] Implement status fields of API: Gateway, HTTPRoute and InferencePool
- Add runtime profile CRD for ModelBooster similar to KServe ClusterServingRuntime
- bug(router): Rate limiter state lost during controller reconciliation
- benchmark: add smoke Job for mock-llm router path
- [Bug] Distributed State Inconsistency in In-flight Request Counters
- Docs
- Go not yet supported