-
Notifications
You must be signed in to change notification settings - Fork 37
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Autoscale based on KubeAI OpenTelemetry active requests metrics (#261)
* Replace existing controller metrics with OpenTelemetric metrics using Prometheus `/metrics` endpoint * Include common http otel metrics * Update autoscaling loop to scrape KubeAI instance instead of backend instances (all the info is contains in KubeAI instances) * Update docs and diagrams * Refactor `modelresolver` package to be named `endpoints` b/c it now also tracks KubeAI server endpoints * Modify integration test cases to run with unique system configurations * Make leader election durations configurable at the system level to facilitate expedient tests * Add integration test that simulates autoscaling calcs with multiple KubeAI replicas * Run `go mod tidy` * Fixes #123 * Fixes #237
- Loading branch information
Showing
38 changed files
with
993 additions
and
435 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.