How should we determine the target metric size?

[arschles]

Right now, the target metric size is hard-coded at 100. We should figure out how to dynamically set this. The scaler has access to all the interceptors so it can, at any time, figure out the size of the HTTP request queue.

[tomkerkhove]

I think we should move it to the CRD and use 100 as a default

[ajanth97]

I think it would be ideal if we could algorithmically calculate the best target metric for a service which would produce the best performance. We can also allow the user to specify a target metric in the HttpScaledObject in order to accommodate an edge case.

[arschles]

@ajanth97 what would be an example of such an algorithm?

[ajanth97]

I haven't looked in to the implementation of the algorithm yet but the high level idea I had was to calculate the target metric based on how our app performs. In particular one of the independent variables we can look in to is the latency of each request. Once we have more load test data we can look in to how to implement such an algorithm.

Meanwhile as a first starting off point I will work on implementing the feature of specifying the target metric in the CRD

[rwkarg]

Something like a hill-climbing algorithm as used in the .NET ThreadPool would be a good fit, but there's a significant amount of work to make a general purpose method that is broadly applicable.

The CLR Thread Pool 'Thread Injection' Algorithm

Optimizing Concurrency Levels in the .NET ThreadPool: A Case Study of Controller Design and Implementation

[arschles]

@rwkarg thanks for those links. my understanding is @ajanth97 is simply providing a user-overridable default for now. as we proceed, we'll keep an eye on whether folks require a more automated approach