75 manager sampler add minimum tokens per second accepted

[AguirreNicolas]

It was created a TimeoutHandler to assing timeouts.
For LLMs it depends of 3 variables:

• Prefill Time a.k.a Time-To-First-Token (TTFT)
• Decode Time a.k.a Time-Per-Output-Token (TPOT)
• Queue Time

Prefill time (TTFT)

It was modeled as a quadratic functions [1]. Besides (0, 0), two more points are nedded. This two point acts as a SLA. For insntance, this points are (8192,2) and (32768, 10).

Decode time (TPOT)

The decode time is modeled as a linear function with respect to the average silent reading time in English. According to the reference [2], the average silent reading speed is 238 words per minute. Given that there are approximately 0.75 tokens per word, the TPOT can be calculated as follows:

$TPOT ~(ms/tok) = \frac{1000 ms}{1seg *~\frac{238 ~words}{min}*\frac{ 0.75 ~tok}{word}*\frac{1min}{60seg} } = \frac{1000ms}{2.975tok} = 336 \frac{ms}{tok}$

Queue time

The queue time has been defined as 30 seconds.

Close #75

[1] https://arxiv.org/abs/2407.07000
[2] https://www.sciencedirect.com/science/article/abs/pii/S0749596X19300786

Close #75

[RawthiL]

some minor changes requiered