Stamina and OpenTelemetry

[gsakkis]

I'm looking into integrating stamina in a project that uses opentelemetry, where some functions are decorated with (a more elaborate version of) @tracer.start_as_current_span("name"), e.g. something like:

    @tracer.start_as_current_span("name")
    async def call_some_api(self, some_id):
        ...

Is it as simple as just adding a @stamina.retry(on=httpx.HTTPError)?

• If yes, should stamina be applied before the opentelemetry decorator or the other way around?
• If no, what else needs to change?

[hynek]

It depends on what behavior you prefer: do you want a span started for every retry or should one span cover all retries?

[gsakkis]

Not sure, is there a common/best practice?

• One span for all retries makes them effectively transparent, there's no way to tell from the outside whether the call succeeded at once or after (one or more) retries. Ideally one would want to log the total number of retries (and possibly the list of errors).
• One span for every retry would provide more info but again the retries are not explicit (e.g. a retry_num span attribute as in the Prometheus integration).
• Since spans can be nested, the above are not mutually exclusive; a span covering all the retries with children spans for each retry could make sense as well.

[hynek]

I’m the wrong person to ask about OTEL best practices, but I would assume you want introspection into failures. Maybe a combination of one for all and then instrument retries using https://stamina.hynek.me/en/stable/instrumentation.html?

[gsakkis]

For starters at least I think i'll go with the simplest approach (one span for all retries) but will look into set_on_retry_hooks if I need a deeper integration with otel and introspection into failures. Many thanks!