RFE: Add support for OpenTelemetry metrics and tracing

[dramdass]

Description

This RFE (Request for Enhancement) is to add the ability for codecarbon to allow collection of OpenTelemetry traces and correlation with codecarbon metrics. See
https://opentelemetry.io/docs/specs/otel/metrics/

This will allow existing codecarbon users to view codecarbon metrics alongside their regular observability tools such as Jaeger, Honeycomb etc. (thanks to the OpenTelemetry standard). This allows per-trace carbon tracking with additional context in addition to trend analysis.

Proposed Implementation

The proposed implementation is to follow a similar pattern to the Prometheus feature:

• Add save_metrics_to_otel and save_traces_to_otel flags to the track_emissions decorator. These flags will govern if full traces are collected along with metrics and persisted in the OpenTelemetry collector.
• Allow configuration of OTel Collector

I will add a more detailed technical proposal and MR if you agree this is a feature worth implementing. I'm very open to discussion and ideas, and look forward to contributing in this space.

[benoit-cty]

Thanks, that's a good idea, I wasn't aware of OpenTelemetry. Feel free to propose a MR, we will be glad to help you.

For Prometheus we work in push mode, could it be the same for OpenTelemetry ?

[dramdass]

Thanks!

For OpenTelemetry, we can use push mode as well - https://opentelemetry.io/docs/collector/configuration/#receivers

I'm thinking of 3 use cases:

• User already has OpenTelemetry setup and does not want codecarbon metrics per trace. In this case, exporting from Prometheus would work well. My MR will add documentation for this case.
• User already has OpenTelemetry setup and wants codecarbon metrics per trace. In this case, the MR will add Exemplars by adding the (trace_id, span_id) to the metrics collection by the emissions tracker.
• User does not have OpenTelemetry setup. In this case, we integrate OpenTelemetry traces if that flag is set.

I will send out an MR later this week

[SaboniAmine]

Hello! A lot of work have been done to integrate into logfire, by building some prometheus / otel-like metrics. We still didn't added the push mode, however it should be easier now.
Are you still interested into this feature ?