Graph job traces with Dynatrace APM

[timja]

What feature do you want to see added?

I can't see a way to graph the Job traces in Dynatrace APM:
https://github.com/jenkinsci/opentelemetry-plugin/blob/master/docs/job-traces.md

The trace itself can be displayed fine:

Trace

But I would like to create a dashboard that includes information like this:

Dashboard

(from the elastic apm dashboard: https://raw.githubusercontent.com/jenkinsci/opentelemetry-plugin/master/docs/images/kibana_jenkins_overview_dashboard.png)

I've managed to graph metrics just fine:

But nothing that is specific to a job seems to be available in 'Data explorer' or 'Multidimension analysis')

This was integrated with the open telemetry collector using this config:
https://github.com/hmcts/cnp-flux-config/blob/a504a799e2dbe955215bd4bedad2cc89643b6c27/apps/monitoring/opentelemetry-collector/opentelemetry-collector.yaml
https://github.com/hmcts/cnp-flux-config/blob/a504a799e2dbe955215bd4bedad2cc89643b6c27/apps/monitoring/opentelemetry-collector/ptl-intsvc/00.yaml

(+ some encrypted secrets) in this format:

    exporters:
      dynatrace:
        api_token: ****
      otlphttp:
        headers:
          Authorization: Api-Token ****

Upstream changes

No response

[v1v]

IIUC, the entry point for the traces is a different one as per https://www.dynatrace.com/support/help/extend-dynatrace/opentelemetry/opentelemetry-traces/opentelemetry-trace-ingest-api

https://{your-domain}/e/{your-environment-id}/api/v2/otlp/v1/traces

https://github.com/hmcts/cnp-flux-config/blob/a504a799e2dbe955215bd4bedad2cc89643b6c27/apps/monitoring/opentelemetry-collector/ptl-intsvc/00.yaml#L14

Might be needed to point to https://ebe20728.live.dynatrace.com/api/v2/otlp/v1/traces ?

[timja]

v1/traces is automatically appended by the SDK.

Traces are being ingested fine as you can see in https://user-images.githubusercontent.com/21194782/162940468-4fa6a3da-6e8e-4826-906a-55e8a06b42cf.png

but I can't do any graphing / dashboard on them from what I can tell.