paymentservice is not emitting traces

[justincastilla]

Bug Report

Which version of the demo you are using?
commit hash

Symptom

When I supply credentials for my Elastic account and run make start, I receive telemetry data in my Elastic Observability UI, yet paymentservice is not listed as a service under APM/services.

What is the expected behavior?
I expect to see paymentservice as a service under the Observability UI

What is the actual behavior?

The payment service is running, yet it is not emitting telemetry data. The logs for the paymentservice container shows the following:

2024-10-17 12:19:29 {"level":30,"time":1729192769616,"pid":17,"hostname":"4d6193718a5f","trace_id":"8b10114fdcf03684c033e4040be8827d","span_id":"1a28b18921741e34","trace_flags":"01","request":{"amount":{"currencyCode":"USD","units":{"low":1886,"high":0,"unsigned":false},"nanos":499999991},"creditCard":{"creditCardNumber":"4532-4211-7434-1278","creditCardCvv":114,"creditCardExpirationYear":2039,"creditCardExpirationMonth":2}},"msg":"Charge request received."}
2024-10-17 12:19:29 {"level":30,"time":1729192769617,"pid":17,"hostname":"4d6193718a5f","trace_id":"8b10114fdcf03684c033e4040be8827d","span_id":"1a28b18921741e34","trace_flags":"01","transactionId":"41fd3dc1-2a32-4219-8385-c8bbbc589b21","cardType":"visa","lastFourDigits":"1278","amount":{"units":{"low":1886,"high":0,"unsigned":false},"nanos":499999991,"currencyCode":"USD"},"msg":"Transaction complete."}
2024-10-17 12:19:34 {"name":"elastic-otel-node","level":50,"msg":"{\"stack\":\"Error: 16 UNAUTHENTICATED: Received HTTP status code 401\\n    at callErrorFromStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/call.js:31:19)\\n    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client.js:193:76)\\n    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:360:141)\\n    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)\\n    at /usr/src/app/node_modules/@grpc/grpc-js/build/src/resolving-call.js:129:78\\n    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)\\nfor call at\\n    at ServiceClientImpl.makeUnaryRequest (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client.js:161:32)\\n    at ServiceClientImpl.<anonymous> (/usr/src/app/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)\\n    at /usr/src/app/node_modules/@elastic/opentelemetry-node/node_modules/@opentelemetry/instrumentation-grpc/build/src/clientUtils.js:131:31\\n    at /usr/src/app/node_modules/@elastic/opentelemetry-node/node_modules/@opentelemetry/instrumentation-grpc/build/src/instrumentation.js:211:209\\n    at AsyncLocalStorage.run (node:async_hooks:346:14)\\n    at AsyncLocalStorageContextManager.with (/usr/src/app/node_modules/@elastic/opentelemetry-node/node_modules/@opentelemetry/context-async-hooks/build/src/AsyncLocalStorageContextManager.js:33:40)\\n    at ContextAPI.with (/usr/src/app/node_modules/@opentelemetry/api/build/src/api/context.js:60:46)\\n    at ServiceClientImpl.clientMethodTrace [as export] (/usr/src/app/node_modules/@elastic/opentelemetry-node/node_modules/@opentelemetry/instrumentation-grpc/build/src/instrumentation.js:211:42)\\n    at /usr/src/app/node_modules/@elastic/opentelemetry-node/node_modules/@opentelemetry/otlp-grpc-exporter-base/build/src/grpc-exporter-transport.js:100:32\\n    at new Promise (<anonymous>)\",\"message\":\"16 UNAUTHENTICATED: Received HTTP status code 401\",\"code\":\"16\",\"details\":\"Received HTTP status code 401\",\"metadata\":\"[object Object]\",\"name\":\"Error\"}","time":"2024-10-17T19:19:34.658Z"}

The error:

..."name":"elastic-otel-node","level":50,"msg":"{\"stack\":\"Error: 16 UNAUTHENTICATED: Received HTTP status code 401...

is very interesting to me, as I can verify that other services are showing up in the APM services dashboard. This very well might be due to an oversight on my end, but I'm quite vexed.

Reproduce

Could you provide the minimum required steps to resolve the issue you're seeing?

1. Clone this repository
2. Provide Elastic endpoint and bearer token in src/otelcollector/otelcol-elastic-config-extras.yaml
3. Run make start
4. Observe logs coming from paymentservice container

We will close this issue if:

• The steps you provided are complex.
• If we can not reproduce the behavior you're reporting.

Additional Context

I encountered this when I was running through the demo to create an instructional video on implementing otel for elastic with our JS client.
Also I took steps to remove the rslave entry in the docker_compose.yaml:
open-telemetry#1739