BatchSpanProcessorBenchmark: spanCount and dropping spans

[iRevive]

BatchSpanProcessorBenchmark defines the spanCount variable:

@Param({"1000", "2000", "5000"})
private int spanCount;

By default, BatchSpanProcessor would keep only 2048 elements in the queue:

static final int DEFAULT_MAX_QUEUE_SIZE = 2048;

This means that 2952 spans are dropped each run when spanCount = 5000.

Is that done on purpose? For example, to measure how the BSP's queue drops incoming spans in a case of overflow.
Or is it an oversight?

[jack-berg]

The benchmarks were written years ago by different maintainers and we have to piece together intent by analyzing git history and the benchmark code and making an informed guess. Its not obvious to me what the benchmark is supposed to demonstrate, and how to interpret its results vs all the other BatchSpanProcessor*Benchmark in the same package.

It is reasonable to try to exercise the queue limits of BatchSpanProcessor, but not sure how that applies here:

• We have a BatchSpanProcessor configured to export to a DelayingSpanExporter
• DelayingSpanExporter will return a success response code after some configurable delay
• The benchmark starts a configurable number of span in the setup code, then in the benchmark code itself, ends all the spans and flushes the processor
• There are parameters for the delay (0ms, 1ms, 5ms) and the number of spans (1000, 2000, 5000)

Not clear from all this what is being measured and why.

What are you trying to understand by running the benchmark?

[iRevive]

Thanks for the detailed answer!

What are you trying to understand by running the benchmark?

I maintain https://github.com/typelevel/otel4s. It's an OpenTelemetry library for https://github.com/typelevel/cats-effect ecosystem.

The library provides scala-friendly API. As for the SDK implementations, there are two options:

• A wrapper for OpenTelemetry Java - the recommended option (JVM only)
• An experimental implementation from scratch. Besides JVM, Scala can be compiled into JavaScript and native binary; hence, our implementation can be used on these platforms.

I'm measuring the overhead of our implementation of BSP versus OpenTelemetry Java, and I was confused about whether measuring the queue limits is relevant in this particular benchmark.