chore: chainsaw improvements

[eddycharly]

Chainsaw improvements (based on comments in #2630)

• simplified makefile
• used resource/namespace templating instead of kubectl commands

[eddycharly]

Things like this could probably be eliminated too

# Name of the daemonset to check
DAEMONSET_NAME="prometheus-kubernetessd-collector"

# Get the desired and ready pod counts for the daemonset
read DESIRED READY <<< $(kubectl get daemonset -n $NAMESPACE $DAEMONSET_NAME -o custom-columns=:status.desiredNumberScheduled,:status.numberReady --no-headers)

# Check if the desired count matches the ready count
if [ "$DESIRED" -eq "$READY" ]; then
  echo "Desired count ($DESIRED) matches the ready count ($READY) for $DAEMONSET_NAME."
else
  echo "Desired count ($DESIRED) does not match the ready count ($READY) for $DAEMONSET_NAME."
  exit 1
fi

[eddycharly]

Did i break a unit test ? 🤔

[swiatekm]

Did i break a unit test ? 🤔

No, I'm fixing it here: #2643. It does seem to break much more frequently here than in any other PR, which made it easier to hunt down.

[swiatekm]

Nice, this looks much better than before.

[eddycharly]

Honestly your tests are of good quality, I often see a lot of hacks to workaround kuttl limitations but not in your tests ! 💪

[eddycharly]

Like this one Kuttl-In-Kuttl 😂 🙈

[swiatekm]

Honestly your tests are of good quality, I often see a lot of hacks to workaround kuttl limitations but not in your tests ! 💪

Have you looked at the target allocator tests with jobs running curl in a loop to check if metrics are available? 🙈

[eddycharly]

Have you looked at the target allocator tests with jobs running curl in a loop to check if metrics are available? 🙈

Not yet ! But metrics is an area we want to address, this is often not well end to end tested and chainsaw could easily be adapted to assert on metrics we believe.

[eddycharly]

I will address this one next https://github.com/open-telemetry/opentelemetry-operator/blob/main/tests/e2e-targetallocator/targetallocator-kubernetessd/check-daemonset.sh

We really can do better than that ! (not in this PR, i'll do another one)