Skip to content

test: Make the mocked run status sequence independent of poll timing - #1028

Draft
vdusek wants to merge 1 commit into
masterfrom
test/deterministic-status-message-mock
Draft

test: Make the mocked run status sequence independent of poll timing#1028
vdusek wants to merge 1 commit into
masterfrom
test/deterministic-status-message-mock

Conversation

@vdusek

@vdusek vdusek commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

test_actor_call_redirect_logs_to_default_logger_sync and its three siblings flake in CI, dropping ('Status: RUNNING, Message: Initial message', 20) (example).

StatusResponseGenerator advanced the status every 5 requests, counted across every caller of the run endpoint — the two setup GETs, the watcher's 1-per-second polls, and wait_for_finish's 4-per-second polling loop. wait_for_finish burned the "Initial message" window in roughly half a second, so the watcher's first poll had a ~0.5s margin to land inside it.

Plain requests and waitForFinish requests are now answered separately: plain requests walk the status sequence, and waitForFinish reports the run finished only once they have walked all of it. The first status covers three plain requests, which is call's two setup GETs plus the watcher's own first poll in any order. With nothing watching there are no plain requests and the run reads as finished immediately, so call(logger=None) still returns at once.

Verified by injecting a delay before the watcher's first poll: on master the tests fail from 0.6s upward; with this change the two call()-based tests pass at 0.6s, 2s, 5s and 10s. (test_redirect_status_message_sync still fails past ~5s, but that is bounded by its own 1s sleep, not by the mock.)

✍️ Drafted by Claude Code

@vdusek vdusek added adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. labels Aug 21, 2026
@vdusek vdusek self-assigned this Aug 21, 2026
@github-actions github-actions Bot added this to the 147th sprint - Tooling team milestone Aug 21, 2026
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Aug 21, 2026
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.95%. Comparing base (df80da1) to head (45dd84c).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1028   +/-   ##
=======================================
  Coverage   94.95%   94.95%           
=======================================
  Files          58       58           
  Lines        5436     5436           
=======================================
  Hits         5162     5162           
  Misses        274      274           
Flag Coverage Δ
integration 91.72% <ø> (ø)
unit 85.81% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants