Skip to content

Rework the CI pipeline and run it locally with make ci - #906

Open
kean wants to merge 4 commits into
mainfrom
ci/rework-pipeline
Open

Rework the CI pipeline and run it locally with make ci#906
kean wants to merge 4 commits into
mainfrom
ci/rework-pipeline

Conversation

@kean

@kean kean commented Aug 16, 2026

Copy link
Copy Markdown
Owner

The job matrix now lives in .scripts/ci.sh, and the workflow does nothing but hand it a group name, so make ci runs exactly what CI runs. This is what stops the drift that left .scripts/validate.sh unused for four years.

make ci                  # every job
make ci-list             # every job, group, and action
make ci-macos            # one GitHub job's worth
make ci-test-nuke-ios    # a single job
make test                # every test job, on every platform

A selector matches a job id, a group, or an action, so there is one way to run any slice of the matrix.

CI

  • Rebalance the 5 macOS jobs (GitHub Free allows no more) instead of letting the iOS one dominate. Critical path goes from 16m24s to ~10m
  • Run SwiftLint in CI for the first time — it had never been wired up. On a Linux runner, so it costs no macOS concurrency. Not --strict yet, since the existing warnings need clearing first
  • Add compile-only coverage for visionOS, watchOS, and NukeVideo, none of which CI built at all
  • Set permissions: contents: read; the token was write-all
  • Resolve simulators at runtime instead of pinning OS=26.4.1, which forced a ci.yml change on every runner image update
  • Upload xcresult bundles on failure, and report retried tests as flaky rather than letting -retry-tests-on-failure hide them
  • Drop the stale validate.sh and install_swiftlint.sh (SwiftLint 0.47.1, pointing at a ./temp/ path that no longer exists), and test.sh
  • Add dependabot for actions; move to checkout/upload-artifact v7

Bugs the missing coverage was hiding

  • NukeVideo did not compile for watchOS, even though Package.swift declares .watchOS(.v8) and ships it as a product. VideoPlayerView.swift was missing the #if !os(watchOS) guard its two sibling files have
  • The Xcode project could not build for visionOS at all — the project-level SUPPORTED_PLATFORMS omitted xros/xrsimulator

Flaky tests

Both registered an observer after the work it was supposed to observe had already started, so the notification could be missed.

  • errorPropagatedToBothCoalescedSubscribers asserted coalescing without holding data loading until both tasks had registered, so the first could fail before the second subscribed and createdTaskCount came back as 2. It now uses the existing withSuspendedDataLoading helper
  • cancellation created the Task before subscribing to MockDataLoader.DidStartTask, which loadData posts synchronously. Lose the race and nothing cancels the task, so it waited on a suspended queue until the suite's 5-minute .timeLimit killed it — a 300s hang, which is what the macOS job was failing on here

Reporting

Three cases where the old script would have reported green, or near enough to hide the problem:

  • A test job that fails without any individual test failing (a hung test killed by its time limit, a crashed runner) no longer prints a clean 599 run · 599 ✓ tally next to a ❌
  • The test count is taken from the first run rather than the last, since -retry-tests-on-failure re-runs only the failures and the last tally counts a subset
  • A local run without SwiftLint installed reports the job as skipped rather than passed

kean added 4 commits August 16, 2026 09:44
The job matrix now lives in `.scripts/ci.sh`, and the workflow does nothing but
invoke it per group, so `make ci` runs exactly what CI runs.

- Rebalance the 5 macOS jobs (GitHub Free allows no more) instead of letting the
  iOS one dominate, which takes the critical path from ~16m to ~10m
- Run SwiftLint in CI for the first time, on a Linux runner so it costs no macOS
  concurrency. Not `--strict` yet, since the existing warnings need clearing
- Add compile-only coverage for visionOS, watchOS, and NukeVideo, none of which
  were built by CI at all
- Set `permissions: contents: read`; the token was write-all
- Resolve simulators at runtime rather than pinning `OS=26.4.1`, which needed a
  ci.yml change on every runner image update
- Upload xcresult bundles on failure, and report retried tests as flaky rather
  than letting `-retry-tests-on-failure` hide them
- Drop the stale `validate.sh` and `install_swiftlint.sh` (SwiftLint 0.47.1,
  pointing at a `./temp/` path that no longer exists), and `test.sh`
- Add dependabot for actions, and move to checkout/upload-artifact v7

Two things the missing coverage was hiding:

- NukeVideo did not compile for watchOS even though Package.swift declares
  watchOS support and ships it as a product; VideoPlayerView.swift was missing
  the `#if !os(watchOS)` guard its sibling files have
- The Xcode project could not build for visionOS at all, because the
  project-level SUPPORTED_PLATFORMS omitted xros/xrsimulator

Also fix a race in errorPropagatedToBothCoalescedSubscribers, which asserted
coalescing without holding data loading until both tasks had registered, so the
first could fail before the second subscribed.
Locally the console now shows a pinned dashboard — progress bar, completed
count and percentage, the running job, and a window on the last few lines of
output — with each job printing a result row as it finishes. Full logs go to a
timestamped directory under TMPDIR, with a `latest` symlink, instead of the
repo and the terminal.

CI is unaffected: without a TTY the full stream is preserved, and
NUKE_CI_OUTPUT_DIR pins the log path so xcresult upload still works.
NUKE_CI_VERBOSE=1 forces the streaming output locally.

The sink reads with a blocking `read` rather than `read -t 1`. bash 3.2, which
is what /bin/bash is on macOS, returns 1 from `read -t` on timeout — the same
as EOF — so a timeout-based loop exited at the first quiet second and killed
the build with SIGPIPE.
`cancellation()` created the Task before subscribing to
`MockDataLoader.DidStartTask`, which `loadData` posts synchronously. Lose
the race and nothing cancels the task, so it waited on a suspended queue
until the suite's 5-minute `.timeLimit` killed it. That 300s hang is what
the macOS job was failing on.

Collapse `--group <name>` and job ids into one selector that matches an
id, a group, or an action, and let the SwiftLint job go through ci.sh
too, so the workflow really does only hand it a name. Replace the tail
window with a single repainted progress line, and drop the XCTest
timeout flags and result parsing — there is no XCTestCase left.

Three cases that used to report green: a job that fails with no
individual test failing now says so instead of printing a clean tally
next to a ❌; the test count comes from the first tally rather than the
last, since -retry-tests-on-failure re-runs only the failures; and a
local run without SwiftLint installed reports skipped, not passed.

The simulator cache also never worked — it was assigned inside a command
substitution, so every job re-resolved and re-booted in a subshell.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant