Skip to content

Make DataCache Sendable and drain it with a single writer task - #925

Draft
kean wants to merge 9 commits into
mainfrom
refactor/data-cache-concurrency
Draft

Make DataCache Sendable and drain it with a single writer task#925
kean wants to merge 9 commits into
mainfrom
refactor/data-cache-concurrency

Conversation

@kean

@kean kean commented Aug 16, 2026

Copy link
Copy Markdown
Owner

DataCache was the last type in the library still coordinating its work with GCD: a serial DispatchQueue, an NSLock for the staging area, asyncAfter for the flush and sweep timers, and @unchecked Sendable to paper over the rest.

It now keeps all of its mutable state – the staging area and the configuration – behind a single OSAllocatedUnfairLock, which makes it Sendable. The 1-second flush timer is replaced by a single self-terminating writer Task that drains the staging area in a loop, so the writes still batch up under load but reach the disk in milliseconds, and there is no timer state to race on.

The blocking file operations run on a serial dispatch queue. Doing them in a task would occupy a cooperative pool thread for the whole duration of a write, so the writer hops into the queue once and drains everything there – blocking work is what GCD threads are for. The queue is .default, not .utility: a block submitted with async never gets its priority escalated, unlike the queue.sync the cache used before, so at .utility the caller of flush() waits on throttled I/O.

Two bugs fixed along the way:

  • Data races on the configuration. sizeLimit, sweepInterval, isSweepEnabled, and trimRatio were plain vars written by the client and read from the I/O queue. They now live in the locked State; sweepDelay and onSweepCompleted became let, since only the internal test initializer ever set them.
  • init performed disk I/O on the calling thread. scheduleSweep() read and JSON-decoded .data-cache-info before returning, i.e. on the main thread for ImagePipeline.Configuration.withDataCache. The check moved into the writer, so onSweepCompleted now also fires only when a sweep actually runs.

One more trap worth recording: Task.sleep at .background priority is subject to timer coalescing, and scheduledSweepUpdatesMetadata went from 0.034s to 11.7s. The sweep task runs at .utility, with a comment explaining why.

Breaking: DataCache.flush() and DataCache.sweep() are now async. DataCache.flush(for:) and DataCache.queue are removed – the queue is an implementation detail now. DataCacheTests used queue to suspend the I/O, so withSuspendedIO moved into the source as an internal test hook.

The DataCaching protocol and the staging area are unchanged. The per-change IDs in the staging area look like ceremony, but they are what lets the writer drop the state lock while writing and then remove only the entries that weren't replaced meanwhile — that's the property that keeps reads parallel to writes.

To test

  1. Run the NukeTests scheme – 991 tests pass, including all 47 DataCacheTests.
  2. Run the NukeThreadSafetyTests scheme – 7 tests pass.
  3. Run NukePerformanceTests/DataCachePeformanceTests – the write benchmarks are unchanged.

The staged changes are now flushed by a `Task` instead of `DispatchQueue.asyncAfter`,
and all of the mutable state moved behind a single `OSAllocatedUnfairLock`, which
makes the class `Sendable` rather than `@unchecked Sendable`.
@kean kean added this to the 14.0 milestone Aug 16, 2026
@kean kean added the improvement Non-functional change that improves existing functionality label Aug 16, 2026
kean added 4 commits August 16, 2026 19:56
… task

DataCaching.cachedData(for:) and containsData(for:) are now async, and the
pipeline reads the disk cache off the ImagePipelineActor. DataCache replaces
the scheduled flush with a single self-terminating writer task that drains
the staging area in a loop: writes batch up automatically under load, data
reaches the disk in milliseconds instead of after a fixed 1-second delay,
and there is no timer state to race on.

flush() and sweep() are now async and wait by awaiting the writer task's
value, which escalates the utility-QoS writer to the caller's priority, so
an explicit flush performs the same as the old queue.sync. flush(for:) is
removed.
The file operations are synchronous and can block for a long time, so
running them in a Task occupied a cooperative pool thread for the entire
duration of a write. The writer task now hops into a serial dispatch queue
and drains the pending work there, which is what GCD threads are for. The
queue also guarantees that the disk operations never overlap. Replace it
with an actor using DispatchSerialQueue as its SerialExecutor once the
deployment target reaches iOS 17.

The queue is .default, not .utility: a block submitted with async never
gets its priority escalated, unlike the queue.sync the cache used before,
so at .utility the caller of flush() waits on throttled I/O. At .utility
the write benchmarks regressed by 40-70%; at .default they match.
@kean kean changed the title Replace the DataCache dispatch queue with Swift concurrency Make DataCache Sendable and drain it with a single writer task Aug 21, 2026
kean added 4 commits August 22, 2026 09:46
The writer started on the very first staged change, so the batching
depended on how slow the disk happened to be: a trickle of writes spaced
further apart than a single disk operation got a task hop, a staging
snapshot, and a separate pass over the disk each, and the repeated writes
to the same key were no longer collapsed into one.

Restore the 1 second window the cache had before the rewrite. The writer
waits through it before draining, so the changes made within it are
written in one pass. Only the automatic drain is throttled – flush() and
sweep() perform the work themselves and never wait on the window.
flushWaitsForPendingWrites and concurrentFlushesAllReturn suspended the
I/O and resumed it while a flush was pending, but suspendIO() only gates
the writer – flush() submits to the I/O queue itself, so both tests passed
without ever reaching the scenario in their comments and would still pass
with resumeIO() deleted. Stage the changes behind a long flush interval
instead, which is a state the cache actually gets into.

The suite also never cleaned up after itself: every run left ~21 cache
directories in ~/Library/Caches for good, and the sustained traffic tests
wrote an unbounded number of files into them. Recycle the traffic keys,
remove the directories when the tests are done with them, and stop the
writer first so that it can't re-create one after the fact.

Bound the number of DataCache operations the thread safety test runs at a
time the way the OperationQueue it replaced did, add the missing coverage
for the writer draining on its own, and rename writeWithFlushIndividual,
which stopped measuring a per-key flush when flush(for:) was removed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Non-functional change that improves existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant