Skip to content

refactor: move Wave 5 touch commands to platform runtime - #1987

Merged
thymikee merged 5 commits into
mainfrom
codex/wave5-touch-runtime
Aug 24, 2026
Merged

refactor: move Wave 5 touch commands to platform runtime#1987
thymikee merged 5 commits into
mainfrom
codex/wave5-touch-runtime

Conversation

@thymikee

@thymikee thymikee commented Aug 24, 2026

Copy link
Copy Markdown
Member

Summary

Moves the Wave 5 touch cluster from legacy capability and dispatch routing to request-bound platform runtime operations for click, press, fill, longpress, and hover.

  • admits exact owner facts and binds only the operations each command requires
  • keeps ref support truthful to the request-scoped provider, including sparse web providers
  • retains the Maestro-only iOS non-hittable route while distinguishing a successful direct element tap from an executed coordinate fallback
  • keeps Apple fused presses and Linux alternate buttons owner-local
  • retires the superseded dispatcher, series helper, static capability cells, and the unmeasured ordinary iOS direct-selector optimization
  • adds singular cutover rows R47-R51 with runtime, provider, planted-red, and full daemon contract coverage

Part of #1739.

Validation

Exact head 444a98633:

  • pnpm check:affected --run passed: 543 related test files / 4,293 tests plus every runnable format, lint, typecheck, layering, fallow, build/package, Node integration, provider integration, conformance, and structural/model gate
  • the direct-Maestro registry test was observed red before the fix; the completed route now carries a typed internal maestro-direct-selector or maestro-non-hittable-fallback path with no new public response metadata
  • daemon contract coverage proves both outcomes: fallback-used suppresses resolution, while fallback-allowed-but-not-used keeps { source: 'direct-ios', kind: 'not-observed' }
  • eager-closure coverage is exact: the new touch-runtime entry is pinned at 4 modules, its four genuine +1 ancestor closures are recorded, and the retired dispatcher closure is ratcheted down from 94 to 88

Live device evidence from predecessor e809d492c (this follow-up changes only internal path classification, contract tests, and closure pins):

  • iOS simulator: click, press, fill, and longpress committed the expected app-state changes; hover returned the canonical typed refusal
  • Android emulator: the same five command cells passed, including committed app-state assertions and the canonical hover refusal
  • exact-head web remains pending because the managed browser device was live-leased by another session; no lease was stolen

The prior Android Smoke failure occurred after the migrated click successfully opened and exposed the native alert. alert dismiss then closed the dialog without the fixture receiving its cancel callback; current main passed the identical alert flow. This is isolated from touch dispatch and is left to exact-head CI rather than widening this PR.

The preceding size report at e809d492c was +6.2 kB npm unpacked (+1.3 kB gzip), entirely in JS/dist; Apple runner source, macOS helper source, and Android helper artifacts were unchanged. Exact-head Bundle Size and device lanes remain authoritative.

Docs and skills are unchanged because the public command surface and semantics are unchanged.

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.39 MB 2.39 MB +5.5 kB
JS gzip 801.1 kB 802.5 kB +1.3 kB
npm tarball 923.3 kB 925.1 kB +1.7 kB
npm unpacked 3.20 MB 3.21 MB +6.3 kB

npm unpacked components

Component Base Current Diff
JS / dist source 2.54 MB 2.54 MB +6.3 kB
Apple runner source/project 564.3 kB 564.3 kB 0 B
macOS helper source 54.5 kB 54.5 kB 0 B
Android helper artifacts 0 B 0 B 0 B
Other package files 44.5 kB 44.5 kB 0 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 18.6 ms 19.2 ms +0.6 ms
CLI --help 49.6 ms 49.7 ms +0.0 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/interaction.js +5.4 kB +1.4 kB
dist/src/sdk-batch-runner.js +1.0 kB +296 B
dist/src/session-snapshot.js +820 B +154 B
dist/src/runtime2.js +428 B +107 B
dist/src/app-inventory-contract.js +308 B +99 B

Top changed packed files

Packed file Base Current Diff
dist/src/dispatch.js 22.1 kB 14.0 kB -8.1 kB
dist/src/interaction.js 29.8 kB 35.2 kB +5.4 kB
dist/src/input-actions2.js 4.3 kB 0 B -4.3 kB
dist/src/interactor.js 11.0 kB 15.2 kB +4.2 kB
dist/src/linux.js 2.0 kB 5.9 kB +3.9 kB
dist/src/validation.js 3.7 kB 0 B -3.7 kB
dist/src/scroll-edge-state.js 0 B 3.4 kB +3.4 kB
dist/src/touch-runtime.js 0 B 3.3 kB +3.3 kB
dist/src/runner-sequence.js 2.5 kB 0 B -2.5 kB
dist/src/sdk-batch-runner.js 75.7 kB 76.7 kB +1.0 kB

@thymikee

Copy link
Copy Markdown
Member Author

Deep code-quality review

The cutover itself is the right shape and the deletions are real (dispatch-interactions.ts 631 lines, dispatch-series.ts 92 lines, the static capability cells). Two moves are genuinely good: shouldUseIosPressSequence's isApplePlatform test dissolves once the series lives in the Apple owner, and fillElementSelector is correctly deleted as dead code (its only producer was handleDirectElementSelectorFill, reachable only from a directElementSelector context that no route ever set with command: 'fill').

But the daemon-side landing does not follow the pattern the four previous waves established, and the cost is visible in the diff. My main objection is structural, and I think it collapses a lot of this code.


1. (blocker) The admission seam is cast away, and six defensive wrappers are the debt

src/daemon/touch-runtime.ts:56:

const bindSelected = admission.bind as unknown as (
  device: DeviceInfo,
  selectedUse: RuntimeUseDeclaration,
) => Promise<BoundTouchOperations>;   // BoundTouchOperations = { operations: Partial<PlatformRuntimeOperations> }

This is the only as unknown as in daemon production code in the repo. It throws away exactly the guarantee this whole arc exists to provide. BoundDeviceRuntime<Use>.operations is Pick<Ops, Required> & Partial<Pick<Ops, Preferred | Conditional>> — after admission, the required operations are statically non-optional. Downgrading to Partial<PlatformRuntimeOperations> erases that, and then six hand-written guards buy it back at runtime:

async function executeTapPoint(runtime, input) {
  if (!runtime.operations.tapPoint) return requiredOperation('tapPoint');
  return await runtime.operations.tapPoint(input);
}
// ×6, plus requiredOperation(): never

Compare the two prior waves, which needed none of this:

// src/daemon/type-text-runtime.ts
/** What the executor needs: only the bound operation, however broad the bind that carried it. */
type BoundTypeTextOperations = Readonly<{
  operations: Readonly<{ typeText: TypeTextRuntimeOperations['typeText'] }>;
}>;
...
const backendResult = await runtime.operations.typeText(...);   // no guard, no cast

executeFocusPoint does the same with an inline exact type. The named-executor-per-operation convention (which the R42–R46 gate rows encode) is fine and I am not asking you to collapse it — I am asking that each executor declare the narrow required operation type the prior waves declare, at which point four of the six if (!op) guards and requiredOperation() disappear, and captureSnapshot / tapElementSelector keep a legitimate presence check because they are genuinely preferred.

The cast exists because resolveTouchRuntimeUse returns a union of two use types, so the generic bind widens. The fix is already in this directory — admitOpenRuntime in src/daemon/handlers/session-open.ts has the identical problem and solves it by switching on the plan kind and binding a literal use per arm:

switch (plan.kind) {
  case 'open': { const runtime = await bind(params.device, openApplicationRuntimeUse); ... }
  case 'open-apply-runtime-hints': { const runtime = await bind(params.device, openApplicationWithRuntimeHintApplyUse); ... }

with the comment "Only the exact bind differs per plan, so each open use keeps its precise operation projection." admitRuntimeOperations's own doc says the same thing. Please follow it. Related: resolveTouchRuntimeUse returns a bare use where every sibling resolver (resolveSnapshotRuntimePlan, resolveScreenshotRuntimePlan, resolveSelectorCaptureRuntimePlan, resolveDeviceReadinessRuntimePlan) returns a { kind, operation, use } plan — that discriminant is precisely what makes the exact-bind switch typecheck.

2. (blocker) A second capture-intent builder, three files from the one that says it is the only one

src/daemon/snapshot-runtime-capture-input.ts:

The one place a daemon request becomes neutral capture intent. snapshot and diff build theirs here; a repeated-capture consumer builds one per capture from its own effective flags and scope. One builder is what stops those shapes drifting on which flag reaches the platform.

The new captureData closure in interaction-snapshot.ts hand-rolls the same ten-field options bag (appBundleId, interactiveOnly, preferredBackend, depth, scope, raw, customActions, includeHiddenContentHints, includeRects, surface), the same execution: runtimeExecutionFromContext(...), and the same conditional signal spread — just off effectiveFlags instead of flags. This is the exact drift that comment exists to prevent, introduced in the same PR. buildRuntimeCaptureInput needs an overload taking already-effective flags and a resolved context; that is a small extension of a builder that already advertises the use case.

3. (blocker) Ref support is encoded three times, and web asserts a fact it cannot know

One invariant, three channels:

  1. nativeRefs: { tap, hover, fill } passed into bindTouch
  2. re-published as a supportsRefTarget boolean stapled onto the function object via Object.assign (targetOperation)
  3. re-checked as interactor.tapRef presence inside the operation — with a fourth, defensive throw when a ref arrives anyway
if (input.target.kind === 'ref' && interactor.tapRef) return await interactor.tapRef(...);
if (input.target.kind === 'ref') throw new AppError('UNSUPPORTED_OPERATION', 'Bound tap target requires a point...');

copy-pasted across tapPoint, hoverPoint, and fillPoint. This codebase already has a channel for "what can this owner do": RuntimeOperationFact. Capability metadata smuggled on a function property is the kind of magic the facts model was built to replace, and the daemon has to read it back through runtime.operations.tapPoint?.supportsRefTarget === true.

Worse, the union itself is loose: TapPointInput carries options: PressPointOptions that are meaningless when target.kind === 'ref'. tapRef computes and validates a full press series (normalizePressOptions({}, context)) and then discards it. Two operations (tapPoint / tapRef) with disjoint inputs would make that unrepresentable and delete the three copy-pasted ref branches with it.

Concretely wrong today: packages/platform-web/src/runtime.ts declares nativeRefs: { tap: true, hover: true, fill: true } statically, but clickRef/hoverRef/fillRef are all optional on WebProvider and resolveWebProvider() resolves a request-scoped provider that may not have them. Before this PR the gate was derived from the real provider (tapTarget: webProvider?.clickRef ? … : undefined), so a provider without clickRef fell back to a coordinate tap. Now tapTarget is always offered and a missing clickRef throws mid-action. That is a fact the owner is not entitled to assert.

4. The interaction backend became a conditional-spread pyramid, and optional-chains inside its own guard

interaction-runtime.ts went from a flat, uniform member list (tapTarget: webProvider?.clickRef ? … : undefined) to an outer conditional spread containing three nested conditional spreads. Inside it, every body calls params.touchExecutor?.tapPoint(...) — optional-chained inside the branch that already proved it non-null, because the closure loses the narrowing. toBackendActionResult(undefined) returns undefined, so a lost narrowing degrades to a silent no-op tap rather than an error. const executor = params.touchExecutor before the block fixes the chains; keeping the flat member: cond ? fn : undefined shape fixes the pyramid.

5. The Apple move dropped every "why" comment and inlined the magic numbers

chunkPressSteps in src/platforms/apple/interactions.ts:

const stepCost = (step.durationMs ?? 0) + (step.pauseMs ?? 0) + 250;
if (current.length > 0 && (current.length >= maxSteps || cost + stepCost > 20_000)) {

Those two numbers were RUNNER_SEQUENCE_CHUNK_BUDGET_MS and RUNNER_SEQUENCE_STEP_OVERHEAD_MS, each with a doc block explaining that the runner executes a chunk inside one DispatchQueue.main block behind a 30s main-thread watchdog, and that exceeding it makes the runner report a timeout while the remaining steps keep mutating the UI. That rationale is now nowhere in the tree. Same loss for the seven-line header on runIosSequenceChunks (which explained why the aggregate keeps the first chunk's frame and the last chunk's gestureEnd) and the header on remapSequenceErrorStepIndex (chunk-local → global rebasing). The code that survived is precisely the code that needed those comments. Please carry them across.

While there: these ~130 lines of sequence mechanics (runApplePressSeries, buildPressSteps, chunkPressSteps, remapSequenceErrorStepIndex) belong next to buildRunnerSequenceCommand / parseRunnerSequenceResult / MAX_RUNNER_SEQUENCE_STEPS in core/runner/runner-sequence.ts, not in the interactor-overrides module, which grew 354 → 545 lines absorbing them.

6. Duplicated dispatch preamble, and a conditional identity wrapper

interaction-touch-press.ts and interaction-touch-fill.ts each carry the same ~20-line block: resolveBoundTouchRuntimeif (!bound.ok)createBoundTouchExecutor → rebuild boundParams with withBoundInteractionCapture. That is one prepareTouchDispatch(params, session, command, requiresCapture) helper.

And withBoundInteractionCapture is a partial-application wrapper whose only job is to bind a newly added optional 6th positional parameter on CaptureSnapshotForSession, with an identity branch when it is absent. captureSnapshotForSession already takes an options object as its 5th argument — putting boundCapture there removes the 6th parameter, the wrapper, and the boundParams rebuild at both call sites. Better still: dispatchRuntimeInteraction already receives touchExecutor, so the capture override can be applied once where it lands.

7. androidFreshnessBaseline left half-migrated

Moving the refresh into the press handler (so it runs through the bound capture) is right, but the old plumbing was left standing:

  • AdmittedTargetedTouch.androidFreshnessBaseline is still a required field, populated from refAdmission.androidFreshnessBaseline, which is now always undefinedadmitTargetedTouchRef returns only {} or { response }.
  • Its return type still advertises androidFreshnessBaseline?, which it can no longer produce.
  • The target.kind === 'ref' discriminant is now tested in both places, and the : admitted.androidFreshnessBaseline fallback in boundAdmitted is dead.

Drop the field from the admission result and compute it once in the press path.

8. Direct-iOS selector tap bails silently, after a side effect

expireRefFrame(session);          // before the try
try {
  if (!touchExecutor.tapElementSelector) return null;   // "fall back to coordinates"

return null means "the direct path declined, use the tree path" — but the ref frame has already been expired. tapElementSelector is preferred on capturedTapUse, so absence is normal, not exceptional; eligibility should be decided where directSelector is computed (readDirectIosSelectorTapTarget), so the direct path is never entered without the operation and there is one eligibility authority instead of two. Also: with fillElementSelector gone, dispatchDirectIosSelectorInteraction's command: 'press' | 'fill' union is now provably unreachable on the 'fill' arm, along with the always-[] positionals — worth collapsing since the function has a single caller.

9. Three idioms for one conditional bind, and one owner off the injected clock

The same "bind if the fact admits" is spelled three ways across six owners in this diff:

  • Apple: whenAdmitted(facts.operations.tapPoint, () => bindLocalTouchInteractor(...))
  • Android / Linux: bindAvailableLocalTouchInteractor({ fact, ... })new in this PR
  • HarmonyOS / Web / WebDriver: ...(facts.operations.tapPoint.available ? bind…(…) : {})

bindAvailableLocalTouchInteractor is fact.available ? bind(...) : {} — exactly what the generic whenAdmitted already does, minted in the shared contracts package for one command family. Promote whenAdmitted and delete the bespoke one, then all six owners read the same.

Separately, provider-webdriver/src/platform-runtime.ts adds import { setTimeout as sleep } from 'node:timers/promises' for its pause, while every other touch owner threads host.clock.sleep. That makes the inter-press interval the one un-fakeable timer in the family.

10. fill response: a dead conditional over a widened passthrough

...(result ?? {}),
...(result && 'maestroNonHittableCoordinateFallbackUsed' in result &&
    result.maestroNonHittableCoordinateFallbackUsed === true
      ? { maestroNonHittableCoordinateFallbackUsed: true } : {}),

The conditional can only fire when the spread above it already emitted the same key with the same value — it is provably dead. It reads as dead because the leading ...(result ?? {}) is new: the retired leaf emitted x, y, text, delayMs, that one allowlisted flag, readFillBackendResult(result), and successText — an allowlist, not a passthrough. Please confirm the widening to raw-owner-result passthrough is intended and that downstream projection still narrows it; if it is intended, delete the now-redundant conditional.


Smaller

  • ref.replace(/^@/, '') is inlined three times in core/interactors/web.ts while stripAtPrefix is the canonical helper — which this PR removed the import of. Layering blocks core → daemon/handlers, so the fix is to move stripAtPrefix to a shared layer, not to re-implement it (the canonical one also handles undefined).
  • UnavailablePlatformRuntimeFacts.touch? is optional while every sibling (focus, typeText, elementText) is required, so a provider can silently omit it and inherit orNetwork. Why is touch different?
  • WEB_INTERACTION_COMMANDS = ['scroll'] — a one-element list with a plural name; fold it into WEB_SUPPORTED_COMMANDS.

Bar

Not approving as-is. (1), (2), and (3) are the ones I would hold on: each has an in-repo precedent or an explicit doc comment pointing the other way, and (1) in particular is buying back at runtime a guarantee the type system was already handing you for free.

@thymikee

Copy link
Copy Markdown
Member Author

Request changes: this head is not merge-ready. It conflicts with current main/#1955, and the exact-head Coverage run has two deterministic PR-owned failures: find.test.ts is 1,232 lines against its 1,204 pin (extract the scenario; do not repin), and interaction-settle-private-ax-route.test.ts now reaches admission without runtime facts/binding (Device runtime facts inspection is unavailable), so the fixture must provide the request-bound runtime. There are also two owning-interface regressions: (1) resolveBoundTouchRuntime casts admission.bind through unknown into operations typed as Partial<PlatformRuntimeOperations>, discarding the narrowing admission just proved and forcing executor-time required-operation checks; bind a discriminated literal use and return its typed BoundDeviceRuntime<typeof use> instead. (2) the web runtime hardcodes nativeRefs: { tap, hover, fill: true } whenever point tap exists, although WebProvider.clickRef/hoverRef/fillRef are optional; providers missing those methods now claim routes that throw rather than falling back through canonical target resolution. Model those as request-scoped facts/bound operations and test the missing-ref-method fallbacks. Finally, this move-dominated change adds +5.6 kB unpacked and has no requested live proof for the five commands; after simplifying the bind/executor seam, itemize the residual size and provide the linked live-device/cloud validation before merge.

@thymikee
thymikee force-pushed the codex/wave5-touch-runtime branch 2 times, most recently from 284b4b1 to 2f7369d Compare August 24, 2026 09:07
@thymikee

Copy link
Copy Markdown
Member Author

Addressed the review on 2f7369dcb:

  • rebased onto current main / merged refactor: migrate back/home/orientation/tv-remote/keyboard to the request-bound device runtime #1955 and preserved its navigation/keyboard runtime cells; touch cutovers now own unique R47-R51 rows
  • replaced the widened cast/partial runtime seam with discriminated exact plans and operation-specific typed executors
  • routed interaction capture through the canonical capture-input builder and shared touch preparation seam
  • split point/ref operations and made owner facts authoritative, including request-scoped web ref availability and WebDriver no-ref facts
  • flattened the interaction backend, unified admitted binding/host clocks, removed stale freshness plumbing, and kept fill output allowlisted
  • moved Apple sequence mechanics and watchdog rationale into the runner-sequence owner
  • supplied request-bound runtime facts/binding to the private-AX fixture
  • extracted the find touch-runtime setup into a sibling fixture; find.test.ts is 1202 lines and its ratchet pin was lowered from 1204 to preserve the gain

Validation: pnpm check:affected --run passes on the rebased head (542 related test files / 4,084 tests, plus all runnable static, layering, fallow, package, integration, and model gates). The initial sandbox-only ps -axEww EPERM was rerun with process inspection enabled.

Merge-readiness evidence still pending: exact-head GitHub Coverage/Bundle Size and live device/cloud proof for the five commands. This update is published and reported, not a merge-ready claim; I did not wait for CI.

@thymikee

Copy link
Copy Markdown
Member Author

Follow-up pushed in 64967560b:\n\n- expose direct iOS selector tap only when the exact owner binding supplies tapElementSelector; otherwise fall back to capture + point tap\n- remove the unused duplicate operation field from the pure touch-runtime plan\n- narrow hover-ref facts through RuntimeOperationFact instead of a property-existence guard\n- add planted-red coverage for the missing preferred operation (old shape skipped capture; fixed shape falls back correctly)\n\nValidation: pnpm check:affected --run passed (542 files, 4,085 tests; all runnable gates). GitHub CI is pending and I am not waiting for it.

@thymikee

Copy link
Copy Markdown
Member Author

Live evidence + the §9 preferred measurement (the two open items in the PR body)

Ran the two follow-ups this PR names as outstanding, against PR head aceb32a4c, local devices only:
iPhone 17 Pro simulator (iOS 26.x), Pixel_7_review AVD, and the managed agent-browser. Fixture is
examples/test-app on a dev build with Metro; the Android leg is helper-backed
(androidSnapshot.backend = android-helper, helperVersion 0.20.10 = package version), so no
UIAutomator fallback is being mistaken for the path under test. Every session was closed, the
emulator IME was restored, and the AVD/simulator were shut down.

1. Live matrix — all cells pass with committed effects

Verified against committed app state, not command success.

command iOS simulator Pixel_7_review web
click ✅ Home → Catalog (tree changes) ✅ Home → Catalog @ref → example.com (2 nodes) → IANA (33 nodes)
press ✅ registers as tap; longpress counter stays 1 In cart: 0 → 1
press --count 3 In cart: 0 → 3 (fused sequence) In cart: 1 → 4
press --hold-ms 800 ✅ registers as longpress (Long presses: 0 → 1) — the fused hold branch
press --double-tap ✅ dispatched, +1 (fixture Pressable coalesces, so it cannot discriminate)
longpress Last input: longpress, counter 0 → 1 Last input: longpress, counter 0 → 1
fill ✅ ×2 — hidden-keyboard start and shown-keyboard/@ref; text reads back; debounced filter commits 12 results → 0 results ✅ IME-helper path; lamp reads back; filter commits 12 → 0
hover ✅ refusal cell ⚠️ refusal cell, wrong hint — see §3 ✅ both routes

The press/longpress discrimination is the strong one: the same control, pressLast input: tap
with the long-press counter unchanged, longpress → counter increments. press --hold-ms 800 routing
to the longpress branch is the direct live proof of the fused holdMs > 0 step kind that moved into
platform-apple.

Web hover confirms the D2 design end to end: hover @e2 returns Hovered @e2 (no coordinates — the
DOM-handle route chosen inside the web owner) while hover text="Learn more" returns
Hovered text="Learn more" (297, 174) (the point route). One operation, owner-selected route, exactly
as the amended native-ref description claims.

2. §9 measurement: the preferred declaration is not supported by the numbers

Instrument: summed daemon-side phase durations from the --debug request ndjson (CLI wall clock is
unusable here — it swings 86–665 ms for the same nominal path on Node startup alone). Every sample
asserts request_success, and each is tagged with its actual path: cap= counts snapshot_capture,
fb= counts ios_direct_selector_tap_fallback. A/B is click id="…" (direct-eligible) versus the same
click with --count 1, which disqualifies the fast path via hasNonDefaultClickOptions while keeping
the semantics identical (one tap). n = 8 pairs each, alternating.

target direct (cap=0, fb=0) runtime (cap=1, fb=0) delta
Home, ~32 nodes 900 ms median [885–1013] 731 ms median [714–769] +170 ms (+23.2%) slower
Catalog, ~46 nodes 1270 ms median [1232–1366] 1198 ms median [1180–1253] +72 ms (+6.0%) slower

The direct path skips the snapshot capture entirely (cap=0 on every sample, confirmed) and is still
slower on both trees. Its fused runner query costs more than capture-plus-coordinate-tap.

The gap narrows as the tree grows (+23.2% → +6.0%), which is the honest caveat: the fast path's
benefit scales with capture cost, so on a large real-world tree — the Bluesky-class app it was designed
for — it plausibly crosses over. What I can state is that on every target I could measure it is a
regression, so the recorded measurement §9 requires does not exist yet.
Either measure on a
large-tree real app and record the crossover, or retire the declaration per §9 ("a preferred operation
declared without a measurement is speculative surface and is rejected in review").

A second, timing-independent finding sharpens this. React Native text= selectors make the direct path
fall back: ios_direct_selector_tap_fallback with error: "selector matched multiple elements", because
it counts raw XCTest matches before hittability can pick a winner, and RN wraps text so a visually
unique string is several raw elements. The cleanest case is the fixture's inert surface — a 4-node
screen where text="Tapping this text changes nothing on screen." is unique in the AX tree and the
direct path still saw multiple raw matches. Every instrumented attempt on that selector fell back,
and the 12 timed runs were uniformly slow (832–950 ms), consistent with all of them paying the failed
attempt plus the full runtime path. id= selectors on unique accessibility identifiers succeeded
(id="inert-target", id="inert-title", id="home-title", id="catalog-title", all fb=0).

So the fast path helps only for id=-shaped selectors, and on text= — the shape agents most often
author — it is pure added latency before the runtime path does the work anyway.

Caveats: one device, warm runner, one RN fixture, trees of 32/46 nodes.

3. Defect: Android hover refusal emits a focus hint (regression against legacy)

$ agent-device hover 'id="catalog-search"'    # Pixel_7_review
Error (UNSUPPORTED_OPERATION): hover is not supported on this device
Hint: focus is supported on Android emulators and physical devices.

The user asked about hover and is told about focus. Root cause is one line —
packages/platform-android/src/runtime.ts:202 sets hover: focusKindUnavailable, reusing the fact
object built for the retired focus bucket, which carries its own focus-worded hint at line 65.
touchRuntimeOperationFacts defaults the correct text with input.hover.hint ?? HOVER_UNAVAILABLE_HINT,
so an owner that supplies any hint suppresses the right one. Apple, HarmonyOS and Linux pass a
hint-less unavailable and therefore print the correct hover text; Android is the only owner affected.

This is a regression, not a pre-existing wart: on main (checked at a pre-#1987 baseline)
unsupportedHintForDevice('hover', android) returns undefined, so legacy refused with no hint.
The PR replaces "no hint" with "a hint about the wrong command."

Suggested fix: give Android a hover-specific hint-less fact so the contract default applies. Worth
considering making it structural rather than a one-line fix — HOVER_UNAVAILABLE_HINT being a
default is what let an unrelated fact silently win; hover's hint could be set unconditionally on the
unavailable branch, which would make this defect unrepresentable.

Minor, non-blocking: the refusal message moved from "hover is not supported on this platform"
(legacy dispatch) to "…on this device". More accurate under per-device facts — flagging only
because the unit record committed to carrying that string through unchanged.

4. Not verified

fill on web (no suitable field on the pages used), HarmonyOS (no nova 14 attached), and Vega
(fact-only refusal cells). The --double-tap cell dispatched but the fixture cannot discriminate it.

@thymikee

Copy link
Copy Markdown
Member Author

Request changes at 64967560b. The prior bind-narrowing, Apple sequence-owner, file-size, conflict, and Coverage blockers are fixed. One P1 ownership bug remains: web runtime facts are derived from createWebInteractor() wrapper methods, but that wrapper always exposes tapRef, hoverRef, and fillRef and only discovers missing optional WebProvider.clickRef/hoverRef/fillRef when invoked; point hover is also advertised for every browser device although WebProvider.hover is optional. Sparse scoped providers therefore claim operations that later throw UNSUPPORTED_OPERATION, and ref commands can select the native-ref route instead of canonical resolution. Snapshot/bind the actual scoped-provider capabilities at the web owner boundary (or omit unavailable wrapper methods), and add sparse-provider cases for clickRef/hoverRef/fillRef/hover facts and bound operations. The redesign otherwise fixes the prior type-proof issues, but the PR still needs an explicit +6.4 kB unpacked-size disposition and representative live iOS/Android/web evidence for all five migrated commands, including the linked cloud-fill risk, before merge-ready status. One iOS Smoke job remains pending.

@thymikee

Copy link
Copy Markdown
Member Author

Addressed the remaining review on 11a12b910:

  • web now snapshots the actual request-scoped provider and omits unavailable optional methods; runtime facts and bindings therefore agree for clickRef, hover, hoverRef, and fillRef, with planted-red sparse-provider coverage
  • Android hover now supplies a hint-less hover-specific unavailable fact, allowing the canonical hover hint instead of leaking the focus hint
  • retired the unmeasured direct-selector optimization for ordinary iOS clicks: tapElementSelector is no longer preferred; it is fact-conditional and eligible only for the explicit Maestro non-hittable fallback semantic contract. Ordinary selectors always use capture + point dispatch. The obsolete direct-path guarantee/contract and fallback suites were removed; Maestro response/error/seam coverage remains

Size disposition: the reported +6.4 kB unpacked delta is the typed Wave 5 runtime/fact/binding seam (Apple runner/helper source remains 0 B), not justification for the speculative selector optimization. This follow-up removes 758 net lines and the separate ordinary-click path; the authoritative packed-size recalculation is the PR size job.

Validation: pnpm check:affected --run passes (541 related files / 4,069 tests, plus all runnable static, layering, fallow, package, integration, conformance, and model gates). The sparse web and hover regressions were observed red before the fixes.

Live evidence remains the linked aceb32a4c iOS/Android/web matrix; I did not relabel it as exact-head evidence. This head changes sparse-provider admission, hover refusal metadata, and ordinary iOS selector route selection, all covered locally above. An exact-head live rerun is still separate from this code-review update; I did not wait for CI.

@thymikee

Copy link
Copy Markdown
Member Author

Addressed the remaining guarantee blocker in e809d492c: the Maestro non-hittable fallback path now truthfully declares its reachable direct click command. I added a focused registry regression, observed it fail before the declaration change, and the registry/direct-iOS/Maestro contract suite now passes 17/17; the full affected gate passes 541 files / 4,070 tests.\n\nThe PR description is refreshed with exact-head iOS/Android results and the explicit +6.2 kB unpacked-size rationale. The exact-head web leg is transparently still pending because agent-browser-chrome is live-leased by another worktree/session (w1987); I did not steal it. I pushed the head and did not wait for CI.

@thymikee

Copy link
Copy Markdown
Member Author

Still not merge-ready at e809d492c. Adding click to maestro-non-hittable-fallback does not truthfully classify the reachable route: dispatchDirectIosSelectorTap can complete a direct selector tap with maestroCoordinateFallbackDispatched: false, while the registry itself says allowed-but-not-taken is the direct path. Restore a distinct narrow direct-Maestro click path/cell and cover both fallback-used and fallback-not-used outcomes, or remove the reachable direct route; array-membership coverage alone does not prove path selection.

Exact-head CI also needs owner action: Coverage has five eager-closure budget failures, and Android Smoke fails in the alert-cancellation interaction flow. The exact-head web live leg remains pending.

@thymikee
thymikee force-pushed the codex/wave5-touch-runtime branch from e809d49 to 444a986 Compare August 24, 2026 12:26
@thymikee

Copy link
Copy Markdown
Member Author

Addressed at 444a98633 after rebasing onto current main.

  • restored a distinct narrow maestro-direct-selector registry cell for fallback-allowed-but-not-used clicks
  • replaced the ambiguous boolean with a typed internal dispatch path selected from the runner outcome; only maestro-non-hittable-fallback suppresses resolution
  • split direct-selector and coordinate-fallback daemon contracts into their matching source topology and covered both outcomes
  • fixed all five eager-closure failures with measured exact pins, including ratcheting the retired dispatcher from 94 to 88 modules

The Android artifact shows the migrated click succeeded, the alert was visible through wait/snapshot/get, and the later alert dismiss closed it without the fixture callback. Current main passed the identical flow, so I did not fold unrelated alert behavior into this runtime cutover.

pnpm check:affected --run passes on the exact head: 543 related files / 4,293 tests and every runnable local gate. Exact-head GitHub lanes and the web live leg remain pending; I did not wait for CI or merge.

@thymikee

Copy link
Copy Markdown
Member Author

Code review is clean at 444a98633. The dedicated maestro-direct-selector path fixes the prior ownership bug: dispatch classification now follows the runner’s actual used outcome, and separate daemon contracts cover fallback-used versus fallback-allowed-but-not-used. The eager-closure failures are also resolved and Android Smoke is green.

Not yet merge-ready: the exact-head iOS Smoke job is still running, and the PR still records the exact-head web live leg as pending after the sparse-provider admission changes. Finish those two evidence items before merge; no further code finding from this review.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Aug 24, 2026
@thymikee
thymikee merged commit 759f175 into main Aug 24, 2026
27 of 29 checks passed
@thymikee
thymikee deleted the codex/wave5-touch-runtime branch August 24, 2026 12:54
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-24 12:55 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant