Skip to content

fix(android): publish covered state from exact order evidence - #1981

Open
thymikee wants to merge 4 commits into
mainfrom
fix/android-occlusion-single-owner
Open

fix(android): publish covered state from exact order evidence#1981
thymikee wants to merge 4 commits into
mainfrom
fix/android-occlusion-single-owner

Conversation

@thymikee

@thymikee thymikee commented Aug 23, 2026

Copy link
Copy Markdown
Member

Summary

Centralize Android covered-state publication without guessing z-order.

  • Carry every snapshotAndroid result in one opaque Android capture envelope so exact clickability and sibling-order evidence cannot be detached from its nodes.
  • Use one publication adapter for the normal interactor, blocking-dialog recovery, and alert get/wait/accept/dismiss routes. buildSnapshotState remains the sole owner of daemon-only interactionBlocked: "covered" publication.
  • Classify replacement surfaces with bounded multi-rect union coverage, so several higher-order actionable descendants may collectively cover a stale surface while sparse overlays remain actionable.
  • Keep exact ordering evidence private. Public snapshot metadata is unchanged.
  • Fail conservative when exact order evidence or work budget is unavailable. API 23 retains androidSnapshot.occlusionScanUnavailable: true and publishes no covered verdict.
  • Preserve the consumer contract: interaction targeting and settle refuse covered targets, response refs omit them, and replay divergence excludes them unless the whole meaningful screen is covered.

This advances #1832 but does not claim API-level neutrality: API 23 cannot provide the exact sibling order required for a reliable covered-state verdict.

Validation

  • Planted red/green at the publication seam: removing the adapter evidence handoff made blocking-dialog recovery select a collectively covered stale Close app action and made alert get/accept select a covered candidate. Both direct routes now fail closed.
  • Planted red/green at the classifier: two higher-order actionable descendants covering separate halves of a stale surface were previously reduced to one rect; bounded union coverage now classifies the stale surface.
  • Production consumer inventory: all three direct snapshotAndroid consumers—normal interaction capture, blocking-dialog recovery, and alert routing—use the single publication adapter. No raw-node production bypass remains.
  • Focused ownership-route suite: 9 files / 63 tests passed. The six timeout-shaped failures from the first contended full run passed in isolation: 6 files / 67 tests.
  • pnpm check:affected --run: all runnable checks passed on the clean rerun, including the related suite and test-size ratchet.
  • Test-size ratchets remain restored: snapshot.test.ts is 1,495 lines; the replay case lives in its focused sibling owner rather than growing the legacy aggregation.
  • Historical live corpus inventory from fix(android): stop empty focusable overlays from hiding app content #1737: 33 API 36/37 screens across Telegram, the repository fixture, Bluesky/dev launcher, Settings, Clock, Files, Messages, Play Store, Photos, Chrome, Calendar, Contacts, alerts, and permission dialogs. The retired pruner fired on 4/33: three Telegram false positives and one legitimate Google Messages stale-surface prune; the other 30 screens preserved identical output. The original raw captures were not retained, so this is inventory evidence rather than a reproducible artifact corpus.
  • Prior fresh Pixel API 36 classifier-route verification on Pixel_9_Pro_XL: a lower actionable 1128x720 sibling covered by two exact-higher-order actionable 1128x360 descendants remained present but published hittable: false, interactionBlocked: "covered", and presentationHints: ["covered"]; the foreground actions remained hittable, and no sibling/order evidence leaked publicly. Settings, Google Messages, and repository-fixture routes produced no false covered verdicts. No Android device was booted or connected for an exact-head rerun of the final adapter-only follow-up.
  • API 23 conservative behavior is covered deterministically with API-23-shaped captures that omit drawing order. No API 23 device is available locally, so live API 23 verification remains unavailable.

@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.40 MB 2.40 MB +1.8 kB
JS gzip 805.3 kB 806.0 kB +776 B
npm tarball 929.6 kB 930.1 kB +510 B
npm unpacked 3.22 MB 3.23 MB +1.6 kB

npm unpacked components

Component Base Current Diff
JS / dist source 2.55 MB 2.56 MB +1.8 kB
Apple runner source/project 570.6 kB 570.6 kB 0 B
macOS helper source 54.5 kB 54.5 kB 0 B
Android helper artifacts 0 B 0 B 0 B
Other package files 44.7 kB 44.5 kB -236 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.6 ms 27.8 ms +0.2 ms
CLI --help 79.7 ms 81.2 ms +1.5 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/session-snapshot.js +3.7 kB +1.2 kB
dist/src/snapshot.js -2.2 kB -669 B
dist/src/session2.js +43 B +10 B
dist/src/runtime.js -48 B +6 B
dist/src/internal/daemon.js -8 B +3 B

Top changed packed files

Packed file Base Current Diff
dist/src/android-system-chrome.js 401 B 4.2 kB +3.8 kB
dist/src/session-snapshot.js 34.9 kB 38.6 kB +3.7 kB
dist/src/snapshot-chrome.js 7.9 kB 5.1 kB -2.8 kB
dist/src/snapshot.js 31.8 kB 29.5 kB -2.2 kB
dist/src/snapshot-capture-annotations.js 843 B 0 B -843 B
dist/src/snapshot-private-evidence.js 0 B 365 B +365 B
package.json 18.5 kB 18.2 kB -236 B
dist/src/android.js 16.9 kB 16.7 kB -189 B
dist/src/snapshot-clickability.js 164 B 0 B -164 B
dist/src/alert-detection.js 3.9 kB 4.0 kB +123 B

@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-1981/

Built to branch gh-pages at 2026-08-24 15:58 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@thymikee thymikee changed the title fix(android): unify snapshot occlusion across API levels fix(android): publish covered state from exact order evidence Aug 23, 2026
@thymikee

Copy link
Copy Markdown
Member Author

Not ready at 91281d2c:

  1. The migration drops the old classifier’s multi-rect union semantics. android-replacement-surface-occlusion.ts reduces a covering subtree to one strongestPaint rect, whereas the prior owner measured the union of every paint rect. Two higher-order actionable descendants can collectively cover a stale surface >=90% while each alone covers <90%, leaving the stale action hittable. Restore bounded union coverage and add a planted regression for collective descendant coverage.
  2. Coverage is deterministically red: the test-file-size ratchet reports growth in src/platforms/android/__tests__/snapshot.test.ts (1495→1542) and src/daemon/handlers/__tests__/session-replay-divergence.test.ts (1215→1218). Move the new scoped evidence/publication case to a focused sibling test owner and avoid growing the replay aggregation.
  3. Before readiness, attach the Android native-a11y → snapshot: conform to the acquire/present engine contracts (sibling of #1797) #1832-required 33-screen true-positive corpus/consumer inventory and live Pixel evidence for this changed path. Current fixture evidence is insufficient for removing the platform pruner.

The private evidence route and size impact otherwise look sound.

@thymikee
thymikee force-pushed the fix/android-occlusion-single-owner branch from 91281d2 to 4275009 Compare August 24, 2026 10:15
@thymikee

Copy link
Copy Markdown
Member Author

Not ready at 42750091b: the previous union, ratchet, exact-order, API 23, and Pixel-evidence findings are resolved, but two shipped consumers still bypass the sole-owner occlusion publication. android-system-dialog.ts rebuilds { nodes, backend } from snapshotAndroid, dropping the private WeakMap sidecar before buildSnapshotState, so covered replacement actions are never classified there. platforms/android/alert.ts passes raw pre-publication nodes directly to findAndroidAlertCandidate, losing the retired platform prune without gaining daemon classification. Route both through an evidence-preserving publication seam (or explicitly prove/remove their dependency) and add regressions that fail when the sidecar is dropped. The red iOS smoke is unrelated to this Android diff and also fails on current main; it is not this blocker.

@thymikee
thymikee force-pushed the fix/android-occlusion-single-owner branch from 4275009 to 7339722 Compare August 24, 2026 15:57
@thymikee

Copy link
Copy Markdown
Member Author

Addressed the remaining ownership-path blocker in 73397228e.

All three production snapshot consumers now receive an opaque Android capture and cross one evidence-preserving publication adapter before daemon classification. Direct regressions cover collectively covered blocking-dialog actions and covered alert get/accept candidates; removing the adapter handoff makes both tests fail.

The clean final pnpm check:affected --run passed. Exact-head Pixel rerun is explicitly marked unavailable because no Android device is currently connected; prior API 36 classifier-route evidence remains distinguished in the PR description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant