feat: frame-to-select, tree navigation, Escape, and inline icon actions

[angusbezzina]

Implements epic VRT-cne0. Two user-requested changes to the annotation overlay.

1. Exit annotate mode with an X

The strikethrough-pencil toggle is replaced by a Lucide X on the far right of the pill — the conventional "close this mode" position.

Idle is unchanged (a lone pencil). Annotating is now point · frame │ copy · export · clear · ✕.

The X is deliberately not gated on hasNotes, unlike copy/export/clear: with zero notes every other control is inert, so gating the exit would leave a row of dead buttons with no live way out. pencilOff and the then-unused PillButton.isActive were deleted.

2. Draw a frame to select an element

Press-drag a rectangle around what you mean; the note binds to the element that frame best identifies. This exists because hitting the one pixel that resolves to a composite component (a card, a row, a cluster) is a game of chance.

Frame selection is an explicit MODE, not an inferred gesture. The pill carries a two-button tool selector with the active tool lit:

• Point mode (default) — clicks select, exactly as before. A drag is a sloppy click and selects where you pressed. No press can produce a frame.
• Frame mode — drags draw a band and resolve; a plain click does nothing, and the pointer becomes .rectSelection so the inert click is self-explanatory before you make it.

An earlier revision branched implicitly on drag distance. That was replaced because a jittery click could cross the threshold and silently plant a framed note bound to whatever container the pointer happened to sit in. The threshold survives with a narrower job: telling a real drag from a wobble inside frame mode.

How a frame resolves

A pure, platform-free rule (MarqueeTargetRule), two passes over the drawn rect:

1. Surrounded — candidates the frame covers to ≥85% of their own area; the largest wins.
2. Enclosing — nothing surrounded, so the frame was drawn inside something; the tightest container wins.
3. Neither — falls back to the existing region-anchor path.

Largest-wins is the deliberate inverse of the point rule's deepest-wins: a click means "this exact spot", a box drawn around a card means "I mean this whole thing", so it must ascend past the labels the box also swallowed. 85% rather than strict containment because hand-drawn rects clip corners, and demoting those would bind the note to the enclosing panel — the exact failure the feature removes.

Ties break on seeded-beats-unseeded, which exists for one real pattern: .axCardSurface(id) hangs a card's identifier on a Color.clear background leaf exactly coextensive with the card's content group. Both get surrounded equally; only the seeded one locates code.

The drawn rect rides along on the note as regionRect (element-relative, additive/optional — old files decode to nil), relativized at capture rather than selection so pressing Widen after framing re-measures it against whatever element the note finally names.

Fixes a shipped bug

The widening ladder was topped by AnnotKit's own overlay window on every selection (#Spec.Card → #Spec.Section → #com.annotkit.overlay-window). The geometric container scan climbs to AXApplication, whose children are the app's windows, including our overlay panel — which carries an identifier and encloses every point in the host. Widening enough times bound the user's note to the annotation tool itself, and the note looked plausible rather than failing loudly.

Pre-existing on the click path; latent because nothing asserted ladder contents against a live tree. Fixed and pinned by a permanent probe assertion. (VRT-cne0.9)

Verification

• Clean build, zero warnings
• 120 tests (74 baseline + 46 new)
• AnnotKitOverlayProbe 8/8 phases against a live accessibility tree, including a new Phase 7 proving a frame around a seeded card binds to the card, not its text child
• iOS module type-checks clean against the iPhoneOS SDK in Swift 6 mode

Not yet verified — needs a human (VRT-cne0.10)

• Dogfood: 10 framed annotations on real VirgilHUD components, agent locating ≥9
• Real mouse: release outside the window, right-/modifier-clicks, band repaint with many pins
• iOS on device: the port is compiled, never run. The 10pt touch threshold is a reasoned default, not measured.

Follow-ups filed: VRT-ahfa (iOS coordinate double-conversion, latent until a window is offset), VRT-sted (iOS tap-widen vs frame-widen ladder asymmetry).

🤖 Generated with Claude Code

[angusbezzina]

Update: epic VRT-mijf folded in (selection navigation + frame-mode anchoring)

Two issues surfaced by dogfooding the original commits are now fixed on this branch, rather than stacked as a second PR that would conflict on the same lines.

Select Parent / Select Child replace "Widen"

Widen was one-way — overshoot the level you wanted and there was no way back, and no way to reach a child at all — and the name described the mechanism ("make the area bigger") rather than the act (choose which component the note is filed against).

Navigation is now bidirectional over a selectionPath. Descending prefers history (walk back down what you climbed) and only queries the source for children at the deepest known rung, prepending the child so a re-ascent returns to the target and a re-descent replays to the same child. Re-running the heuristic would make the same key produce different results on consecutive presses under a live UI. Both buttons are always present and disable at the range ends — the old button appeared and disappeared, which is much of why it was unlearnable.

This fixed a silent corruption prepending would otherwise have introduced. The note's component — the string an agent greps to find your view — was derived as "the rung above the target". That held only while the path was strictly upward, because every upward rung is seeded by construction. Insert a child at index 0 and index 1 becomes the original target, which may be unseeded — and an unseeded Element.id is a slash-joined path. The note would have exported AXWindow[0]/AXGroup[0] as the thing to grep for and looked entirely plausible doing it. It now skips to the first seeded rung above the bound one, pinned by a mutation-verified test.

Frame mode shows only the frame, and the note ties to it

Frame mode was still hover-highlighting whatever the pointer passed over: hover() gated only on annotate mode and knew nothing about the selection tool. The gate is now in the session, so it is unit-testable, no UI path can reintroduce it, and it removes a cross-process AX hit-test per pointer-motion event. setTool also clears a standing hover, fixing a stranded highlight when switching point → frame.

The drawn rectangle is now the visual anchor for the highlight, composer and pin (solid once committed, dashed while drawing, no element name tag). The resolved element is named in the composer header as Frame → <element> rather than drawn, so a note is never captured against a target the user could not see without putting a second rectangle on screen. Pressing Parent/Child reveals the bound element and dims the frame — that press is the question "which element?", so the answer has to become visible.

Also fixed

A reachable degenerate-drag bug: a perfectly horizontal drag clears the gesture threshold but yields a zero-height rect, and the rejection ran after the per-selection clears — so a stray horizontal drag stripped an open frame selection's anchor, re-anchoring a composer mid-typing, and did it without assigning selected, so the re-render never fired and the overlay kept stale geometry. Mutation-verified test.

Verification

Clean build (zero warnings) · 134 tests · AnnotKitOverlayProbe 9/9 phases against a live accessibility tree, including a new Phase 8 covering climb/descend round-trips, history-not-re-query, the component fix, frame-mode hover inertness, and frame anchoring · iOS module type-checks clean against the iPhoneOS SDK in Swift 6.

The probe fixtures needed extending: the existing ones are AX leaves or children: .ignore surfaces, so the descent assertions would have been vacuously green against them.

Still needs a human (VRT-mijf.4)

Dogfood on VirgilHUD; whether the seeded-only parent chain skips structural levels a user can see (recorded as unresolved in DECISIONS.md); whether the header alone is enough when a frame and its bound element disagree; and the visual treatments, which have no headless witness. Nothing here has run on iOS — that is a verification gap, documented as such in PARITY.md, not a capability gap.

Follow-up filed: VRT-bjtp — an unseeded Element.id differs between snapshot() and the hit-test paths (pre-existing; a second independent argument for the component fix above).

[angusbezzina]

Update 2: epic VRT-dp47 folded in (Escape, pill visibility, inline icon actions)

Three more dogfooding issues, fixed on this branch. Two of them turned out to be worse than reported.

Escape backs out one level

Escape previously did almost nothing, and could not be made to work where it lived: .onExitCommand needs the overlay panel to be key, and the panel is only key while a card's text field has focus. In annotate mode with nothing open — the state you'd want to leave — the host window is key and no panel-scoped handler ever sees the keystroke.

It's now an NSEvent local monitor (correct because AnnotKit is in-process with its host), driven by a pure EscapeRule: cancel an in-flight frame drag → else close the open card → else exit annotate mode → else pass through untouched. A handled Escape is consumed, so the host never double-acts on one press. The composer draft now clears whenever the composer closes for any reason, not just via the Cancel button — otherwise dismissing with Escape left the draft to reappear over the next selection.

The pill was not "hidden" — it was unreachable

Reproduced in the probe before any fix: AppKit's own windowNumber(at: pillCentre) returned 0. Cause confirmed — constrainFrameRect pins a window's top below the menu bar but never lifts its bottom, so a host taller than the display hangs below visibleFrame, and both modes anchor the toolbar exactly there. The Dock alone (61pt) is enough.

Placement now clamps to the screen's visible frame, and axOrigin/surfaceSize are re-derived from the clamped panel, not the host. That second half matters: probe sub-phase 9d shows a host-derived origin would query the wrong point and resolve to nil on a menu-bar-tucked window — a fix that made the pill reachable while breaking every hit-test would have been worse than the bug.

Scroll-while-annotating was also dead. Measured: a wheel over the catcher reached the panel and died at NSWindow's default, so the host received nothing — you could not scroll to anything below the fold. The panel now forwards to the host.

Two other candidates were ruled out with evidence: scroll-driven relayout (scrolling moves a content offset, not the window frame) and the host auto-picker (wrong window would show the pill elsewhere, not nowhere).

Real elliptical arcs, then inline icon actions

The glyph parser drew every SVG arc as a straight line to its endpoint — fine for the pencil's corner rounds, fatal for undo-2, whose 5.5-radius loop is the icon. It now implements SVG 1.1 F.6.5 endpoint→centre conversion emitting ≤90° cubics, with rotation, both flags, and the spec's degenerate cases.

That work was cross-checked against an independent implementation of the spec before any expected value was pinned, and mutation-verified: 10 of 18 arc tests fail against the old parser. The undo-2 test has teeth — a bounding-box check alone would survive flattening (a chord shares its endpoints), so it asserts the chord midpoints sit >1.5 units off the drawn curve.

Both cards now show one inline row of icon-only actions (composer ↑ ↓ … ↺ ➤, editor 🗑 … ➤), each with a tooltip and an accessibility label — icon-only controls are unusable to VoiceOver with a tooltip alone. send keeps its prominence through accent colour rather than a filled label. PillButton was generalised to IconButton(palette:) so the pill and the cards share one implementation; the pill's palette restates its previous literals exactly, so it is unchanged.

Side effect worth knowing: three shipped glyphs now render their true curves for the first time. The pencil is the visible one — its eraser end was an authored round cap the old parser flattened, so it moves ~2.8 units and finally looks rounded.

Verification

Ran the exact three commands CI runs, plus the probe: swift build -Xswiftc -warnings-as-errors (zero warnings) · swift test — 182 tests · iOS simulator cross-compile · AnnotKitOverlayProbe 10/10 phases, including the new Phase 9 that reproduces the pill bug on a host hanging off the visible screen.

Still needs a human

Glyph legibility at 16pt (send is a stroked dart; undo-2's loop is mathematically confirmed but "reads as undo" is a visual call), whether accent-tinted send still reads as primary under a non-blue system accent (graphite is close to .secondary), real trackpad scrolling and momentum through the new forward, and multi-display / Dock-position clamping. Escape's central claim — that the keystroke reaches the monitor while the host is key — is the one thing no headless test can assert.