feat(3ds): pocket term — remote terminal multiplexer over the svc wire - #345
Draft
doodlewind wants to merge 12 commits into
Draft
feat(3ds): pocket term — remote terminal multiplexer over the svc wire#345doodlewind wants to merge 12 commits into
doodlewind wants to merge 12 commits into
Conversation
Bring the host service channel (spec ops 30..32) to the 3DS and ship its first consumer: a terminal multiplexer whose Mac companion holds the PTYs and one authoritative @xterm/headless core per session (the zhongduan authority/replica shape), streaming full grid snapshots on attach and ordered row diffs after, inside gen/seq fences that degrade a lossy device queue into a resync instead of a wrong screen. Native: hosts/3ds/src/svcwire.c speaks the SVC WIRE (PKNT) protocol — UDP-8621 beacon discovery (sdmc:/pocketjs/host.txt override), one TCP connection, ctrl JSON lines — as a non-blocking frame-pumped state machine in the devserver.c shape; soc.c makes socInit shared between the dev wire and the svc transport; capture builds compile the transport to inert stubs so goldens stay deterministic. Guest: apps/term renders a 57x17 cell grid on the top screen (12 px mono snapped to an integer 7 px advance via negative tracking, host-resolved SGR colors, translucent cursor) and puts session tabs plus a five-row touch keyboard with one-shot Shift/Ctrl on the touch screen. D-pad repeats arrows, L/R switch sessions, START is ^C, SELECT opens a session, the circle pad scrubs scrollback. Companion: apps/term/host/serve.ts (run with Node >= 23.6 — under Bun this machine's node-pty never execs the child; its spawn-helper blocks forever in the slave-reattach open) with dependency-free wire/grid/keys modules unit-tested at the root. The PKNT listener falls back to an ephemeral port when 8622 is taken and the beacon advertises the actual port, so several companions share one machine. Verified: Azahar end-to-end over real sockets (beacon -> handshake -> snapshot -> colored diffs -> session switching -> hot-push reconnect), new term goldens plus the existing 3ds-demo set byte-exact, 12/12 test stages, tsc gate green. On current hardware the guest hot-pushes and shows its no-companion screen; live networking needs one native reflash of the rebuilt .3dsx. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
socInit can plausibly fail transiently when the app launches while WiFi is still re-associating (e.g. right after ftpd exits), and both the SOC latch and the one-shot devserver_init treated any such hiccup as final for the process: the guest would boot to its connect screen but stay network-less until relaunched. soc_ensure now retries behind a 3 s cooldown instead of latching (a genuinely stackless environment costs one cheap check per window), svcwire_open drops its own permanent latch and leans on that cooldown, and the main loop re-runs a failed devserver_init every ~5 s until the dev wire comes up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
Hardware-verified. After reflashing the rebuilt Two deployment lessons landed as follow-ups in the branch:
|
Four things the console demo was missing, and the desktop half of the companion that two of them needed. **A read-only desktop window per session.** apps/term-mirror is the console's top screen and nothing else — the same grid component (apps/term/grid.tsx, extracted for this), the same store, the same protocol, drawn by PocketJS through gpui instead of the PICA200. The companion opens one window per session and closes it with the session. `role: "mirror"` makes the daemon refuse input from that connection and never resize its PTY, so a window that only watches cannot disturb the console. hosts/desktop is the stock host of macos-app AND linux-app, so the window follows the console to Linux. **hosts/desktop/src/net.rs** is what makes that possible: the desktop host now speaks the same SVC WIRE (PKNT) transport the console does, over `--svc-connect host:port`, reusing the codec already linked in (pocketjs_core::wire). Threaded like hosts/vita/src/net.rs so the fixed-step tick never blocks; when a wire is present it owns the svc queues and the in-process editor dialect sees an empty drain. **CJK, delivered at runtime.** The console bakes atlases from the app's own string literals, so a session printing 你好 had nothing to draw with. The companion now rasterizes what the screen actually shows — the repo's own bakeSlot over a system face, lifted out of a .ttc since every CJK font macOS ships is a collection — into a FONT ATLAS for the spare slot 19, and streams it as paced base64 chunks the guest loads through the spec `loadFontAtlas` op. This is the note widget's runtime-coverage design (docs/WIDGET.md) with the rasterizer moved to the companion, because the console has neither a font file nor a rasterizer. Advances are rewritten to whole cells so mixed CJK and ASCII stay on the grid, and the terminal's zero-width continuation columns are skipped rather than spaced. **Closable tabs and a held modifier.** The active tab carries an ×; ZL is Ctrl for as long as it is down, with the on-screen keyboard's Ctrl cap lit from the same state. ZL needed native work: it is New-3DS-only and arrives through ir:rst, not the HID pad, so input.c brings that service up and BTN.ZL/ZR take two bits the PSP never assigned (an Old 3DS simply never sets them, and L held is the same modifier there). Also: a reconnecting console names the session it was on, so it comes back to what it left; ctrl reaches single-byte named keys (ctrl+space is NUL); and bake-font.ts imports opentype.js in a way Node's ESM loader accepts, which is what lets a companion bake glyphs at all. Verified on hardware and on the desktop: the console and a mirror window showing the same session and the same CJK line at the same moment, two sessions opening two windows, and the atlas re-sent after a hot push (the native transport outlives the guest, so a hello means "loaded nothing", not "new socket" — the bug that first showed as blanks). 12/12 test stages, 18/18 3DS goldens byte-exact, contract drift guard green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
Round 2: mirror windows, runtime CJK, closable tabs, held ZLA read-only desktop window per session. That needed the desktop host to speak the wire: CJK without a font on the device. The console bakes atlases from the app's own string literals, so a session printing 你好 had nothing to draw with. The companion now rasterizes what is actually on screen — the repo's own Closable tabs (× on the active tab) and ZL as a held Ctrl, with the on-screen keyboard's Ctrl cap lit from the same state. ZL needed native work: it is New-3DS-only and arrives through ir:rst, not the HID pad, so Verified
Needs a reflashZL is native. |
A terminal running a coding agent draws its interface with ⏺ ⏵ ⎿ ✳, and the companion had exactly one face to bake from. Anything that face lacked became the alignment placeholder — a screen of question marks where the agent's transcript markers belong. No single font on a Mac covers a terminal, which is the actual finding here: the mono face has ❯ and the box drawing but no CJK, a CJK face has neither ⏺ nor ⎿, ⎿ lives in Apple Symbols, and ⏺/⏵ have real outlines only in a math face (elsewhere they are colour-bitmap emoji, which rasterize to blanks). So the companion now keeps a CHAIN of faces, one atlas each in slots 19..23, and routes a codepoint to the first face that both maps it and has outlines for it. A run carries its slot on the wire; the placeholder is now reserved for codepoints nobody can draw. Two things fall out of holding several faces at once: - The px is searched down until a cell fits both one terminal row AND the narrowest column span in that atlas. A proportional face's ⏺ is as wide as it is tall, and at row height in a one-column cell it painted over its neighbour. Symbol atlases land at 9-11px, the CJK atlas stays at 13. - A run's width in columns is no longer its character count, since one atlas can hold both one- and two-column glyphs, so the run carries its span when the two differ. Verified on hardware: ⏺ ⏵ ⎿ ✳ ❯ 你好 ✓ ✗ → all render, from five different files, aligned on the grid. 27 unit tests, 12/12 stages, 18/18 goldens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…face Two rendering faults a coding agent's own interface exposed. **A question mark beside the prompt.** Claude Code pads its input box with U+00A0. A space glyph has no outline, so the fallback chain's "has outlines" test — there to keep colour-bitmap emoji from baking blank cells — rejected every face and fell through to the alignment placeholder. The other Unicode spaces now resolve to an ordinary blank at the point cells are read, before anything tries to find a font for them. Blank runs also count their columns rather than one each, because the ideographic space is two wide and counting it as one pulls the rest of the row a column left. **A shattered logo.** Block elements have to FILL their cell to tile with their neighbours, and a runtime atlas cannot promise that: it holds a proportional face scaled down until its widest glyph fits one column, which turns a solid block into a smaller rectangle and a drawn logo into rubble. The complete box-drawing and block-element ranges are now in the app's baked literal, so they come from the device's own monospace face at the size the grid was measured for. The mono atlas goes from 177 to 304 glyphs. Which exposed a third fault, caught by the test written to pin the new invariant: TERM_GLYPHS is both the compiler's charset and the companion's idea of what the device can draw, so a codepoint listed there that the mono face does NOT map is dropped at bake time and then never routed to a fallback either — it renders as tofu with no second chance. ⌐ and ✔ had been in that state all along; they belong to the chain. Also: the companion logs each codepoint no face can draw, so a placeholder on screen can be traced to a character instead of guessed at. Verified on hardware against the reference: the logo tiles solid, ⚠ ▶▶ ❯ and a line of Chinese all render, no placeholders and no tofu. 29 unit tests, 12/12 stages, 18/18 goldens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"POCKET TERM" is wider than the 92 px the host name was pinned at, so the title painted over it. The title is a prop — the console says POCKET TERM, a desktop mirror window says MIRROR — so the column after it is now measured in the slot the title is actually drawn in, and the host name is bounded on the right so a long one cannot reach the grid size and the connection dot. The bar clips its own overflow, which is what keeps a bounded run of text from spilling out of a 14 px strip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI runs `cargo fmt --check` and `cargo clippy -- -D warnings` against hosts/desktop, and net.rs was written without either. Formatting is rustfmt's; the two lints are real tidying — a try_recv loop that is a `while let`, and a nested `if` that collapses into a let-chain. The `connected` flag went with them: nothing has read it since the accessor was dropped, and an unread atomic in a networking module reads like state that matters. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A mirror window is a full replica of a terminal that is drawing a live shell; refusing its keyboard made it a screenshot. It types now, and the console and the window are typing into the same PTY, seeing the same echo. The plumbing already existed on both ends and only the gate was missing. `hosts/desktop` forwards a window's input as svc lines, but only for the note editor and the System UI shell; a window whose companion is reached over the wire now forwards too — the app on the far end is the only thing that knows what a keystroke means, and a window that can only watch is not much of a window. The replica relays those lines to the companion, which owns the PTY, so nothing is echoed locally and the two views cannot drift. One thing the host was not sending at all: control and option chords produce no text, so they never reached the input handler and were simply dropped. That is most of a terminal's vocabulary — ctrl-C, ctrl-D, alt-B — so a modified single key now goes out as a key line. Apps that do not use chords ignore it. `role: "mirror"` keeps its real job: the window may type into its session but cannot resize a PTY, and cannot re-point itself at another session, open one, or close one. That binding is enforced on the companion rather than trusted to the window. A mirror that outlived its companion now binds to the newest session on reconnect instead of being refused — being refused left it reconnecting forever against a window showing nothing. Verified against the console: a command typed on the Mac keyboard appears and runs on the 3DS, and a ctrl-C from the window interrupts `sleep 99` there — same session, same echo, both screens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
**L and R only switch sessions now.** They briefly doubled as the control modifier, which forced a tap-vs-hold discrimination — and a shoulder that acts on release rather than on press feels broken. ZL is the modifier; a console without one still has the keyboard's Ctrl cap, which arms for a single key. **Closing the last session leaves an empty state** rather than conjuring a shell nobody asked for. The companion detaches the replica instead of opening a replacement, the screen clears, and the next session to appear — opened from either end — takes the replica with it, because an empty state that persists while sessions exist is an empty state that lies. **The echo is four times closer.** Two fixed delays sat between a keystroke and the character: - the companion serialized on a 33 ms poll, so a byte waited for a tick that had nothing to do with it. Output now schedules its own pass with a 2 ms coalescing window (a screenful still collapses into one), with a slow interval left as the backstop for changes no PTY byte announces. Measured round trip through the companion: 21 ms median, now 4.6 ms. - the desktop host drained the wire AFTER the guest frame that reads it, so every arriving line cost a whole tick. It drains before the frame now, which is the order hosts/3ds/src/main.c has always pumped in. Verified on hardware: the empty state after closing the last tab, a session opened from it taking the console with it, and the companion round trip measured before and after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
**The window's echo was waiting on a read timeout.** Outgoing frames were queued for the read loop to forward between its blocking reads, so whenever the terminal was quiet a keystroke sat until the 250 ms read timeout expired. Measured keystroke-to-grid, against a companion whose own round trip is 4.6 ms: p50 133 ms, max 234 ms. The writer now owns an outbox woken by a condvar, so a frame leaves the moment it is queued — p50 16.7 ms, which is one tick, meaning the whole round trip now fits inside a single frame. The console never had this: its transport pumps send and receive independently every frame (hosts/3ds/src/svcwire.c). Coupling them here was the difference the operator could feel between the two screens. Measuring first was the point. The tick loop was the obvious suspect and was innocent — gaps were p50 16.7 ms, max 17.5 ms, a metronome. **Closing a session is now a hold, a slide and a release.** An 18 px × inside a 72 px tab is a coin flip on a resistive panel, and losing it kills a shell. Holding a tab slides a full-width bar out from under the strip; the tab reddens, the bar says what it will close, and releasing on it closes while releasing anywhere else does not. Nothing about it needs precision in x, and arming it is deliberate. The trailing "open a session" cell shrinks from a full tab width to 30 px, so it reads as sitting beside the last tab rather than as an empty tab of its own. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… knows Swaps the companion's terminal core from @xterm/headless to @wterm/ghostty — libghostty compiled to WASM, the same engine zhongduan makes authoritative. It runs headless under Node once the module is handed over as a data URL, because the loader fetches its path and Node's fetch has no `file:` scheme. The point is not the swap; it is how much of our own code the swap deletes. Work this daemon was doing by hand is work the core already does properly: - **Colour.** A hand-written 256-entry palette with a bright-bold rule resolved SGR into RGB. The core resolves it — its palette, its bold rule, its theme — and reports `fgRgb`/`bgRgb` per cell, absent when the cell is on the default. What is left here are the three attributes that are a renderer's decision and not a lookup: faint, reverse, concealed. - **Application cursor keys** came from reaching into xterm's `modes`; it is `cursorKeysApp()`, a question the core answers. - **Cursor visibility** came from two guesses at private fields behind a `try`; it is `getCursor().visible`. - **Scrollback** was arithmetic on a viewport's base row; the core indexes history directly, so a scrolled view is `getScrollbackCell(offset, col)`. - **Wide characters** were inferred from xterm's width call; `width` is a documented field, and grapheme clusters arrive whole in `chars`. Two capabilities we simply did not have: - **Terminal queries are answered.** Programs ask what terminal this is and where the cursor sits, and wait. The core composes the replies and nobody but the daemon can put them back on the PTY, so `getResponse()` is drained after every write. Until now those questions went unanswered. - **Bracketed paste.** More than one character at a time is a paste, and a program that asked to be told gets `\e[200~`-wrapped text instead of a burst of keystrokes an editor would execute. Two behaviour changes worth naming. The bell is gone: the core's interface has no bell event, and sniffing BEL from the byte stream would fire on every OSC-terminated title. And tab titles now follow the session's foreground process — the core deliberately ignores OSC 1, the icon name, which is all many prompts set, so `vim` in a tab now says `vim` rather than echoing a prompt string. That is what a multiplexer's tabs are for. Verified on hardware: colours, bold, reverse video, CJK and a full-screen vim, with the tab following zsh into vim and back. Companion round trip 5.2 ms. 31 unit tests, 12/12 stages, 18/18 goldens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The soft keyboard's keys were flat rectangles that changed colour on contact. Each one is now a cap set into a dark socket: a three-stop vertical gradient for a lit, slightly domed face, sitting above a 2 px lip of shadow. Pressed, the cap moves down into the socket, the face darkens and the stops run the other way, so the only light left is along the bottom edge. Darkening alone reads as disabled and flipping the gradient alone reads as highlighted; together they read as pushed in. The rows lost their container views — a key is placed absolutely in the plate and a row is now an offset, not a node. That is not tidying. A mount descends the JSX tree and a QuickJS call frame is expensive, so the 192 KiB JS stack this host granted was spent on nesting depth, and Pocket Term sat within a hair of it: adding one level under each key overflowed the stack at boot. The host reported `InternalError: stack overflow` and nothing else, on a device that had rolled back to the previous package. So the host now says where. take_exception appends the exception's frames to the message, which is what named the failing component here, and POCKETJS_JS_STACK_SIZE goes to 384 KiB — twice what this app needs, still leaving the C and Rust frames 640 KiB of the thread's 1 MiB, and costing nothing until it is used. The same cliff was hiding one layer down: devserver_send_ctrl held OUTGOING records to MAX_CTRL_BYTES, the bound on what a tool may send, and discarded anything larger without a word. The devtools tree dump passes 16 KiB on any app of a few hundred nodes, so `3ds:dev probe` had started timing out on this app with no way to tell a large tree from a hung device. Outgoing records may now fill a frame — tx_buffer is already sized for one, and screenshots push 48 KiB through it — and one that cannot fit any frame reports its size instead of vanishing, which the tool turns into an error rather than a 15 second wait. Verified on hardware (192.168.8.152) and in Azahar: 18/18 goldens, with the auxiliary frames rebaselined for the new keys.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
A working remote-terminal demo for the Nintendo 3DS, patterned on the zhongduan architecture: a companion daemon on the Mac holds the PTYs and one authoritative terminal core per session (@xterm/headless standing in for Ghostty), and the 3DS is a passive replica — a full cell-grid snapshot on attach, ordered row diffs after,
gen/seqfences so a lossy device queue degrades into a resync, never into a wrong screen. Multiplexing is N sessions server-side with per-connection attach; sessions keep running detached.The three layers
Native transport —
hosts/3ds/src/svcwire.c. The host service channel (spec ops 30..32,svcOpen/svcPoll/svcSend) over the existing SVC WIRE (PKNT) protocol (contracts/spec/spec.ts, the Vita'shosts/vita/src/net.rsis the reference implementation). Where the Vita uses threads, this is the devserver.c discipline instead: non-blocking sockets, one bounded pump per frame on the main thread. Discovery is the companion's once-a-second UDP-8621 beacon (source address + advertised TCP port), with ansdmc:/pocketjs/host.txtoverride that alternates back to beacon discovery after a failed connect.soc.cmakessocInitshared between the dev wire and this transport, so the channel works with or without adev.key. Capture builds compile the transport to inert stubs — goldens stay deterministic (the Vita3K contract).Guest app —
apps/term. Top screen: a 57×17 grid of 12 px JetBrains Mono cells — the natural ~7.2 px advance is snapped to an integer 7 px cell with negative tracking so every column lands on a pixel — with host-resolved SGR colors (bold-brightening, inverse, dim all resolve on the authority side; the device paints exactly what it is told), per-run background fills, a translucent cursor block, and box-drawing glyphs baked through the pass-1 literal scan. Touch screen: session tabs (tap to attach,+to spawn) and a five-row touch keyboard (action strip + 4 layers, one-shot Shift/Ctrl) hand-rolled insideAuxiliarySurface— the framework Osk portals to the primary viewport. Buttons: D-pad = repeating arrows, A ⏎, B ⌫, X ⇥, Y space, L/R switch sessions, START = ^C, SELECT = new session, circle pad scrubs scrollback.Companion daemon —
apps/term/host/serve.ts(Node ≥ 23.6; under Bun this machine's node-pty never execs the child — spawn-helper blocks forever in its slave-reattachopen(), verified by sampling — while Node is fine). PKNT listener + beacon, PTY sessions, 33 ms diff flushes, semantic key encoding (DECCKM-aware arrows, C0 masking) on the authority side. Falls back to an ephemeral TCP port when 8622 is taken (the pocket-youtube companion holds it) — the beacon advertises the actual port, so companions coexist. The pure halves (wire.ts/grid.ts/keys.ts) are dependency-free and unit-tested at the repo root; the native deps live in the directory's own package (root tsconfig excludes onlyserve.ts).Verified
ls -lacolumns, ^C and Enter through the key channel, session create/switch with snapshot re-fencing, and hot-push guest reloads that reconnect cleanly.termframes (connect overlay + keyboard layer machine driven by a touch tape) and the existing3ds-demoset — all 18 byte-exact on re-run; the soc.c refactor is pixel-neutral.bun run test12/12 stages,bunx tsc --noEmitclean, daemon's own tsconfig clean..3dsx(dist/3ds/pocketterm-main.3dsx, already built) — the usual ftpd swap.Screens
Azahar, connected to the Mac daemon (top: live zsh; bottom: tabs + keyboard): the
typed2capture in the session shows red/green/yellow/blue/bold-magenta output andls -larendered on the top screen.🤖 Generated with Claude Code