Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/accessibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Switchify Remote is designed for VoiceOver, TalkBack, iOS Switch Control, and An
- Connected Remote control screens add one "Scroll to top" scan stop after the Surface selector pins. The 48-point control stays above the PC switcher, announces "Top of Remote" once after use, and does not animate when Reduce Motion is enabled.
- Remote keeps a quick PC switcher immediately above the tab bar in every connection state. It announces the active connection, exposes selected state for the current saved PC, and restores focus after its modal closes.
- Headings, connection changes, failures, repeat state, pairing approval, and pointer-profile recovery transitions are announced once without moving focus unexpectedly.
- In live typing, Return and the visible Enter control restore the text field and software keyboard after delivery. Failed Enter attempts keep the text available for retry and return focus without sending it again.
- The pairing code is announced one digit at a time.
- Text scales with the operating system; controls grow rather than shrinking text below the user's chosen size.
- Control labels are never truncated. Action groups reduce their columns as available width or text scale requires.
Expand All @@ -26,4 +27,4 @@ Switchify Remote is designed for VoiceOver, TalkBack, iOS Switch Control, and An

## Manual smoke test

On both platforms, clear the app data before starting. Enable the screen reader and then the platform switch-access feature. Traverse both setup steps and confirm the headings, step positions, explanations, desktop download, Continue, Back, Not now, and Allow Bluetooth are announced in logical order. Confirm no Bluetooth prompt appears before Allow Bluetooth, and that the layout remains usable in both orientations and at 200% text. Then navigate PCs, pairing, every available remote screen, Settings, and Diagnostics in logical order. Confirm every action can be selected, read-only text does not become a switch-scan stop, pointer repeat exposes Stop movement, held modifiers and drag expose state, disabled controls are announced, and backgrounding returns the desktop to a neutral input state. On each connected Remote control screen, scroll past the title and confirm the Surface selector stays visible, gains its backdrop only while pinned, remains one scan stop, and causes no focus move or announcement. Confirm Scroll to top then appears as one 48-point scan stop above the PC switcher, does not cover the final control, returns to the title, announces completion once, and disappears at the top. Interrupt the return gesture and confirm it remains usable without a stale announcement. Repeat with Reduce Motion and confirm the return is immediate. Confirm disconnected, recovering, and unavailable states do not expose Scroll to top. On Android, confirm Forwarding keeps its profile choices available before Start but adds mapped external switches to scanning only while forwarding is active. The mapped switches and overflow notice must disappear after manual or safety stops. In Settings, edit the Remote name and confirm validation and save status are announced without adding read-only scan stops. Confirm Use device model restores the model-based name. Confirm Privacy policy is one scan stop, announces that it opens in a browser, opens the Remote-specific policy, and returns safely if the browser cannot open. During a delayed pointer-profile response, confirm "Restoring controls" is announced once, followed by either restored controls or "Controls unavailable" after all retries. Open the Surface selector while it is pinned and confirm the underlying screen and bottom tabs are not scannable, options scan linearly, Close works, Android back and VoiceOver escape dismiss the modal, and focus returns to the selector after selection or dismissal. Repeat those checks with the Remote PC switcher, confirm it stays above the tabs while content scrolls, and switch between two saved PCs. Press Unpair for a saved computer and confirm the native alert reads the full computer name. Verify Cancel, Android Back, and outside dismissal preserve the pairing, then confirm Unpair removes it. Repeat in light and dark appearance, at 200% text scaling, with reduced motion enabled, and with iOS Reduce Transparency enabled.
On both platforms, clear the app data before starting. Enable the screen reader and then the platform switch-access feature. Traverse both setup steps and confirm the headings, step positions, explanations, desktop download, Continue, Back, Not now, and Allow Bluetooth are announced in logical order. Confirm no Bluetooth prompt appears before Allow Bluetooth, and that the layout remains usable in both orientations and at 200% text. Then navigate PCs, pairing, every available remote screen, Settings, and Diagnostics in logical order. Confirm every action can be selected, read-only text does not become a switch-scan stop, pointer repeat exposes Stop movement, held modifiers and drag expose state, disabled controls are announced, and backgrounding returns the desktop to a neutral input state. In live typing, send several lines with software Return and the visible Enter control. Confirm each sends one Enter, clears only after success, restores the text field and software keyboard, and accepts the next line without backspaces. Simulate a failed Enter and confirm the text remains, focus returns, and Retry Enter sends no duplicate text. On each connected Remote control screen, scroll past the title and confirm the Surface selector stays visible, gains its backdrop only while pinned, remains one scan stop, and causes no focus move or announcement. Confirm Scroll to top then appears as one 48-point scan stop above the PC switcher, does not cover the final control, returns to the title, announces completion once, and disappears at the top. Interrupt the return gesture and confirm it remains usable without a stale announcement. Repeat with Reduce Motion and confirm the return is immediate. Confirm disconnected, recovering, and unavailable states do not expose Scroll to top. On Android, confirm Forwarding keeps its profile choices available before Start but adds mapped external switches to scanning only while forwarding is active. The mapped switches and overflow notice must disappear after manual or safety stops. In Settings, edit the Remote name and confirm validation and save status are announced without adding read-only scan stops. Confirm Use device model restores the model-based name. Confirm Privacy policy is one scan stop, announces that it opens in a browser, opens the Remote-specific policy, and returns safely if the browser cannot open. During a delayed pointer-profile response, confirm "Restoring controls" is announced once, followed by either restored controls or "Controls unavailable" after all retries. Open the Surface selector while it is pinned and confirm the underlying screen and bottom tabs are not scannable, options scan linearly, Close works, Android back and VoiceOver escape dismiss the modal, and focus returns to the selector after selection or dismissal. Repeat those checks with the Remote PC switcher, confirm it stays above the tabs while content scrolls, and switch between two saved PCs. Press Unpair for a saved computer and confirm the native alert reads the full computer name. Verify Cancel, Android Back, and outside dismissal preserve the pairing, then confirm Unpair removes it. Repeat in light and dark appearance, at 200% text scaling, with reduced motion enabled, and with iOS Reduce Transparency enabled.
2 changes: 1 addition & 1 deletion docs/physical-smoke-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Run the matrix on a physical Android phone and iPhone against current Switchify
- Confirm a new pairing shows the phone model rather than a generic Remote name. Change Remote name while connected and confirm both desktop platforms update without re-pairing. Change it while offline, reconnect, and confirm the new name appears. Restart the PC application and confirm the name persists.
- On the OPD2403 against Windows, complete at least 20 connections and reconnects. Confirm controls become ready every time. Simulate one dropped pointer-profile response and confirm the app shows "Restoring controls" before controls appear without another reconnect.
5. Exercise eight-way movement, repeat/stop, all clicks, both scroll directions, drag cleanup, speed limits, and monitor movement.
6. Exercise live typing, backspace/replacement, stream recovery, draft persistence/send/clear, and every displayed PC key using non-sensitive fixture text.
6. Exercise live typing, backspace/replacement, stream recovery, draft persistence/send/clear, and every displayed PC key using non-sensitive fixture text. Send repeated live lines with both software Return and the visible Enter control; confirm each sends one Enter, clears only after success, and restores the text field and software keyboard. Simulate a failed Enter and confirm the text remains focused for Retry Enter without duplicate text or backspaces.
7. Exercise held modifiers, shortcuts, app switching, task view, desktop, minimize, maximize, and close. Confirm labels follow Windows/macOS conventions.
- On Android, open Forwarding and confirm profile choices remain available while stopped. Confirm mapped switches and the overflow notice appear only after forwarding starts and disappear after manual, hold-to-stop, configuration-change, and inactivity stops.
8. With VoiceOver/TalkBack and Switch Control/Switch Access, traverse every destination at maximum text size in portrait and landscape. Confirm selected, disabled, busy, error, and pairing states are announced.
Expand Down
54 changes: 50 additions & 4 deletions src/remote/RemoteSurfaces.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ import { MouseSurface } from './MouseSurface';
import { RemoteSession } from './RemoteSession';
import { TypingSurface } from './TypingSurface';
import { WindowSurface } from './WindowSurface';
import { focusLiveTextInput } from './focusLiveTextInput';
import { scheduleLiveTextInputFocus } from './focusLiveTextInput';

jest.mock('./focusLiveTextInput', () => ({ focusLiveTextInput: jest.fn() }));
jest.mock('./focusLiveTextInput', () => ({ scheduleLiveTextInputFocus: jest.fn(() => jest.fn()) }));

function profile(supportedCommands: string[]): PointerProfile {
const repeat = supportedCommands.includes('mouse.repeat.start') && supportedCommands.includes('mouse.repeat.stop');
Expand All @@ -21,7 +21,7 @@ const originalPlatform = Platform.OS;

describe('capability-driven remote surfaces', () => {
beforeEach(() => {
jest.mocked(focusLiveTextInput).mockClear();
jest.mocked(scheduleLiveTextInputFocus).mockReset().mockImplementation(() => jest.fn());
});

afterEach(() => {
Expand Down Expand Up @@ -56,7 +56,7 @@ describe('capability-driven remote surfaces', () => {
});

it.each(['visible Enter control', 'software keyboard Return'] as const)('submits and clears live text from the %s', async (source) => {
const focusTextInput = jest.mocked(focusLiveTextInput);
const focusTextInput = jest.mocked(scheduleLiveTextInputFocus);
const send = jest.fn(async (_type: string, _payload?: unknown) => true);
const session = new RemoteSession(
{ send } as unknown as ConnectionManager,
Expand Down Expand Up @@ -84,6 +84,7 @@ describe('capability-driven remote surfaces', () => {
});

it('retains live text after a failed Enter and offers a specific retry', async () => {
const focusTextInput = jest.mocked(scheduleLiveTextInputFocus);
let enterAttempt = 0;
const send = jest.fn(async (type: string) => {
if (type !== 'keyboard.textStream.key') return true;
Expand All @@ -102,12 +103,57 @@ describe('capability-driven remote surfaces', () => {

await waitFor(() => expect(view.getByLabelText('Retry Enter')).toBeTruthy());
expect(view.getByLabelText('Live text').props.value).toBe('keep me');
await waitFor(() => expect(focusTextInput).toHaveBeenCalledTimes(1));
await act(async () => { fireEvent.press(view.getByLabelText('Retry Enter')); });
await waitFor(() => expect(view.getByLabelText('Live text').props.value).toBe(''));
await waitFor(() => expect(focusTextInput).toHaveBeenCalledTimes(2));
expect(send.mock.calls.filter(([type]) => type === 'keyboard.textStream.chunk')).toHaveLength(1);
expect(send.mock.calls.filter(([type]) => type === 'keyboard.textStream.key')).toHaveLength(2);
});

it.each(['mode change', 'session replacement', 'unmount'] as const)('cancels scheduled live focus on %s', async (transition) => {
const cancelFocus = jest.fn();
jest.mocked(scheduleLiveTextInputFocus).mockReturnValue(cancelFocus);
const send = jest.fn(async (_type: string) => true);
const supported = profile(['keyboard.textStream.open', 'keyboard.textStream.chunk', 'keyboard.textStream.key', 'keyboard.textStream.close']);
const session = new RemoteSession({ send } as unknown as ConnectionManager, supported);
const view = await render(<TypingSurface session={session} mode="live" draft="" />);

await act(async () => { fireEvent.changeText(view.getByLabelText('Live text'), 'done'); });
await waitFor(() => expect(send.mock.calls.some(([type]) => type === 'keyboard.textStream.chunk')).toBe(true));
await act(async () => { fireEvent.press(view.getByLabelText('Enter')); });
await waitFor(() => expect(scheduleLiveTextInputFocus).toHaveBeenCalledTimes(1));

if (transition === 'mode change') await view.rerender(<TypingSurface session={session} mode="draft" draft="" />);
else if (transition === 'session replacement') {
const replacement = new RemoteSession({ send: jest.fn(async () => true) } as unknown as ConnectionManager, supported);
await view.rerender(<TypingSurface session={replacement} mode="live" draft="" />);
} else await view.unmount();

expect(cancelFocus).toHaveBeenCalledTimes(1);
expect(scheduleLiveTextInputFocus).toHaveBeenCalledTimes(1);
});

it('cancels the previous focus restoration when another Enter submission starts', async () => {
const cancelFirstFocus = jest.fn();
jest.mocked(scheduleLiveTextInputFocus).mockReturnValueOnce(cancelFirstFocus).mockReturnValue(jest.fn());
const send = jest.fn(async () => true);
const session = new RemoteSession(
{ send } as unknown as ConnectionManager,
profile(['keyboard.textStream.open', 'keyboard.textStream.chunk', 'keyboard.textStream.key', 'keyboard.textStream.close']),
);
const view = await render(<TypingSurface session={session} mode="live" draft="" />);

await act(async () => { fireEvent.changeText(view.getByLabelText('Live text'), 'first'); });
await act(async () => { fireEvent.press(view.getByLabelText('Enter')); });
await waitFor(() => expect(scheduleLiveTextInputFocus).toHaveBeenCalledTimes(1));
await act(async () => { fireEvent.changeText(view.getByLabelText('Live text'), 'second'); });
await act(async () => { fireEvent.press(view.getByLabelText('Enter')); });

expect(cancelFirstFocus).toHaveBeenCalledTimes(1);
await waitFor(() => expect(scheduleLiveTextInputFocus).toHaveBeenCalledTimes(2));
});

it('blocks duplicate live submissions and further input until Enter completes', async () => {
let releaseEnter!: (value: boolean) => void;
const pendingEnter = new Promise<boolean>((resolve) => { releaseEnter = resolve; });
Expand Down
39 changes: 34 additions & 5 deletions src/remote/TypingSurface.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import { StatusBadge } from '@/components/StatusBadge';
import { commandPayloads } from '@/domain/protocol/commands';
import { preferencesStore, type TypingMode } from '@/storage/PreferencesStore';
import { useTheme } from '@/theme/ThemeContext';
import { focusLiveTextInput } from './focusLiveTextInput';
import { scheduleLiveTextInputFocus } from './focusLiveTextInput';
import { LiveTypingController } from './LiveTypingController';
import type { RemoteSession } from './RemoteSession';

Expand All @@ -18,10 +18,13 @@ export function TypingSurface({ session, mode, draft }: { session: RemoteSession
const [liveText, setLiveText] = useState('');
const [liveFailure, setLiveFailure] = useState<'text' | 'enter' | null>(null);
const [liveSubmitting, setLiveSubmitting] = useState(false);
const [liveFocusRequest, setLiveFocusRequest] = useState(0);
const cancelLiveFocus = useRef<(() => void) | null>(null);
const liveFocusPending = useRef(false);
const liveRevision = useRef(0);
const liveSubmittingRef = useRef(false);
const liveInputRef = useRef<TextInput>(null);
const latestMode = useRef(mode);
const latestSession = useRef(session);
const mounted = useRef(true);
const live = useMemo(() => new LiveTypingController(session), [session]);
const { colors, radii, spacing, typography } = useTheme();
Expand All @@ -40,7 +43,11 @@ export function TypingSurface({ session, mode, draft }: { session: RemoteSession
};
const submitLive = async () => {
if (!liveSupported || liveSubmittingRef.current) return;
cancelLiveFocus.current?.();
cancelLiveFocus.current = null;
liveFocusPending.current = false;
liveSubmittingRef.current = true;
const submittingSession = session;
const revision = ++liveRevision.current;
setLiveFailure(null);
setLiveSubmitting(true);
Expand All @@ -52,15 +59,37 @@ export function TypingSurface({ session, mode, draft }: { session: RemoteSession
}
liveSubmittingRef.current = false;
setLiveSubmitting(false);
setLiveFocusRequest((request) => request + 1);
liveFocusPending.current = latestMode.current === 'live' && latestSession.current === submittingSession;
};
useEffect(() => {
if (liveFocusRequest > 0) focusLiveTextInput(liveInputRef.current);
}, [liveFocusRequest]);
cancelLiveFocus.current?.();
cancelLiveFocus.current = null;
if (liveSubmitting || !liveFocusPending.current || mode !== 'live') return;
liveFocusPending.current = false;
const cancel = scheduleLiveTextInputFocus(() => liveInputRef.current);
cancelLiveFocus.current = cancel;
return () => {
if (cancelLiveFocus.current !== cancel) return;
cancel();
cancelLiveFocus.current = null;
};
}, [liveSubmitting, mode, session]);
useEffect(() => {
const changed = latestMode.current !== mode || latestSession.current !== session;
latestMode.current = mode;
latestSession.current = session;
if (!changed) return;
cancelLiveFocus.current?.();
cancelLiveFocus.current = null;
liveFocusPending.current = false;
}, [mode, session]);
useEffect(() => {
mounted.current = true;
return () => {
mounted.current = false;
cancelLiveFocus.current?.();
cancelLiveFocus.current = null;
liveFocusPending.current = false;
liveRevision.current += 1;
liveSubmittingRef.current = false;
};
Expand Down
Loading