fix(fs): tolerate FatFS VFS limits in atomic dir writes - #324
Conversation
ESP-IDF's FATFS VFS diverges from POSIX in two places that break dir_write's atomic-overwrite path: - O_EXCL opens fail with ENOENT, so the temp-file create_new never succeeds and every atomic write errors out. Fall back to create+truncate: the tmp dir is module-owned and swept, and the monotonic counter keeps the name unique. - FatFS f_rename refuses to overwrite an existing destination (surfaced as EEXIST) where POSIX rename replaces it. Fall back to remove+rename; the atomic-overwrite contract is not representable on FatFS. Both fallback arms are unreachable on POSIX filesystems, so behavior there is unchanged. Verified with cargo test -p pocket-fs (12/12) and on an ESP32-P4 board (ESP-IDF v6.1, FatFS on SD over the VFS) where data.fs writes and rewrites land and persist across reboots.
4536128 to
1e361c7
Compare
doodlewind
left a comment
There was a problem hiding this comment.
Thanks for the hardware reproduction and the narrow error handling. I am requesting changes because the current fallback breaks the fs contract in two merge-blocking ways:
-
After the first rename returns AlreadyExists, the code removes the old destination before attempting the second rename. If that rename fails or power is lost in between, the old file is gone; on the next FsModule construction the temp directory is swept, so the new file is lost too. FS_WRITE_TRUNCATE promises old content or new content after power loss, never a missing target. Please use a recoverable replace/journal sequence that preserves that invariant instead of silently downgrading the reference core.
-
The public dir_rename path is unchanged, so FatFS still cannot replace an existing destination even though OP_RENAME requires atomic file replacement and is the documented whole-file update path for files larger than one IO chunk. Please centralize the filesystem-specific replace behavior so truncate writes and public rename cannot diverge.
The existing 12 host tests pass only on the POSIX path and do not execute either fallback. The hardware receipt proves normal write/rewrite persistence, but not interruption of the remove-then-rename window. Please add deterministic fault-injection coverage for the ENOENT and EEXIST fallbacks, second-rename failure, and startup recovery, and add pocket-fs to a PR CI path.
Problem
dir_write's atomic-overwrite path assumes two POSIX behaviors that ESP-IDF's FATFS VFS does not provide, so every atomic write through the fs module fails on ESP32 targets with a FatFS-backed root:create_newcall never succeeds and the write errors out before any payload lands.f_renamerefuses to overwrite an existing destination (FR_EXIST, surfaced as EEXIST) where POSIXrenamereplaces it, so rewriting an existing file fails even after the temp file landed and synced.Fix
Both fallbacks arm only on the exact error kinds where the filesystem is known to diverge from POSIX:
Behavior on POSIX filesystems is unchanged — both fallback arms are unreachable there.
Verification
cargo test -p pocket-fs— 12/12 pass on the host.data.fswrites and rewrites land and persist across reboots. Board-side receipts: pocketjs-d1001 verification doc.