feat: add filesystem-backed SerDes stage

[zhongkechen]

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Issue Link, if available

Closes #463.

Implements Approach A from ADR-005 and provides the SerDes context requested in #509.

Description

• Extend SerDes with then(SerDesStage), returning the SerDes interface so fluent callers do not depend on the concrete pipeline implementation.
• Define a pipeline as exactly one root SerDes value codec followed by zero or more reversible String → String SerDesStage components. Serialization runs in declaration order and deserialization runs in reverse order.
• Require stages to use self-identifying formats: deserialization reverses recognized valid input, rejects recognized malformed or unsupported input, and passes unrecognized input through unchanged. This lets raw external payloads reach the root value codec without pipeline-specific bypass control flow.
• Keep root SerDes value codecs context-free and pass SerDesContext explicitly only to SerDesStage and nested BinarySerDesStage calls. Serialization-stage context exposes the root input object through originalValue(); deserialization-stage context has a null originalValue(). No SerDes-specific thread-local storage is used.
• Make CloudDurableTestRunner and LocalDurableTestRunner serialize initial invocation input with a separate context-free value codec. By default, they use the persisted SerDes when it is a plain value codec or the root value codec when it is composable; withInputSerDes(...) provides an explicit value-codec override. Persisted pipeline stages are not used to encode initial input.
• Add BinarySerDesStage for reversible byte[] → byte[] transformations and ComposableBinarySerDesStage to expose an ordered binary chain as one top-level string stage.
• Make both binary boundaries customizable through the same StringBinaryCodec interface. The processing-order builder uses startWith(codec), zero or more then(binaryStage) calls, endWith(codec), and build(). The outer stage adds a reserved versioned frame for reliable format recognition.
• Add RetryableSerDesException, RetrySerDesStage, and RetryBinarySerDesStage. The wrappers retry explicitly retryable failures using existing bounded RetryStrategy implementations and pass the same stage context to every attempt.
• Add FileSystemSerDesStage to the core SDK as a reversible SerDesStage for filesystem-backed payload storage. The stage, storage/path configuration, and preview APIs live in the dedicated software.amazon.lambda.durable.serde.filesystem Java package without adding a Maven module.
• Support string payloads, versioned envelopes, ALWAYS/OVERFLOW modes, immutable content-hashed files, attempt-aware operation identities, and pass-through of unrecognized input.
• Publish every offloaded payload to a unique path with one CREATE_NEW write. Publication does not use hard links or renames, so the implementation works on both EFS and S3 Files without overwriting data referenced by an earlier checkpoint.
• Add structured preview parity with the Python and TypeScript SDKs: include-all/exclude-all modes, include/exclude/mask selectors, anywhere/exact-path matching, configurable mask text, a default 4 KB budget, and a standalone preview builder. Custom preview callbacks receive both the preceding stage string and SerDesContext, including originalValue().
• Allow FileSystemSerDesStage to be followed by other stages that transform its inline or file-reference envelope. Filesystem overflow and preview-size checks apply at the filesystem stage boundary; later stages are responsible for their own size expansion.
• Harden filesystem references with producer execution/entity ownership, content-hash validation, filesystem-provider-correct path resolution, path containment, and symlink rejection.
• Carry stage context through durable runtime and persisted-result testing paths, including operation metadata, replay attempts, chained invoke results, executor-backed processing, binary substages, and retry attempts.
• Update the ADR, design, configuration, and SerDes guides for pipeline composition, stage-only context, serialization originalValue, binary stages, retry wrappers, filesystem stages, executor behavior, core SDK packaging, stage format recognition, structured previews, S3 Files compatibility, and independent test-runner input codecs.
• Document a binary-only top-level pipeline as a rejected alternative: it simplifies the type model and can benefit binary-heavy pipelines, but adds full-payload UTF-8 conversion and allocation to the common direct Jackson string path.

Demo/Screenshots

Not applicable. Configuration examples are included in the SerDes guide.

Checklist

• I have filled out every section of the PR template
• I have thoroughly tested this change

Testing

Coverage includes:

• Root SerDes plus SerDesStage pipeline creation, immutable appending, ordering, reverse processing, pass-through of unrecognized input, malformed recognized input, nulls, and failure metadata.
• API validation that only SerDesStage components can be appended after the root value codec and that fluent composition returns the SerDes interface.
• Naming and package audit confirming every named built-in stage implementation ends in Stage and filesystem-specific APIs live under software.amazon.lambda.durable.serde.filesystem.
• Explicit SerDesContext propagation through top-level stages, binary substages, executor-backed calls, and every retry attempt.
• Serialization originalValue() propagation to string stages, binary substages, retry wrappers, and filesystem preview callbacks; deserialization verifies that it is absent.
• Ordered binary serialization and reverse deserialization with custom boundary codecs and versioned format recognition.
• Inline and executor-backed execution, stage context propagation, caching, and executor isolation.
• String and binary retry selection, exhaustion, delay handling, interruption, and context propagation.
• Filesystem stage behavior, stages after filesystem storage, storage modes, envelope recognition and validation, immutable unique publication, providers without hard-link support, ownership, content-hash validation, provider-specific paths, path containment, and symlink rejection.
• Structured preview include/exclude/mask behavior, exact-path and anywhere matching, nested objects and arrays, custom masks, byte budgets, JSON pipeline integration, original-object access, and malformed input.
• Root input/output, step replay and retry attempts, child contexts, callbacks, invoke boundaries, and local/cloud runner behavior.
• Default and custom test-runner input value codecs, persisted-pipeline isolation, composable-input rejection, and filesystem integration without explicit input configuration.

Validated with:

• The full eight-module Maven reactor on Java 17
• Focused filesystem, pipeline, binary-stage, retry, and structured preview tests
• SDK Javadoc generation
• mvn spotless:check
• git diff --check

Cloud integration tests remain opt-in through -Dtest.cloud.enabled=true.

Examples

The SerDes guide, ADR, and design documentation include configuration examples.

[github-actions]

Codex AI review · Finding arf_v1_3ixetnh3553v2zi7h3dw3ay6gm

[P2] nextPath is resolved by pathname while current is a previously opened directory handle. If another process renames or replaces currentPath between the relative lookup and this call, Files.createDirectory(nextPath) can follow the replacement and create a directory outside basePath; reopening relative to current cannot undo that side effect. This violates the stated TOCTOU confinement guarantee on shared writable mounts. Create directories relative to the held handle using an atomic mkdirat-style operation, or require/pre-provision the tree and fail closed when a component is missing.

[github-actions]

Codex AI review

Found one filesystem confinement issue in directory creation.

Reviewed commit 828cfa36a6df27f6fa2100c87d87c133ffd381ec. Workflow run