Skip to content

fix(producer): assert render artifact duration and frame count before commit - #3429

Open
santhiprakash wants to merge 2 commits into
heygen-com:mainfrom
santhiprakash:fix/3395-artifact-validation-frame-count
Open

fix(producer): assert render artifact duration and frame count before commit#3429
santhiprakash wants to merge 2 commits into
heygen-com:mainfrom
santhiprakash:fix/3395-artifact-validation-frame-count

Conversation

@santhiprakash

Copy link
Copy Markdown
Contributor

Closes #3395

What

ArtifactTransaction.validate() now refuses to publish a render output that is significantly shorter or has fewer frames than the capture pipeline just reported. The readable-non-empty check is unchanged; this adds a second gate on top, fed by ffprobe on the staged file and the values the orchestrator already holds.

Why

The multi-worker encode path can drop ~half the frames between capture and mux. Today the only validation is "the file exists and is non-empty", so a 148-of-300 render ships as Render complete and reaches the viewer. The information needed to catch it was already in hand at validation time — the captured duration and frame count live on the job — but no comparison was ever made.

The maintainer's field report (#3395, second comment) gives both halves of the bug surface: a media-free --workers 4 run lands at 300/300 because captureMode: "beginframe" keeps it off the parallel-disk path, while the trigger composition does go through it and ends up at 148/300 / 4.933s. Either a duration or a frame-count check against expected values catches the truncated case regardless of cause, so we add both.

How

ArtifactTransaction.validate() now accepts an optional expected: ArtifactValidationExpectation. The existing readable-non-empty check still runs first; when expected is supplied, validate() also calls a pluggable durationProbe (default: producer's extractMediaMetadata wrapper) and compares:

  • Duration: expectedDurationSeconds - probed.durationSeconds > tolerance (default tolerance is one frame at the job fps, or 20 ms when fps is unknown). Both field packets from the issue (52.2s → 41.333s and 10.0s → 4.933s) are well past this threshold; the --workers 1 workaround's exact-10.0s / 300-frame run is within it.
  • Frame count: when the caller supplies expectedFrames and the probe returns a frames count, expectedFrames - probed.frames > 1 rejects. This is the case the issue's second paragraph names as the "container reports correct duration but stream is shorter" failure mode — currently dormant in the default probe (which returns only duration), but the wire is in place for a richer probe without a follow-up.

Probe errors are surfaced as validation errors rather than silently passing the gate, and a probe returning 0/NaN is rejected outright — both explicitly to keep this from regressing back into "validation passes when it can't answer".

renderOrchestrator.ts now passes expected only for non-PNG-sequence, non-GIF video outputs with a finite positive captured duration, so PNG/GIF validation keeps its existing contract.

The validation still runs inside the staging directory, before the atomic rename to destinationPath — a hard failure never touches the destination and costs only a retry.

Test plan

  • bun test ./src/services/render/artifactTransaction.test.ts from packages/producer/ — 16/16 pass (9 pre-existing + 7 new). The new cases cover: probe is not called when no expectation is set, probe is not called for directory artifacts, truncation rejection at the field-packet scale (52.2s expected / 41.333s probed), within-tolerance acceptance of the --workers 1 workaround (10.000s / 300 frames / 30 fps), frame-count shortfall rejection (300 expected / 148 probed), probe failure propagation, and a probe returning zero duration being treated as "cannot verify".
  • bun run typecheck from packages/producer/ — clean.
  • bunx oxfmt --check and bunx oxlint on the three touched files — clean.
  • Manual: traced the orchestrator call site at packages/producer/src/services/renderOrchestrator.ts:3937 to confirm the captured duration / fps / frameCount path is correct, and that PNG-sequence and GIF outputs skip the new check.

Files touched:

  • packages/producer/src/services/render/artifactTransaction.ts
  • packages/producer/src/services/render/artifactTransaction.test.ts
  • packages/producer/src/services/renderOrchestrator.ts

… commit

Refuse to publish an artifact that is significantly shorter or has fewer frames

than the capture pipeline just reported. Adds a duration/frame-count gate on top

of the existing readable-non-empty check inside ArtifactTransaction.validate(),

keyed off the values the orchestrator already carries. Closes heygen-com#3395.
The frame-count gate added in heygen-com#3395 accepts an expectedFrames value from
the orchestrator, but defaultArtifactDurationProbe was still returning
only durationSeconds - so the wire was half-built and the assertion
short-circuited on undefined for every real render. Forward meta.frames
from ffprobe so the field-packet case the issue names (container duration
correct, stream shorter) is actually caught by the frame-count check,
not just the duration one.

extractMediaMetadata now populates a new frames field from the video
stream's nb_frames tag, returning undefined when the demuxer did not
report one (fragmented MP4, malformed streams, muxes that require
-count_packets). Callers that gate on the count must treat undefined as
no answer; the assertion already does.

The previous CI run (#32589981916) cancelled shard-6 at the 1h job
timeout after bun install failed to extract the aws-cdk-lib tarball
mid-Docker-build - a cache flake, not a code regression. Pushing a
follow-up commit retriggers CI against the now-populated cache layer;
the regression should clear without further code changes.
@santhiprakash
santhiprakash force-pushed the fix/3395-artifact-validation-frame-count branch from 83171fe to f15951e Compare August 23, 2026 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Artifact validation only checks the file is non-empty, so a truncated render ships as success

1 participant