Fibo Edit: multi-reference conditioning and batching - #14566
Open
ShivamShrirao wants to merge 2 commits into
Open
Fibo Edit: multi-reference conditioning and batching#14566ShivamShrirao wants to merge 2 commits into
ShivamShrirao wants to merge 2 commits into
Conversation
ShivamShrirao
force-pushed
the
fibo-edit-multi-reference
branch
from
August 23, 2026 14:35
07d3dd6 to
91796cd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds multi-reference image conditioning to
BriaFiboEditPipelineand fixes the two remaining items from thebria_fiboreview issue.Fixes the remaining items (issues 2 and 5) of #13618, issues 1/3/4/6 were fixed by #13981.
Multi-reference conditioning
imagenow accepts aPIL.Image.Imageor a list of them. A list means multiple references, not a batch:iis placed on RoPE time planei(generated tokens stay on plane 0).height/widthstill override it.Batching
num_images_per_prompt > 1now works, upstream crashed at a latent reshape because reference latents were never duplicated (fixes issue 5 of bria_fibo model/pipeline review #13618).test_num_images_per_promptis un-skipped and passes.Other fixes
timestepsare now honored in bothBriaFiboPipelineandBriaFiboEditPipeline, they were accepted and documented but hard-coded toNonein theretrieve_timestepscall (fixes issue 2 of bria_fibo model/pipeline review #13618).(batch, 1, 1, seq)instead of a float outer-product matrix, and is dropped entirely when nothing is padded, this keeps backends without mask support (flash-attn 2/3) usable atguidance_scale <= 1.BriaFiboTransformer2DModeldeclares_repeated_blocksfor regional compilation.do_patching=Truepreviously produced mismatched token widths between generated and reference latents (references were always packed unpatched); reference packing now followsdo_patching.Intentional behavior changes (please review)
torch.Tensorimage input now raises. References are PIL-only, following the Flux2 / QwenImage Edit Plus convention. Tensor input only became functional recently (fix(bria_fibo): fix guidance_embeds, prompt_embeds, tensor-image and multi-image crashes #13981 fixed a crash);np.ndarraywas already rejected.preprocessas a batch; it now means multiple references.image=[img]behaves identically to before.height/width(taken from the reference) behavior is equivalent; it only diverges when an explicit output size differs from the reference.Validation
i→ time idi, per-reference variable resolution, target-first sequence order).bria_fibo+bria_fibo_edittests pass. The pre-existingtest_inference_batch_single_identicalfailure in the basebria_fibosuite reproduces on currentmainwithout this PR, not introduced here.make style,make fix-copiesclean.Docs: added a "Multiple reference images" section to
bria_fibo_edit.mdwith a runnable two-reference example; fixed and extendedEXAMPLE_DOC_STRING.Self-review notes (final round)
Rubric:
.ai/review-rules.md(+.ai/AGENTS.md,.ai/pipelines.md,.ai/testing.md,.ai/models.md, model-integration pitfalls). Scope: full diff vs upstream base.Verdict: READY
No blocking findings. Reference preparation is a single public lifecycle method (
prepare_reference_latents) called from__call__, one code path handles one or N references, and tests assert on public API and end-to-end outputs only.Checks: full
bria_fibo+bria_fibo_editfast suites pass (aside from the pre-existing failure noted below),make styleclean,check_copiesclean.Behavior changes (disclosed intentionally)
torch.Tensorimage input now raises (was supported upstream since fix(bria_fibo): fix guidance_embeds, prompt_embeds, tensor-image and multi-image crashes #13981). References are PIL-only, following the Flux2 / QwenImage Edit Plus convention.np.ndarraywas already rejected upstream, unchanged.preprocessas a batch; a list now means multiple references.image=[img]behaves identically to before;image=[a, b]is a two-reference edit.timestepsare now honored in both pipelines (fixes issue 2 of bria_fibo model/pipeline review #13618): they were accepted and documented but hard-coded toNonein theretrieve_timestepscall, silently falling back to the default schedule.num_images_per_prompt > 1now works in the edit pipeline (fixes issue 5 of bria_fibo model/pipeline review #13618): upstream crashed at a latent reshape because reference latents were never duplicated. Everything now keys off the encoded prompt batch (batch_size * num_images_per_prompt);test_num_images_per_promptis un-skipped and passes.Left for the reviewer
_vae_safe_dims/_vae_safe_sizecarrybase_resolution=1024/multiple=16defaults no caller overrides, named constants vs. unused knobs is a judgment call.test_inference_batch_single_identicalfailure in the basebria_fibopipeline reproduces onmainwithout this PR, not introduced here.Dead-code analysis (advisory)
Traced
__call__end to end:paste_mask_on_image,is_valid_mask,get_mask_size, and bothdo_patchingbranches are all reachable. No likely-dead code beyond the default params noted above.Docs
docs/source/en/api/pipelines/bria_fibo_edit.mdgained a "Multiple reference images" section (semantics + runnable two-reference example);EXAMPLE_DOC_STRINGfixed (imports) and extended with a multi-reference call.Before submitting
self-reviewskill on the diff?Who can review?
@yiyixuxu @sayakpaul