InstantFusion✳

A shared latent interface for
heterogeneous image generators.

Xuanlang Dai^1,2,*Jiazi Bu^2,3,*Yujie Zhou³Yuhong Liu^2,4 Beichen Zhang^2,4Penghui Yang^2,3Ziyu Liu^2,3Yuhang Zang^2,†

¹ Fudan University² Shanghai AI Laboratory³ Shanghai Jiao Tong University⁴ The Chinese University of Hong Kong

^* Equal contribution.^† Corresponding author.

SCROLL TO CONNECT

I / METHODLATENT STATES / DENOISING DYNAMICS

INSTANTFUSION

Shared latent
interface.

A learned connection between independently trained image generators.

Four SD3 examples of snowy mountains and flowers beneath a blue sky. — **SD3**Base model

Each frozen generator connects to a common representation through a model-specific Latent AutoEncoder (LAE). Paired states from the same image and noise level are aligned while preserving the information needed for reconstruction.

Alignment also constrains local denoising dynamics, allowing an intermediate state to be translated into another model’s latent space and generation to continue.

01.A / LATENT ALIGNMENT

L_align State alignment

Aligns encoded states from the same image at matched noise levels, establishing correspondence in the shared space.

01.B / RECONSTRUCTION

L_rec Latent reconstruction

Reconstructs each model’s input latent from its shared representation, preserving the information needed for subsequent denoising.

01.C / DYNAMICS ALIGNMENT

L_dyn Denoising dynamics

Aligns local denoising updates projected into the shared space, encouraging compatible evolution across models.

InstantFusion training and inference pipelines, showing latent autoencoders and shared-space alignment. — Training and inference pipelines of InstantFusion.

II / INTERFACE VALIDATIONRECONSTRUCTION / CROSS-MODEL DENOISING

Preserve states.
Continue denoising.

We first evaluate what the shared interface preserves, then examine generation when denoising moves between models.

02.A / RECONSTRUCTION QUALITY

Repeated latent round trips

Reconstruction fidelity is measured after 1, 5, and 10 round trips. A single round trip retains high fidelity; repeated translations accumulate reconstruction error.

SD3–Qwen
Trips	PSNR ↑	SSIM ↑	LPIPS ↓
1	31.375	0.9417	0.04028
5	22.977	0.8431	0.12630
10	20.043	0.7670	0.18892

FLUX–Qwen
Trips	PSNR ↑	SSIM ↑	LPIPS ↓
1	32.579	0.9496	0.03663
5	22.663	0.8450	0.13064
10	19.106	0.7621	0.21014

SD3–FLUX–Qwen
Trips	PSNR ↑	SSIM ↑	LPIPS ↓
1	32.397	0.9511	0.03257
5	23.445	0.8580	0.11006
10	20.119	0.7808	0.17591

Visual reconstruction after 1, 5, and 10 latent round trips.

02.B / CROSS-MODEL DENOISING

Generation across model boundaries

Translate an intermediate state through the shared space and continue denoising with another generator. The examples compare model combinations; the table reports their generation quality.

Cross-model generation with SD3, FLUX.1, and Qwen-Image.

III / APPLICATIONS01 / GENERATION ACCELERATION

Faster
generation.

Combine efficient generators with Qwen-Image to reduce inference time. Adaptive switching uses the relative change in predicted-clean latents to decide when to hand off.

III / APPLICATIONS02 / MULTI-PREFERENCE COMPOSITION

Multiple preferences.
One generation.

Combine existing preference experts during denoising without training a new joint model for every preference combination.

Qualitative examples of multi-preference composition using existing experts.

III / APPLICATIONS03 / CROSS-MODEL DISTILLATION

SD3 baseline: a fox carrying a green backpack in a snowy forest.

Cross-model on-policy distillation at training step 2400: a fox carrying a green backpack in a snowy forest.

3006009001,2001,5001,8002,1002,400

SUBJECT / A fox with a green backpack in a snowy pine forest.

IV / ANALYSISOBSERVATION / ALIGNMENT / TRAINING

Inside the
interface.

A / OBSERVATION

Cross-model compatibility

Related spatial patterns and preserved coarse content suggest partial compatibility between independently trained generators.

B / DYNAMICS ALIGNMENT

The role of dynamics alignment

Adding the dynamics objective preserves state alignment and improves velocity alignment between models.

C / TRAINING DYNAMICS

LAE training dynamics

Reconstruction, state alignment, and dynamics alignment are optimized together while the generators stay frozen.

InstantFusion✳

Shared latentinterface.

Lalign State alignment

Lrec Latent reconstruction

Ldyn Denoising dynamics

Preserve states.Continue denoising.