Q1
Does looping work?
Yes, with the right training. Looping alone barely helps, but deep supervision and self-modulating attention unlock a clear gain.
+4.0over no looping, against +1.0 for naive looping
Just loop it! More depth, not more parameters.
Improving text-to-image models has traditionally relied on increasing model size or the number of denoising steps. In this work, we explore an alternative way to scale computation by repeatedly running shared Transformer blocks within each denoising step, effectively increasing computational depth while keeping the parameter count fixed. This looped computation enables iterative refinement of internal representations without explicit reasoning tokens. However, naive looping fails to consistently improve image quality. We trace this problem to weak supervision across intermediate loops and unregulated attention updates that progressively erode local information. To overcome these challenges, we propose Looped Diffusion Transformer (Looped-DiT), which combines deep supervision across intermediate loops with self-modulating attention to stabilize looped feature updates. Under matched-parameter and matched-compute settings, Looped-DiT consistently outperforms non-looped baselines. Notably, a 260M-parameter looped model can surpass a model 6.5× larger across multiple text-to-image benchmarks while requiring 4.9× lower inference compute. Beyond this performance gain, we find that looped computation can offer a more effective form of iterative computation for diffusion models, with increasing loop depth yielding larger gains than adding more denoising steps under a fixed inference budget. Furthermore, deeper loops can progressively correct mistakes made in earlier loops, exhibiting behaviors suggestive of latent reasoning. Together, these results show that looped computation offers a promising way to scale visual generation models.
Looped-DiT reruns the same middle blocks several times in every denoising step. Deep supervision makes every loop useful, and self-modulating attention stops later loops from overwriting.
Supervising only the last loop leaves earlier loops without direct supervision. Deep supervision fixes this by decoding every loop and training each prediction against the clean image.
Plain attention can write large updates that erode earlier progress. Self-modulating attention helps preserve progress by modulating each token's update with a gate recomputed at every loop. We tested two gates, and exclusive self-attention (XSA) worked better than gated attention, so we adopt it.
Looped-DiT B/16 has the highest average on six benchmarks and the best score on five. The next-best model is 6.5× larger.
Yes, with the right training. Looping alone barely helps, but deep supervision and self-modulating attention unlock a clear gain.
Yes, with half the parameters. At equal compute, it outscores deeper and wider networks nearly twice its size.
Yes, with half the compute. At every budget, extra loops beat extra denoising steps.
Yes, it corrects its own mistakes. Later loops fix counts, layouts and text, with no reasoning tokens.
Yes, and together they gain the most. Looping fixes constraints that must hold together, while chain of thought fills in implied content.
Even one loop beats the non-looped model, and extra loops hold their score.
With XSA, updates shrink each loop and the layout survives.
A small network decides after each loop whether to stop.
Deep supervision alone adds +1.3, and XSA alone adds +2.3.
Each selected benchmark prompt is generated at 1–4 loops from the same noise.
Our looped models are ~260M-parameter pixel-space MMDiTs at 512 × 512. Larger looped models, VAE latent-space generation and higher resolutions are untested.
We fix a [6, 5, 6] block split and four training loops. Other splits, placements and depths are largely unexplored.
Deep supervision raises training compute about 1.54× over the deeper and wider baselines.
Our evidence is mainly behavioral and representational. We see errors corrected across loops but do not explain the mechanism or when refinement counts as reasoning.
@misc{chng2026loopeddiffusiontransformer,
title = {Looped Diffusion Transformer},
author = {Yong Xien Chng and Tianyi Chen and Wenwen Tong and Haiwen Diao and
Zhongang Cai and Lei Yang and Ziwei Liu and Lewei Lu and Dahua Lin and
Gao Huang},
year = {2026},
eprint = {2609.40305},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2609.40305}
}