Fractal Generative Models
Free while signed in. Answers cite the passages they came from.
Researchers from MIT CSAIL & Google DeepMind introduce a novel fractal-based framework for generative modeling, where entire generative modules are treated as atomic "building blocks" and invoked recursively-resulting in self-similar fractal architectures:
Atomic generators as fractal modules - They abstract autoregressive models into modular units and stack them recursively. Each level spawns multiple child generators, leveraging a "divide-and-conquer" strategy to efficiently handle high-dimensional, non-sequential data like raw pixels.
Pixel-by-pixel image synthesis - Their fractal approach achieves state-of-the-art likelihood on ImageNet 64×64 (3.14 bits/dim), significantly surpassing prior autoregressive methods (3.40 bits/dim). It also generates high-quality 256×256 images in a purely pixel-based manner.
Strong quality & controllability - On class-conditional ImageNet 256×256, the fractal models reach an FID of 6.15, demonstrating competitive fidelity. Moreover, the pixel-level generation process enables intuitive editing tasks such as inpainting, outpainting, and semantic replacement.
Scalable & open-sourced - The fractal design drastically cuts compute at finer levels (modeling small patches), making pixel-by-pixel approaches feasible at larger resolutions.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack