From Inference to Generative Diffusion: How Deep Networks Uncover Structure in Hierarchical Data
Salle WOur understanding of how neural networks learn from structured data is caught between two extremes: theory largely relies on high-dimensional random data, which lacks any structure, while experiments rely on real-world benchmarks, whose structure is not understood. This talk takes the middle road and works with a synthetic data model that is simple enough to be controlled yet rich enough to be informative. Sequences are generated by a branching process on a tree, so that elements inherit nested correlations across all scales, in a manner reminiscent of language or images, […]