BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Département de mathématiques et applications - ECPv6.2.2//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://www.math.ens.psl.eu
X-WR-CALDESC:évènements pour Département de mathématiques et applications
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:Europe/Paris
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20260329T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20261025T010000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Europe/Paris:20261022T110000
DTEND;TZID=Europe/Paris:20261022T120000
DTSTAMP:20261027T133627
CREATED:20261021T091630Z
LAST-MODIFIED:20261021T091631Z
UID:22323-1792666800-1792670400@www.math.ens.psl.eu
SUMMARY:From Inference to Generative Diffusion: How Deep Networks Uncover Structure in Hierarchical Data
DESCRIPTION:Our understanding of how neural networks learn from structured data is caught between two extremes: theory largely relies on high-dimensional random data\, which lacks any structure\, while experiments rely on real-world benchmarks\, whose structure is not understood. This talk takes the middle road and works with a synthetic data model that is simple enough to be controlled yet rich enough to be informative. Sequences are generated by a branching process on a tree\, so that elements inherit nested correlations across all scales\, in a manner reminiscent of language or images\, while the Bayes-optimal predictor remains known exactly. This provides a ground truth against which trained models can be compared\, both in what they learn and in what they produce. In a first part\, we ask how the structure is learned. Using a filtering procedure that controls the range of correlations\, we show that transformers recover the exact inference algorithm\, incorporating larger scales sequentially\, in time during training and in space across attention layers. In a second part\, we ask whether the structure\, once learned\, can be reproduced. Turning to diffusion models\, we identify a phase of biased generalization\, during which the test loss keeps decreasing while generated samples drift anomalously close to the training data. We quantify this bias by comparing networks trained on disjoint datasets\, demonstrate its presence on real images\, and use the exact scores available on tree-based data to characterize its onset. The mechanism is again sequential: coarse structure is learned early and independently of the specific data\, while finer features are resolved later in a way that increasingly depends on individual training samples. As a consequence\, early stopping at the test loss minimum may be insufficient for privacy-critical applications.
URL:https://www.math.ens.psl.eu/evenement/from-inference-to-generative-diffusion-how-deep-networks-uncover-structure-in-hierarchical-data/
LOCATION:Salle W
CATEGORIES:CSD seminar
END:VEVENT
END:VCALENDAR