Valentin De Bortoli – Generative modelling with diffusion: theory and practice

25 May 2023, 13h00-14h00 (Paris time), room Amphi Jaures (29 Rue d’Ulm).
Valentin De Bortoli (CNRS and ENS)
Title: Generative modelling with diffusion: theory and practice
Abstract: Generative modeling is the task of drawing new samples from an underlying distribution known only via an empirical measure. There exists a myriad of models to tackle this problem with applications in image and speech processing, medical imaging, forecasting and protein modeling to cite a few. Among these methods score-based generative models (or diffusion models) are a new powerful class of generative models that exhibit remarkable empirical performance. They consist of a ‘noising’ stage, whereby a diffusion is used to gradually add Gaussian noise to data, and a generative model, which entails a `denoising’ process defined by approximating the time-reversal of the diffusion. In this talk I discuss three aspects of diffusion models. First, I will present some of their theoretical guarantees with an emphasis on their behavior under the so-called manifold hypothesis. Such theoretical guarantees are non-vacuous and provide insight on the empirical behavior of these models. Then, I will turn to the extension of diffusion models to non Euclidean data. Indeed, classical generative models assume that data is supported on a Euclidean space, i.e. a manifold with flat geometry. In many domains such as robotics, geoscience or protein modeling, data is often naturally described by distributions living on Riemannian manifolds which require new methodologies to be appropriately handled. Finally, I will turn to constraints on the generative process itself. A well-known limitation of diffusion models is that the forward-time stochastic process must be run for a sufficiently long time for the final distribution to be approximately Gaussian. In contrast, solving the Schrödinger Bridge problem, i.e. an entropy-regularized optimal transport problem on path spaces, yields diffusions which generate samples from the data distribution in finite time. I will present Diffusion Schrödinger Bridge, an original approximation of the Iterative Proportional Fitting procedure to solve the Schrödinger Bridge problem.