La recherche

Turbulent cascades characterize the transfer of energy injected by a random force at large scales towards the small scales. We construct a linear equation that mimics the phenomenology of energy cascades when the external force is a statistically homogeneous and stationary stochastic process. In the Fourier variable, this equation can be seen as a wave equation, which corresponds to a wave operator of degree 0 in physical space. Our results give a complete characterization of the solution: it is smooth at any finite time, and, up to smaller order corrections, it converges to a fractional Gaussian field at infinite time. We apply a finite volume method in the Fourier variables formulation in order to reach the invariant measure of the equation.

In this paper, we investigate the properties of the Sliced Wasserstein Distance (SW) when employed as an objective functional. The SW metric has gained significant interest in the optimal transport and machine learning literature, due to its ability to capture intricate geometric properties of probability distributions while remaining computationally tractable, making it a valuable tool for various applications, including generative modeling and domain adaptation. Our study aims to provide a rigorous analysis of the critical points arising from the optimization of the SW objective. By computing explicit perturbations, we establish that stable critical points of SW cannot concentrate on segments. This stability analysis is crucial for understanding the behaviour of optimization algorithms for models trained using the SW objective. Furthermore, we investigate the properties of the SW objective, shedding light on the existence and convergence behavior of critical points. We illustrate our theoretical results through numerical experiments.

This PhD thesis presents contributions to the field of deep learning. From convolutional ResNets to Transformers, residual connections are ubiquitous in state-of-the-art deep learning models. The continuous depth analogues of residual networks, neural ODEs, have been widely adopted, but the connection between the discrete and continuous models still lacks a solid mathematical foundation. In this manuscript, we will show that for a formal correspondence between residual networks and neural ODEs to hold, the residual functions must be smooth with depth, and we will present an implicit regularization result of deep residual networks towards neural ODEs. We will then present two applications of this analogy to the design and study of new architectures. First, we will introduce a drop-in replacement for any residual network that can be trained with the same accuracy, but with much less memory. Second, by viewing the attention mechanism as an interacting particle system, where the particles are the tokens, we will study the impact of attention map normalization on the Transformer model. Finally, we will present some other contributions to Transformers: how Transformers perform in-context autoregressive learning and how to differentiably route tokens to experts in Sparse Mixture of Experts Transformers.