La recherche

We study nonlocal vector fields on the space of probability measures with finite second moment. Using Lions derivatives, we define their Lie-Lions bracket, which adds to the spatial Lie bracket a correction due to the evolution of the law, and prove that its vanishing is equivalent to commutation of the global flows, under first-order regularity and linear growth assumptions. We then construct a Lie algebra of complete polynomial nonlocal fields. For this class, we establish commutator and second-order Baker-Campbell-Hausdorff formulas with cubic remainders in the Wasserstein distance W_2, controlled by the second moment and uniform in the number of particles. The proofs rely on finite-particle realizations, which preserve the bracket and jointly determine the nonlocal field. We also obtain uniform estimates for smooth bounded kernels and a Leibniz rule for the action of polynomial fields on polynomial observables. Examples from collective dynamics show that the vanishing of the spatial bracket at a fixed law is neither necessary nor sufficient for commutation. They concern symmetries, aggregation-dependent rotation, synchronization, the transmission of a turn along a hierarchy, effective three-body interactions and occupancy-dependent circulation.

This PhD thesis presents contributions to the field of deep learning. From convolutional ResNets to Transformers, residual connections are ubiquitous in state-of-the-art deep learning models. The continuous depth analogues of residual networks, neural ODEs, have been widely adopted, but the connection between the discrete and continuous models still lacks a solid mathematical foundation. In this manuscript, we will show that for a formal correspondence between residual networks and neural ODEs to hold, the residual functions must be smooth with depth, and we will present an implicit regularization result of deep residual networks towards neural ODEs. We will then present two applications of this analogy to the design and study of new architectures. First, we will introduce a drop-in replacement for any residual network that can be trained with the same accuracy, but with much less memory. Second, by viewing the attention mechanism as an interacting particle system, where the particles are the tokens, we will study the impact of attention map normalization on the Transformer model. Finally, we will present some other contributions to Transformers: how Transformers perform in-context autoregressive learning and how to differentiably route tokens to experts in Sparse Mixture of Experts Transformers.