Foundation models are a key platform which implicitly encodes world knowledge. In this talk we first focus on vision-language models, such as CLIP, and investigate their geometric behavior and logic behind the high-dimensional feature encoding. For instance, we find that as image or text become more rare and distinct they are encoded further from the center of the embedding. We explain why InfoNCE loss leads to that behavior. We also find out empirically that each modality can be well modeled statistically as admitting a multivariate Gaussian distribution. This finding is […]

Vladimir Petrovic: "Repulsive Monte Carlo on the sphere for the sliced Wasserstein distance" Damien Rouchouse: "Discrete Diffusion Design: An Asymptotic Error Analysis Approach"

Antonin de Courcel (ENS) “Effective models in high-dimensional unstable regimes: optimization dynamics at the edge of stability” Nicolas Mil-Homens Cavaco (Louvain) “Expressivity of lazy sinusoidal neural networks: A joint kernel-based and frame-theoretic approach” Antonin de Courcel (DMA) “Effective models in high-dimensional unstable regimes: optimization dynamics at the edge of stability” Nicolas Mil-Homens Cavaco (Université catholique de Louvain) “Expressivity of lazy sinusoidal neural networks: A joint kernel-based and frame-theoretic approach”