La recherche

We prove a bumpy metric theorem in the sense of Ma\~{n}e for non-convex Hamiltonians that are satisfying a certain geometric property.

As context windows in large language models continue to expand, it is essential to characterize how attention behaves at extreme sequence lengths. We introduce token-sample complexity: the rate at which attention computed on $n$ tokens converges to its infinite-token limit. We estimate finite-$n$ convergence bounds at two levels: pointwise uniform convergence of the attention map, and convergence of moments for the transformed token distribution. For compactly supported (and more generally sub-Gaussian) distributions, our first result shows that the attention map converges uniformly on a ball of radius $R$ at rate $C(R)/\sqrt{n}$, where $C(R)$ grows exponentially with $R$. For large $R$, this estimate loses practical value, and our second result addresses this issue by establishing convergence rates for the moments of the transformed distribution (the token output of the attention layer). In this case, the rate is $C'(R)/n^β$ with $β<\tfrac{1}{2}$, and $C'(R)$ depends polynomially on the size of the support of the distribution. The exponent $β$ depends on the attention geometry and the spectral properties of the tokens distribution. We also examine the regime in which the attention parameter tends to infinity and the softmax approaches a hardmax, and in this setting, we establish a logarithmic rate of convergence. Experiments on synthetic Gaussian data and real BERT models on Wikipedia text confirm our predictions.

The work presented here is concerned with breaking water waves, a well-known phenomenon arising as an oceanic wave approaches the shore: its crest starts to move faster than the trough up front, which ultimately leads to the appearance of an overhanging region that quickly curls over while falling down until it collides with the water lying below. An important contemporary issue concerns the incorporation of the viscous dissipation associated with the breaking into the many models that have been introduced to describe the ocean. This is mostly done empirically. In the present work, we follow a different path: we aim at modelling wave breaking up to the free surface self-intersection (the splash singularity), relying thus on a more geometrical approach to the subject. The first part of this thesis will be devoted to the motivation of a set of equations that describes overhanging waves in the inviscid irrotational regime, with either a one-dimensional or a two-dimensional free surface. This is done by setting aside the commonly used Eulerian framework and working in (pseudo)Lagrangian coordinates instead. This should be seen as an extension of the Zakharov-Craig-Sulem formulation of the Water Waves problem. The non-canonical Hamiltonian structure of these partial differential equations is investigated and it is shown that in the absence of breaking, they can be reduced to the usual set of equations. Emphasis is put on the various physical assumptions that are made along the way. In a second moment, we come back to these very hypotheses and put them to the test. This is done numerically using a Navier-Stokes based computational framework based on the Finite-Element Method (FEM). The major novelty compared to other studies lies in the use of the Arbitrary Lagrangian-Eulerian method (ALE), which diminishes the interpolation error greatly. The viscosity can therefore be decreased to values that allow the comparison with the inviscid solution (computed using another method, based on potential theory in the complex plane) to be carried out. Over a flat topography, it is found that both the free-surface and bed boundary layers are sufficiently well-behaved as to not perturb the bulk irrotational flow. Water being characterised by a relatively small viscosity, the consequence is that, in this regime the inviscid models accurately describe the oceanic flow. We do not prove this assertion rigorously, however. Difficulties seem to arise, however, when a non-flat topography is considered. Indeed, the typical velocities associated with the wave are high enough to eventually trigger boundary layer separation near curved-enough portions of the bed, resulting in vorticity being shed in the initially irrotational flow, far from the topography. The convergence to the inviscid solution is therefore compromised.