La recherche

We prove a bumpy metric theorem in the sense of Ma\~{n}e for non-convex Hamiltonians that are satisfying a certain geometric property.

As context windows in large language models continue to expand, it is essential to characterize how attention behaves at extreme sequence lengths. We introduce token-sample complexity: the rate at which attention computed on $n$ tokens converges to its infinite-token limit. We estimate finite-$n$ convergence bounds at two levels: pointwise uniform convergence of the attention map, and convergence of moments for the transformed token distribution. For compactly supported (and more generally sub-Gaussian) distributions, our first result shows that the attention map converges uniformly on a ball of radius $R$ at rate $C(R)/\sqrt{n}$, where $C(R)$ grows exponentially with $R$. For large $R$, this estimate loses practical value, and our second result addresses this issue by establishing convergence rates for the moments of the transformed distribution (the token output of the attention layer). In this case, the rate is $C'(R)/n^β$ with $β<\tfrac{1}{2}$, and $C'(R)$ depends polynomially on the size of the support of the distribution. The exponent $β$ depends on the attention geometry and the spectral properties of the tokens distribution. We also examine the regime in which the attention parameter tends to infinity and the softmax approaches a hardmax, and in this setting, we establish a logarithmic rate of convergence. Experiments on synthetic Gaussian data and real BERT models on Wikipedia text confirm our predictions.

In this thesis, we study interpretable groups and fields in various theories of enriched fields, using tools from geometric model theory. The work is divided into the following three parts. The group configuration theorem for generically stable types. Following the proof of the group configuration theorem in the usual stable setting, we generalize it to the case of generically stable group configurations in arbitrary theories. More explicitly, we show how one can construct a type-definable group (action) from a generically stable sextuple of points satisfying the usual algebraicity and independence properties of a group configuration. On groups and fields interpretable in NTP2 fields. We show that, in NTP2 theories of enriched fields, under mild model-theoretic and algebraic assumptions, any definably amenable interpretable group admits a definable morphism to an algebraic group with purely imaginary kernel, i.e. that does not admit definable maps to the field sort with infinite image. We deduce a structure theorem for interpretable fields, which we instantiate for henselian valued fields of characteristic 0. We also extend these results to NIP (possibly enriched) differential fields, and prove a full classification of interpretable fields for differentially closed valued fields. In passing, we prove that in arbitrary theories, if K and F are definable fields such that the group of affine transformations F+ ⋊ F × can be definably embedded into an algebraic group over K, then F admits a definable field embedding into a finite extension of the field K. On groups and fields definable in D-henselian fields. Finally, we focus on the theory of D-henselian valued fields with differentially closed residue field and divisible value group, studied by Scanlon and Rideau-Kikuchi. Adapting the proof of Hrushovski’s p-configuration theorem, we prove that groups definable in the valued field sort with generically stable generics orthogonal to all differentially algebraic types, admit definable group homomorphisms to alge- braic groups, with kernels of finite rank. We then show that any definable field, in the valued field sort, with a generating subring admitting such a generic, is definably isomorphic to the valued field itself, assuming its Kolchin closure is of infinite rank.