ENS-Data Science colloquium – Lénaïc Chizat (EPFL)
Amphi Jaurès (29 Rue d'Ulm)04 Avril 2024, Lénaïc Chizat (EPFL) Title: A Formula for Feature Learning in Large Neural Networks Abstract: Deep learning succeeds by doing hierarchical feature learning, but tuning hyperparameters such as initialization scales, learning rates, etc., only give indirect control over this behavior. This calls for theoretical tools to predict, measure and control feature learning. In this talk, we will first review various theoretical advances (signal propagation, infinite width dynamics, etc) that have led to a better understanding of the subtle impact of hyperparameters and architectural choices on the training dynamics. We will then introduce […]