Séminaire Data de l’ENS

The recent success of machine learning suggests that neural networks may be capable of approximating high-dimensional functions with controllably small errors. As a result, they could outperform standard function interpolation methods that have been the workhorses of scientific computing but do not scale well with dimension. In support of this prospect, here I will review what is known about the trainability and accuracy of shallow neural networks, which offer the simplest instance of nonlinear learning in functional spaces that are fundamentally different from classic approximation spaces. The dynamics of training […]

The young field of Machine learning has changed the ways we interact with data and neural networks have made us appreciate the potential of working with millions of parameters. Interestingly, the vast majority of scientific discoveries today are not based on these new techniques. I will discuss the contrast between these two regimes and I will show how an intermediate approach, i.e. neural network inspired but mathematically defined statistics (scattering and phase harmonic transforms), can provide the long-awaited tools in scientific research. I will illustrate these points using astrophysics as […]

Florence d'Alché-Buc (Telecom ParisTech) Title: Learning to predict complex outputs: a kernel view Abstract: Motivated by prediction tasks such as molecule identification or functional regression, we propose to leverage the notion of kernel to take into account the nature of output variables whether they be discrete structures or functions. This approach boils down to encode output data as vectors of the Reproducing kernel Hilbert Space associated to the so-called output kernel. We present vector-valued kernel machines to implement it and discuss different learning problems linked with the chosen loss function. Eventually large scale […]

Gerard Ben Arous (New York University) Title: Effective dynamics and critical scaling for Stochastic Gradient Descent in high dimensions Abstract: SGD in high dimension is a workhorse for high dimensional statistics and machine learning, but understanding its behavior in high dimensions is not yet a simple task. We study here the limiting 'effective' dynamics of some summary statistics for SGD in high dimensions, and find interesting and new regimes, i.e. not the expected one given by the population gradient flow. We find that a new corrector term is needed and that the phase […]

Chris Wiggins (Columbia & NYT) Data Science @ New York Times  The Data Science group at The New York Times develops and deploys machine learning solutions to newsroom and business problems. Re-framing real-world questions as machine learning tasks requires not only adapting and extending models and algorithms to new or special cases but also sufficient breadth to know the right method for the right challenge. I'll first outline how  - unsupervised,  - supervised, and  - reinforcement learning methods are increasingly used in human applications for  - description,  - prediction, and […]

Freddy Bouchet (ENS Lyon) Probabilistic forecast of extreme heat waves using convolutional neural networks and rare event simulations Understanding extreme events and their probability is key for the study of climate change impacts, risk assessment, adaptation, and the protection of living beings. Extreme heatwaves are, and likely will be in the future, among the deadliest weather events. Forecasting their occurrence probability a few days, weeks, or months in advance is a primary challenge for risk assessment and attribution, but also for fundamental studies about processes, dataset and model validation, and […]

Thursday December 15 2022, Wolfram Pernice (University of Münster) Title: Computing beyond Moore's law with photonic hardware Abstract: Conventional computers are organized around a centralized processing architecture, which is well suited to running sequential, procedure-based programs. Such an architecture is inefficient for computational models that are distributed, massively parallel and adaptive, most notably those used for neural networks in artificial intelligence. In these application domains demand for high throughput, low latency and low energy consumption is driving the development of not only new architectures, but also new platforms for information processing. Photonic circuits are […]

Thursday 2nd of February 2022, 12h00-13h00 (Paris time), room Amphi Jaures (29 Rue d'Ulm). Remi Gribonval (INRIA) Title: Rapture of the deep: highs and lows of sparsity in a world of depths Abstract: Promoting sparse connections in neural networks is natural to control their complexity. Besides, given its thoroughly documented role in inverse problems and variable selection, sparsity also has the potential to give rise to learning mechanisms endowed with certain interpretability guarantees. Through an overview of recent explorations around this theme, I will compare and contrast classical sparse regularization for inverse problems […]

29 March 2023, Thomas Serre (Brown University) Title: Feedforward and feedback processes in visual processing Abstract: Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching - and sometimes even surpassing - human accuracy on a variety of visual recognition tasks. In this talk, however, I will show that these neural networks (and recent extensions) exhibit a limited ability to solve seemingly simple visual reasoning problems. Our group has developed a computational neuroscience model of the […]

25 May 2023, 13h00-14h00 (Paris time), room Amphi Jaures (29 Rue d'Ulm). Valentin De Bortoli (CNRS and ENS) Title: Generative modelling with diffusion: theory and practice Abstract: Generative modeling is the task of drawing new samples from an underlying distribution known only via an empirical measure. There exists a myriad of models to tackle this problem with applications in image and speech processing, medical imaging, forecasting and protein modeling to cite a few. Among these methods score-based generative models (or diffusion models) are a new powerful class of generative models that exhibit remarkable […]

04 October 2023, 12h30-13h30 (Paris time), room Amphi Jaures (29 Rue d'Ulm). Julia Kempe (NYU Centre for Data Science and Courant Institute) Title: Towards Understanding Adversarial Robustness Abstract: Adversarial vulnerability of neural nets, their failure under small, imperceptible perturbations, and subsequent techniques to create robust models have attracted significant attention; yet we still lack a full understanding of this phenomenon. In this talk I will introduce the problem and current defenses and then explore how tools and insights coming from statistical physics, in particular certain infinite-width limits of neural nets, help shed more […]