Solving most image processing tasks, such as denoising, deblurring, inpainting, etc, require explicitly or implicitly a prior probability model of natural images. Classically, both learning (by maximizing likelihood) and using (via Bayes' rule) such prior models are intractable due to the curse of dimensionality. Diffusion models take a different approach, where the prior density is replaced by a family of score vector fields across noise levels. They have led to impressive success in generative modeling, but the learned density is not explicit nor is it readily usable as a prior […]

Understanding how deep neural networks learn useful internal representations from data remains a central open problem in the theory of deep learning. We introduce Neural Low-Degree Filtering (Neural LoFi), a stylized limit of gradient-based training in which hierarchical feature learning becomes an explicit iterative spectral procedure. In this limit, the dynamics at each layer decouple: given the current representation, the next layer selects directions with maximal accessible low-degree correlation to the label. This yields a tractable surrogate mechanism for deep learning, together with a natural kernel-space interpretation. Neural LoFi provides […]