Department of Mathematics

Search

DACO seminar

×

Modal title

Modal content

Please subscribe here if you would you like to be notified about these events via e-mail. Moreover you can also subscribe to the iCal/ics Calender.

Autumn Semester 2026

Date / Time Speaker Title Location
* 8 October 2026
13:45-15:00 		Yatin Dandi
EPFL
Details

DACO Seminar

Title Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
Speaker, Affiliation Yatin Dandi, EPFL
Date, Time 8 October 2026, 13:45-15:00
Location TBD
Abstract Understanding how deep neural networks learn useful internal representations from data remains a central open problem in the theory of deep learning. We introduce Neural Low-Degree Filtering (Neural LoFi), a stylized limit of gradient-based training in which hierarchical feature learning becomes an explicit iterative spectral procedure. In this limit, the dynamics at each layer decouple: given the current representation, the next layer selects directions with maximal accessible low-degree correlation to the label. This yields a tractable surrogate mechanism for deep learning, together with a natural kernel-space interpretation. Neural LoFi provides a mathematically explicit framework for studying multi-layer feature learning beyond the lazy regime. It predicts how representations are selected layer by layer, explains how emergence of concepts arises with given sample complexity, and gives a concrete mechanism by which depth progressively constructs new features from old ones through low-degree compositionality. We complement the theory with mechanistic experiments on fully connected and convolutional architectures, showing that Neural LoFi improves over lazy random-feature baselines, recovers meaningful structured filters, and predicts representations aligned with early gradient-descent feature discovery with real datasets. The talk is based on joint work with Matteo Vilucchio, Luca Arnaboldi, Hugo Tabanelli, and Florent Krzakala (https://arxiv.org/abs/2605.13612).
Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learningread_more
TBD

Notes: the highlighted event marks the next occurring event and events marked with an asterisk (*) indicate that the time and/or location are different from the usual time and/or location.

Organisers:

Archive: AS 26 SS 26 AS 25 SS 25 AS 24 SS 24 AS 23 SS 23 AS 22 SS 22 AS 21 SS 21 AS 20 SS 20

JavaScript has been disabled in your browser