How Neural Networks Learn Features: Low-Degree Correlations and Collective Dynamics

Yatin Dandi traces a line of work on how neural networks learn features, from hidden directions in two-layer models to the intermediate representations that let deeper networks learn complex targets with far fewer samples. A unifying theme is that networks learn features through low-degree correlations in the data. This insight led to Neural Low-Degree Filtering, a spectral algorithm that describes early layer-wise learning on real data, and to its extension to the joint training of layers. The talk closes with approaches to understanding the full dynamics of hierarchical learning across layers, drawing on a rigorous version of the dynamical cavity method. The talk is one of the Kempner Institute Research Fellow candidate presentations.