Seminar, Hosted by One World Mathematics of INformation, Data, and Signals (1W-MINDS) Seminar
Connectedness and non connectedness of zero error sets in ReLU neural networks
UW-Madison
- Topology, Deep Learning and
Abstract
Chenxi Wu studies the topology of the set of parameters at which a feedforward ReLU network fits a finite sample with zero loss, building on earlier connectivity results for loss sublevel sets by Freeman, Bruna, Nguyen, Hein and others. In joint work with Eli Grigsby and Kathryn Lindsey, the zero-loss set is shown to be path connected whenever every hidden layer is at least as wide as the number of samples, and this width bound is optimal: for shallow networks whose hidden layer has one neuron fewer than the sample size, the sample points can be chosen so that the set is disconnected. The talk also presents further examples of the mechanism behind disconnectedness and discusses what these may imply for training dynamics.