45 min
Seminar, Hosted by Alberta Machine Intelligence Institute
Selective Reinitialization Algorithms for Preventing Plasticity Loss in Artificial Neural Networks
University of Alberta
Recording
Abstract
J. Fernando Hernandez-Garcia studies how deep networks lose the ability to learn from changing data. Experiments span fully connected, convolutional and residual networks and vision transformers. The talk compares selective unit reinitialization with reinitializing individual weights, examining how the choice of utility measure, pruning and replacement affect plasticity and disruption of existing behavior.