Topic: Hebb rules

To learn a new task and predict accurately on new stimuli, the brain must extract patterns from limited observations. The learning rule and initial synaptic weights determine how well a network will generalize on new data. Here, we investigate how initial synaptic variability and representation learning affect the learning curves of neural networks (generalization error vs sample size). We show that two scaling regimes hold for many plausible learning rules including feedback alignment (FA), direct feedback alignment (DFA) and Hebb rules, as well as implausible gradient descent (GD). In the sm