Topic: feedback alignment

To learn a new task and predict accurately on new stimuli, the brain must extract patterns from limited observations. The learning rule and initial synaptic weights determine how well a network will generalize on new data. Here, we investigate how initial synaptic variability and representation learning affect the learning curves of neural networks (generalization error vs sample size). We show that two scaling regimes hold for many plausible learning rules including feedback alignment (FA), direct feedback alignment (DFA) and Hebb rules, as well as implausible gradient descent (GD). In the sm

Training deep neural networks typically relies on backpropagating high-dimensional error signals---a computationally intensive and biologically implausible process. However, since most tasks involve low-dimensional outputs, we propose that low-dimensional error signals may suffice for effective learning. To test this hypothesis, we introduce a novel local learning rule based on Feedback Alignment that leverages indirect, low-dimensional error feedback to train large networks. Our method decouples the backward pass from the forward pass, enabling precise control over error signal dimensionality