Learning Cortical Hierarchies With Temporal Hebbian Updates

A key driver of mammalian intelligence is the ability to represent sensory inputs at different levels of abstraction. In deep learning, hierarchical neural networks learn such abstractions through top-down signals that drive synaptic plasticity. This is commonly done with backpropagation, an algorithm that is considered biologically implausible. In this work, we present a novel dynamic target propagation framework for hierarchical learning that uses Differential Hebbian (DH) updates, a rate-based version of Spike-Time-Dependent Plasticity. Our framework is based on the Deep Feedback Control (DFC) model and one of its extensions, where top-down feedback is used to shape neural activity, and learning relies on a prediction error calculated locally in each neuron. In contrast to the original DFC and similar bio-plausible learning models, our DH learning framework relays on experimentally observed learning rules, rather than on the comparison of somatic and dendritic activity which has not -- to the best of our knowledge -- been observed and its possible biological implementation remains unclear.

Both our DH framework and DFC are consistent with predictive coding, where learning is based on encoding predictions and errors. However, in contrast to standard predictive coding, our framework does not require explicit encoding of errors, as errors are encoded implicitly by the change of activity within the same unit. Relaxing this need for error encoding is relevant from an experimental neuroscience perspective since error encoding is a key source of criticism of predictive coding due to its lack of interpretability in experimental settings.

In summary, we propose a new framework that allows learning in multilayer networks with temporal Hebbian updates. We test our DH framework on a modest computer-vision benchmark (MNIST), showing competitive performance to backpropagation and its original multi-compartment DFC model. Finally, we discuss how the assumptions of our model relate to observations in experimental neuroscience.