A complementary systems theory of meta-learning

Humans have the ability to quickly adapt to new tasks and generalise what they learned to improve the learning process itself. At the core of this capacity for meta-learning lies a difficult credit assignment problem. After a learning episode, the system needs to determine how to change the components of plasticity such that the next time a similar task is encountered, a better learning outcome can be arrived at more quickly. In machine learning, this problem is often approached by storing the entire learning trajectory and revisiting it in reverse-time order, a process that places large demands on memory and is non-local in time. Here, we propose that hippocampal replay enables meta-learning in the neocortex without storing the entire learning trajectory. By contrasting the outcome of learning with the outcome of an auxiliary learning problem prescribed by the hippocampus, neocortical learning can extract long-term credit assignment information with causal synaptic plasticity rules that only require temporarily buffering one intermediate state. Our framework can be understood as a generalisation of contrastive Hebbian learning. It is agnostic to the underlying learning procedure and can accommodate different models of meta-plasticity. We study two distinct models that cast synaptic consolidation as meta-learning either through synaptic changes on multiple timescales or in combination with a model of top-down modulation. Testing our theory on few-shot prediction problems and reward-based learning tasks reveals that our plasticity rules configure the slow components to enable fast adaptation and generalisation. Our theory extends the conventional view on the hippocampus and the neocortex as complementary learning systems, providing a unified perspective of systems-level consolidation and synaptic consolidation.