Topic: synaptic consolidation

Humans have the ability to quickly adapt to new tasks and generalise what they learned to improve the learning process itself. At the core of this capacity for meta-learning lies a difficult credit assignment problem. After a learning episode, the system needs to determine how to change the components of plasticity such that the next time a similar task is encountered, a better learning outcome can be arrived at more quickly. In machine learning, this problem is often approached by storing the entire learning trajectory and revisiting it in reverse-time order, a process that places large deman

The brain is remarkably adept at learning from a continuous stream of data without significantly forgetting previously learnt skills. Conventional machine learning models struggle at continual learning, as weight updates that optimize the current task interfere with previously learnt tasks. A simple remedy to catastrophic forgetting is freezing a network pretrained on a set of base tasks, and training task-specific readouts on this shared trunk. However, this assumes that representations in the frozen network are separable under new tasks, therefore leading to sub-par performance. To continual