Topic: catastrophic forgetting

Neuronal assemblies — groups of neurons with strong recurrent connectivity — are thought to be the basic building blocks of perception and memory in the brain: representations of specific concepts. Activation of an assembly signals the presence of its associated concept to downstream targets. An efficient implementation should allow similar concepts to be represented by similar assemblies, sharing a subset of neurons. However, learning overlapping assemblies in computational models is difficult. Similar assemblies tend to either merge or diverge during the learning process. The same mechanism

Learning and memory consolidation in the brain occur over multiple timescales. Inspired by this observation, it has been shown that catastrophic forgetting in reinforcement learning agents can be mitigated by consolidating Q-value function parameters at multiple timescales. In this work, we combine this approach with successor features, and show that by consolidating successor features and preferences learned over multiple timescales we can further mitigate catastrophic forgetting. In particular, we show that agents trained with this approach rapidly recall previously rewarding sites in large

Metaplasticity is the change in plasticity of a synapse, and an important mechanism for resolving the stability-plasticity dilemma. However, the exact synaptic or neuron properties that modulate metaplasticity still remains unclear. In this work we use meta-continual learning to discover the importance of factors that contribute to metaplasticity. We use a linear model to assign the relative contribution of four commonly used synaptic properties to metaplasticity: the Hessian, gradient, magnitude of the weights, and activity of the post-synaptic neuron. The coefficients of the linear model ar

In animals and humans, curriculum learning---presenting data in a curated order---is critical to rapid learning and effective pedagogy. A long history of experiments has demonstrated the impact of curricula in a variety of animals---including rats, mice, dogs, pigeons and humans---but, despite its ubiquitous presence, a theoretical understanding of the phenomenon is still lacking. Surprisingly, in contrast to animal learning, curricula strategies are rarely used in machine learning and recent simulation studies conclude that curricula are moderately effective or ineffective in most cases. T

The brain is remarkably adept at learning from a continuous stream of data without significantly forgetting previously learnt skills. Conventional machine learning models struggle at continual learning, as weight updates that optimize the current task interfere with previously learnt tasks. A simple remedy to catastrophic forgetting is freezing a network pretrained on a set of base tasks, and training task-specific readouts on this shared trunk. However, this assumes that representations in the frozen network are separable under new tasks, therefore leading to sub-par performance. To continual

Learning is fundamental for interacting with a changing environment. Critical to this process is the ability to rapidly adapt previously acquired skills to external perturbations while avoiding catastrophic forgetting of previous tasks. For the learning system, this corresponds to a trade-off between being able to maintain stable memories of learned tasks, and the ability to swiftly adapt such memories to changes. We hypothesize that such a problem can be addressed by using a two-timescale [1,2] strategy, where memorization and adaptation are performed by two separate modules. This structure a

Continual learning involves a network adapting to new data while retaining previously learned information. In Artificial Neural Networks , this process faces the challenge of catastrophic forgetting, where updating parameters with new data causes an exponential decline in accuracy over time for previously learned tasks. Several methods have been proposed to address this problem and enable continual learning on future neuromorphic hardware. One such neuroscience-inspired method is metaplasticity, which refers to a network's ability to change the learning capacity of its synapses based on its cu

Continual learning focuses on systems that incrementally acquire and update their knowledge. It is a pressing problem in machine learning as it is challenging to learn new tasks without forgetting knowledge about old ones, a phenomenon known as catastrophic forgetting [1]. Since biological systems are able to efficiently learn and update their knowledge continually during their lifetime, we are interested in principles inspired by these systems to try to tackle this challenge. One of the ideas in biology that is often overlooked in artificial neural networks is Dale’s principle [2], stating th

Recurrent Neural Networks (RNN) are useful models to study neural computation. Several approaches are available to train RNNs on neuroscience-related tasks and reproduce neural population dynamics. However, a comprehensive understanding of how the generated dynamics emerge from the underlying connectivity is largely lacking. Previous work derived such an understanding for specific types of constrained RNNs (Mastrogiuseppe and Ostojic, 2020; Curto et al., 2019; Biswas and Fitzgerald, 2020), but if and how the resulting insights apply more generally is unclear. In this work, we study how network

Continual learning (CL), the capability to acquire knowledge and skills over time without forgetting old ones, is fundamental to how animals survive in a non-stationary world. This ability has proven difficult to realize in artificial neural networks (ANNs) [1], which suffer from catastrophic forgetting of previously learned information. To understand this contrast, we sought to understand what aspects of a neural system and the tasks it learns are key to its CL performance. We developed a theory of deep, nonlinear feedforward networks sequentially learning multiple tasks and derived exact exp