Continual learning

ACML brings together research on the theory and practice of machine learning, with conference and journal publication tracks, invited talks, tutorials and workshops. The 2026 tutorial programme covers efficient and trustworthy reasoning with foundation models, robust statistical learning with alpha-divergence, topological deep learning, multimodal time series, generative methods for causal inference, distribution shifts, and continual learning toward self-evolving agents. Researchers from all regions are welcome. The event runs 1–4 December 2026. Its registration system lists an overall 08:00

Develop an independent research and teaching programme in fundamental machine learning. Relevant directions include post-transformer architectures, continual learning, probabilistic and causal models, uncertainty quantification, learning theory, symbolic-statistical methods, robust learning and scientific discovery. The appointee will supervise master’s and doctoral research, develop graduate teaching in Advanced Computer Science, and contribute to undergraduate machine learning and mathematics. The employer lists salary bands of £48,337–£61,165 and £64,878–£68,817.

…The discussion connects continual learning, efficient adaptation and automated science with the difficulty of evaluating tasks that lack easily checked answers. It also covers interfaces, distillation, open models, safety and regulation, multilingual tokenization, and possible successors to current Transformer systems. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

…between pretraining, reinforcement learning and learning after deployment. Drawing on his optimization work at Google and Anthropic, he examines why additional post-training, distillation and longer contexts may not solve continual learning. Other topics include second-order optimizers, distributed training, low-level coding agents and the difficulty of coordinating very large GPU clusters. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

…overfitting, persistent agent errors, continual learning, world models, and where information theory helps explain AI. Hosted by Ravid Shwartz Ziv and Allen Roush; the full conversation is available on the publisher's YouTube channe

…computation and reinforcement learning with verifiable rewards. The discussion connects these methods with self-consistency, model architectures, efficient attention and long contexts, while examining limits in continual learning and the reliability of agent workflows.

…accommodate new knowledge, a form of ‘continual learning’. Unlike mainstream artificial networks, which require complex and computationally expensive systems to achieve similar versatility, biological brains efficiently generalize prior learning, making inferences to acquire new tasks and perform multiple tasks based on context. Despite being an ideal species for neural investigation, mice have typically been thought to exhibit limited cognitive capacity for such complex learning p

Continual learning (CL), the capability to acquire knowledge and skills over time without forgetting old ones, is fundamental to how animals survive in a non-stationary world. This ability has proven difficult to realize in artificial neural networks (ANNs) [1], which suffer from catastrophic forgetting of previously learned information. To understand this contrast, we sought to understand what aspects of a neural system and the tasks it learns are key to its CL performance. We developed a theory of deep, nonlinear feedforward networks sequentially learning multiple tasks and derived exact exp

…these characteristic phenomena during learning remains elusive. We develop a model that optimizes for a normative goal, which is to maximize cumulative rewards. Place fields are modelled using Gaussian basis functions to represent spatial information of an environment, and directly synapse to an actor-critic for policy learning

…This continual learning remains a major challenge for artificial neural networks and has been suggested to require purpose-built mechanisms, such as memory replay or carefully designed learning rules. Sensorimotor learning provides a particularly useful testbed for theories of continual learning, as previous experiments have revealed a rich phenomenology of the complex and often intriguing dependence of motor adaptation on both rec

Reservoir computing is a machine learning framework in which only the output layer is trained to perform a task; the rest – composed of an input layer and recurrent neural network – has fixed connections. The benefit of restricting training to the output layer is that it does not require the computationally expensive and biologically-implausible backpropagation typically used to train deep-learning networks. However, while reservoirs can do tasks well, their performance is unreliable; depending on the random initialisation of fixed weights within the network, their capabilities vary wildly, si

The brain is remarkably adept at learning from a continuous stream of data without significantly forgetting previously learnt skills. Conventional machine learning models struggle at continual learning, as weight updates that optimize the current task interfere with previously learnt tasks. A simple remedy to catastrophic forgetting is freezing a network pretrained on a set of base tasks, and training task-specific readouts on this shared trunk. However, this assumes that representations in the frozen network are separable under new tasks, therefore leading to sub-par performance. To continual

…However, ANNs still struggle with continual learning, adaptable knowledge transfer, robustness, and resource efficiency - capabilities that biological systems handle seamlessly (3–8). Neuromodulators, the chemical messengers that target neurons and synapses in specific brain regions,

Continual learning involves a network adapting to new data while retaining previously learned information. In Artificial Neural Networks , this process faces the challenge of catastrophic forgetting, where updating parameters with new data causes an exponential decline in accuracy over time for previously learned tasks. Several methods have been proposed to address this problem and enable continual learning on future neuromorphic hardware. One such neuroscience-inspired method is metaplasticity, which refers to a network's ability to change the learning capacity of its synapses based on its cu

Recent studies have shown that synaptic spines undergo spontaneous size fluctuations, a phenomenon that is hypothesized to be critical to understanding synaptic plasticity and neural network stability [1,2]. In this work, we develop a model for spine size dynamics using a simple stochastic process based on minimal biological assumptions. This model effectively captures the key characteristics of spontaneous spine fluctuations, particularly replicating the observed lognormal distribution of spine sizes. We then use an artificial neural network structure where synaptic strengths fluctuate accor

Continual learning focuses on systems that incrementally acquire and update their knowledge. It is a pressing problem in machine learning as it is challenging to learn new tasks without forgetting knowledge about old ones, a phenomenon known as catastrophic forgetting [1]. Since biological systems are able to efficiently learn and update their knowledge continually during their lifetime, we are interested in principles inspired by these systems to try to tackle this challenge. One of the ideas in biology that is often overlooked in artificial neural networks is Dale’s principle [2], stating th

…primary visual cortex resemble that of continual learning in the presence of a large amount of drop

…how changes in neural activity during learning avoid interference with activity underlying similar behaviors. Recently, Losey et al. (bioRxiv, 2022) identified a possible explanation by examining monkey motor cortical population activity under a brain-computer interface (BCI) paradigm where linear mappings controlled a computer cursor. The activity produced for a known mapping was altered after learning a new mapping in a way that made the activity more beneficial for this new mapping. That is, the activi

Animals thrive in a constantly changing environment and leverage the temporal structure to learn well-factorized causal representations. In contrast, traditional neural networks have no representation of these changes and suffer from forgetting. In the prefrontal-thalamic circuit, reciprocal feedback connections produce a compressed thalamic representation of the computations performed in the prefrontal cortex. In inspiration, we use a neural network with a separate latent task representation and introduce a simple algorithm that uses optimization to rapidly infer and generate representations

…even when the animal isn’t overtly learning or forgetting[2]. While some researchers consider representational dynamics to be an inevitable consequence of ongoing learning, others suggest that drift may be beneficial. For instance, theoretical work by Kappel et al.,[3] suggests that if initial learning leads the system t

Over the course of a lifetime, we process a continual stream of information. Extracted from this stream, memories must be efficiently encoded and stored in an addressable manner for retrieval. To explore potential mechanisms, we consider a familiarity detection task where a subject reports whether an image has been previously encountered. We design a feedforward network endowed with synaptic plasticity and an addressing matrix, meta-learned to optimize familiarity detection over long intervals. We find that anti-Hebbian plasticity leads to better performance than Hebbian and replicates experim

Deep learning has made tremendous progress in the last year but it's high computational and memory requirements impose challenges in using deep learning on edge devices. There has been some progress in lowering memory requirements of deep neural networks (for instance, use of half-precision) but there has been minimal effort in developing alternative efficient computational paradigms. Inspired by the brain, Spiking Neural Networks (SNN) provide an energy-efficient alternative to conventional rate-based neural networks. However, SNN architectures that employ the traditional feedforward and feed