Topic: multiple timescales

In order to thrive in complex environments, animals and artificial agents have to learn to act adaptively to maximize fitness and rewards. Reinforcement learning (RL) algorithms have been successful at training artificial agents to superhuman performance in an array of complex tasks and the activity of dopaminergic neurons has been described by one of their key components, the reward prediction error (RPE). However, classical RL occurs at a single timescale, set by the discount factor, while agents have to make decisions at multiple timescales and behaviorally, humans and animals are known to

Learning and memory consolidation in the brain occur over multiple timescales. Inspired by this observation, it has been shown that catastrophic forgetting in reinforcement learning agents can be mitigated by consolidating Q-value function parameters at multiple timescales. In this work, we combine this approach with successor features, and show that by consolidating successor features and preferences learned over multiple timescales we can further mitigate catastrophic forgetting. In particular, we show that agents trained with this approach rapidly recall previously rewarding sites in large

The temporal activity of neural circuits exhibits fluctuations simultaneously varying over a large range of timescales (Murray et al. 2014). Recent experimental evidence presents a heterogeneity of timescales across neurons within the same local cortical circuit, ranging from milliseconds to minutes (Bernacchia et al. 2011). While this phenomenon is well documented, the underlying neural mechanism is still unknown. We present a novel random neural network whose units represent functional neural clusters of different sizes, given by a heterogeneous distribution of self-couplings. We find that t

The simple decision to seek and consume food requires integrating slowly changing information about internal state with rapid sensory processes indicating food availability. Whether fast and slow components unite in a single integrating location [2] or whether they are also integrated in a more distributed fashion remains unknown. Can neural population dynamics simultaneously encode multiple timescales? Could modeling such temporal multiplexing help us understand how the brain integrates temporal information to guide appetitive behavior? To examine these questions, we recorded neural populati