Topic: deep reinforcement learning

How does the motor cortex (MC) achieve generalizable and purposeful movements from the complex non-linear musculoskeletal system? This understanding can help us uncover how does the cortical activity relates to the movement. To elucidate this relationship, we build a model of the MC with realistic properties. To mimic the recurrent connections between cortical neurons, we use recurrent neural networks (RNNs) to model the MC. The MC model receives sensory feedback and delivers muscle excitations to anatomically accurate musculoskeletal models to produce experimentally observed movements. The se

The representation of “what happened when” is central to encoding episodic and working memories. Recently discovered hippocampal “time cells” are theorized to provide the neural substrate for such representations by forming distinct sequences that encode both time elapsed and sensory content. However, while multiple neurophysiological studies have presented contradictory results on the role of “time cells” in memory, little work has directly addressed this discrepancy. Here, we hypothesize that this discrepancy is a result of different studies using tasks that involve different cognitive deman

When an agent is trained on a complex episodic task, different task parts may be learned at different rates. How do we determine which part of the task challenged the agent the most? Since reward is provided usually only at the end of each trial, it cannot be used to infer within-trial learning trends. Behavioral features such as speed or trial duration capture trends in the agent's decision-making, but do not necessarily indicate that the agent is getting better at the task. We address this issue by modeling the task as a Markov Decision Process (MDP). The Q values of the optimal policy measu

In the deep reinforcement learning (RL) community, motor control problems are usually approached from a reward-based learning perspective. However, humans are often believed to learn motor control through directed error-based learning. Within this learning setting, the control system is assumed to have access to exact error signals and their gradients with respect to the control signal. This is unlike reward-based learning, in which errors are assumed to be unsigned, encoding relative successes and failures. Here, we try to understand the relation between these two approaches, reward- and erro

The last few years have seen some dramatic developments in artificial intelligence research. What implications might these have for neuroscience? Investigations of this question have, to date, focused largely on deep neural networks trained using supervised learning, in tasks such as image classification. However, there is another area of recent AI work which has so far received less attention from neuroscientists, but which may have more profound neuroscientific implications: Deep reinforcement learning. Deep RL offers a rich framework for studying the interplay among learning, representation

Central place foraging insects like the honeybee (Apis mellifera) are the masters of visual navigation of the insect world: They are able to reliably return to their nest under a wide range of visual conditions after only a few learning flights, (a feat so far unrivalled by any artificial system). Furthermore, they perform complex navigational tasks, such as ‘computing’ novel shortcuts between previously known but unrelated food sources, supporting claims about the existence of a ‘cognitive map’ in honeybees. This raises the question which internal representations of the world (and the animal’

Animals often make use of information from multiple sensory systems, such as vision, proprioception, olfaction, and audition to guide their behavior. They must thus integrate these different sensory inputs or select one of them in order to effectively make decisions in the environment. How exactly the brain accomplishes this sensory integration and to what extent each signal contributes under what circumstances remains to be fully understood. We use a deep reinforcement learning model to study sensory integration in a navigation task in two different scenarios. In the first scenario, there is