Topic: efference copy

I will discuss work with Jack Lindsey modeling reinforcement learning for action selection in the basal ganglia. I will argue that the presence of multiple brain regions, in addition to the basal ganglia, that contribute to motor control motivates the need for an off-policy basal ganglia learning algorithm. I will then describe a biological implementation of such an algorithm that predicts tuning of dopamine neurons to a quantity we call "action surprise," in addition to reward prediction error. In the same model, an implementation of learning from a motor efference copy also predicts a novel

During the course of an animal’s interaction with its environments, activity within central neural circuits is orchestrated exquisitely to structure goal-oriented movement. During walking, for example, the head, body and limbs are coordinated in distinctive ways that are guided by the task at play, and also by posture and balance requirements. Hence, the overall performance of goal-oriented walking depends on the interplay between task-specific motor plans and stability reflexes. Copies of motor plans, typically described by the term efference copy, modulate stability reflexes in a predictive