Inferring implicit sensorimotor costs by inverse optimal control with signal dependent noise

Normative computational models of sensorimotor behavior based on optimal feedback control with signal-dependent noise (Todorov, 2005) have been able to account for many phenomena including online corrections, redundancy, synergies, and uncontrolled manifolds of movements. In past research, a cost function needed to be assumed for the respective task, and agreement between the predictions of optimal behavior by the model and empirical movement trajectories had to be assessed. However, not all behavioral goals may be known a priori and costs internal to the subject such as biomechanical or subjective cognitive costs including effort are generally hard to measure independently. Thus, relating neuronal activities to sensorimotor behavior may miss crucial components.

Here, we show how a recently developed algorithm for inverse optimal control with signal-dependent noise allows inferring the cost function underlying behavior from observed trajectories. We use a formalization of sequential sensorimotor behavior as a partially observable Markov decision process and distinguish between the subject’s and the experimenter’s inference problems. Specifically, we employ a probabilistic formulation of the evolution of states and belief states and an approximation to the propagation equation in the linear-quadratic Gaussian problem with signal-dependent noise. We extend the model to the case of partial observability from the experimenter’s point of view, in which internal states of the subject are unobserved by the experimenter.

First, we validate the algorithm using synthetic data of eye and reaching movements. Then, we apply this framework to experimental reaching data to infer the cost functions implicit in the subject’s behavior. Additionally, we show how the subject’s dynamic perceptual belief throughout the experiment can be inferred. Taken together, our approach enables recovering the costs and rewards implicit in sequential sensorimotor behavior, thereby reconciling normative and descriptive approaches in a computational framework, which should be of great value to researchers in sensorimotor neuroscience.