Topic: hidden states

When faced with a constantly changing, uncertain environment it is necessary to infer its underlying structure to guide behaviour. This has often been formalised as a value updating problem – where actions are chosen based on an incrementally updated weighted average of past reward. Activity of midbrain dopamine neurons has been commonly associated with the error in such value prediction, and it has been proposed to be crucial for learning and updating of values that inform decision making. However, it has become increasingly apparent that both humans and animals often use an alternative strat

The orbitofrontal cortex (OFC) has long been considered critical for value-based decision making, but its precise role is a point of contention. One hypothesis is that OFC computes subjective value and drives economic choice [1]. A second hypothesis is that OFC represents a cognitive map or state space (i.e., a representation of the different states in a task and the transitions between them) [2]. We trained rats on a novel temporal wagering task with partially hidden states (blocks of trials with low or high rewards). Rats must determine how long to wait for a reward, providing an explicit be