Probabilistic Rewards
probabilistic rewards
Reward foraging task, and model-based analysis reveal how fruit flies learn the value of available options
Understanding what drives foraging decisions in animals requires careful manipulation of the value of available options while monitoring animal choices. Value-based decision-making tasks, in combination with formal learning models, have provided both an experimental and theoretical framework to study foraging decisions in lab settings. While these approaches were successfully used in the past to understand what drives choices in mammals, very little work has been done on fruit flies. This is even though fruit flies have served as a model organism for many complex behavioural paradigms. To fill this gap we developed a single-animal, trial-based decision-making task, where freely walking flies experienced optogenetic sugar-receptor neuron stimulation. We controlled the value of available options by manipulating the probabilities of optogenetic stimulation. We show that flies integrate a reward history of chosen options and forget value of unchosen options. We further discover that flies assign higher values to rewards experienced early in the behavioural session, consistent with formal reinforcement learning models. Finally, we show that the probabilistic rewards affect walking trajectories of flies, suggesting that accumulated value is controlling the navigation vector of flies in a graded fashion. These findings establish the fruit fly as a model organism to explore the genetic and circuit basis of value-based decisions.
The Desire to Know: Non-Instrumental Information Seeking in Mice
Animals are motivated to acquire knowledge. A particularly striking example is information seeking behavior: animals often seek out sensory cues that will inform them about the properties of uncertain future rewards, even when there is no way for them to use this information to influence the reward outcome, and even when this information comes at a considerable cost. Evidence from monkey electrophysiology and human fMRI studies suggests that orbitofrontal cortex and midbrain dopamine neurons represent the subjective value of knowledge during information seeking behavior. However, it remains unclear how the brain assigns value to information and how it integrates this with other incentives to drive behavior. We have therefore developed a task to test if information preferences are present in mice and study how informational value is imparted on stimuli. Mice are trained to enter a center port and receive an initial odor that instructs them to either go to an informative side port, go to an uninformative side port, or choose freely between them. The chosen side port then yields a second odor cue followed by a delayed probabilistic water reward. The informative port’s odor cue indicates whether the upcoming reward will be big or small. The uninformative port’s odor cue is uncorrelated with the trial outcome. Crucially, the two ports only differ in their odor cues, not in their water value since both offer identical probabilities of big and small rewards. We find that mice prefer the informative port. This preference is evident as a higher percentage choice of the informative port when given a free choice (67% +/- 1.7%, n = 14, p < 0.03), as well as by faster reaction times when instructed to go to the informative port (544ms +/- 21ms vs 795ms +/- 21ms, n = 14, p < 0.001). The preference for information is robust to within-animal reversals of informative and uninformative port locations, and, moreover, mice are willing to pay for information by choosing the informative port even if its reward amount is reduced to be substantially lower than the uninformative port. These behavioral observations suggest that odor stimuli are imparted with informational value as mice learn the information seeking task. We are currently imaging neural activity in orbitofrontal cortex with microendoscopes to identify changes in neural activity that may reflect value associated with the acquisition of knowledge.