ePoster, COSYNE 2023
A normative framework for balancing reward- and information-seeking behaviors in dynamic environments
Nicholas Barendregt, Zachary Kilpatrick, Joshua Gold and Kresimir Josic
University of Colorado Boulder; Applied Mathematics; University of Pennsylvania; University of Houston
- Montreal, Canada
- Computational Neuroscience, Cognition and
Abstract
Our understanding of how the brain makes decisions has benefited greatly from normative theories that have, for example, established benchmarks for how specific forms of evidence accumulation and decision rules can optimally balance quantities like decision speed and accuracy. These theories have recently been expanded to include adaptive components that are necessary for effective decision making in dynamic environments. However, these extensions remain incomplete, because they have focused primarily on how an unconstrained agent manages single, independent decisions [1]. In contrast, real-world decision-makers are often constrained by limitations on resources like energy and time. These constraints can have complicated effects on not just single decisions, but also on sequences of decisions in volatile environments. Here we present a new theoretical framework for understanding quantitatively, and optimizing with respect to, the costs and benefits of different sequential decision-making behaviors in dynamic environments.
Our framework is based on a Bayesian model of decision making in environments with temporally correlated structure. Specifically, we assumed that after each trial, the correct choice changes according to a discrete-state Markov process, and that the agent can take a limited number of actions per block of trials. The resulting rich, optimal choice behaviors depend strongly on task conditions, but generally exhibit the explore-exploit trade-off characteristic of many foraging tasks. Our major contribution is a predicted, sharp transition from exploratory to exploitative strategies when choice feedback provides more information about the next trial than the expected environmental evidence, and thus a comparison of reward-optimal and information-optimal strategies [2]. More generally, our work provides a broad, quantitative account, including experimentally testable predictions, of the effectiveness of different decision strategies under natural constraints.
Topics
- decision making, search World Wide
- normative theories, search World Wide
- evidence accumulation, search World Wide
- Bayesian model, search World Wide
- sequential decision-making, search World Wide
- dynamic environments, search World Wide
- temporally correlated structure, search World Wide
- discrete-state Markov process, search World Wide
- explore-exploit trade-off, search World Wide
- reward-optimal and information-optimal strategies, search World Wide
Details
- Session
- Poster Session III