OPTIMAL FORAGING UNDER NATURALISTIC TEMPORAL DYNAMICS

Patch foraging, deciding when to leave a depleting resource to search for alternatives, is a fundamental aspect of animal behavior and offers a window into ethologically grounded decision processes. Several theories, most notably the Marginal Value Theorem (MVT), have proposed strategies for optimal foraging. However, they typically simplify the details of the spatiotemporal structure of the environment, and particularly the dynamics of patch replenishments. We investigate how richer replenishment dynamics affect optimal foraging decisions. We employ the average-reward reinforcement learning (RL) framework to find the optimal policy given various environmental statistics, and compare its behavior with MVT and other, more complex, heuristic policies. We show that under slow replenishment timescales, optimal policies leverage the statistics of the environment to generate higher reward rates, which depend on patterns of behavior distinct from both MVT and other fixed-threshold policies. In particular, the optimal policy applies flexible leave thresholds to each patch depending on the replenishment state of other patches, allowing it to maximize its acquisition of resources. These results suggest that in the presence of replenishment with realistic timescales, acting on the basis of a complete model of the environment is required to make optimal foraging decisions. Since slow replenishment is a realistic property of natural environments, we hypothesize that animals learn a world model and leverage it to inform their foraging decisions (perhaps, but not necessarily, using model-based RL). We are testing this hypothesis in mice using a naturalistic patch-foraging paradigm designed based on our theoretical predictions.