A RESOURCE-RATIONAL ACCOUNT OF HUMAN EYE MOVEMENTS DURING IMMERSIVE VISUAL SEARCH

Human visual search in naturalistic environments requires the brain to coordinate perception, attention, and action under severe computational constraints, yet most models of eye movements have been developed for highly simplified stimuli. Here, we present a resource-rational account of eye movements during immersive visual search in virtual reality. Human participants searched for target objects embedded in complex 3D indoor scenes while their gaze was tracked. We formalized visual search as a meta-level Markov decision process in which fixations are treated as costly information-gathering actions that update beliefs over object identity. To solve this problem in high-dimensional scenes, we trained deep reinforcement learning agents over different object representations, including explicit shape and color features and embeddings derived from a pre-trained convolutional neural network, providing one of the few examples of reinforcement learning applied to naturalistic, embodied visual behavior. Agents consistently learned a belief-guided policy that fixates the object with the highest posterior probability of being the target and terminates search when confidence exceeds a threshold, closely approximating classic ideal-observer models developed for simplified tasks. When simulated forward, this policy reproduced key aspects of human behavior, including accuracy, reaction time distributions, fixation sequences, and gaze transition structure. Critically, the degree of alignment between model and human gaze depended on the representational features available to the agent, with CNN-based embeddings providing the best quantitative match. These results suggest that eye movements in complex scenes reflect resource-rational sequential decision-making over structured visual representations, offering a unifying computational account of naturalistic visual search.