HUMAN GAZE STRATEGY DURING VISUAL REASONING RELIES ON OBJECTS' AFFORDANCE UNDER THE TASK

Reasoning is the cognitive process use to make inferences based on knowledge. Gaze human behavior while performing reasoning and problem-solving tasks is highly bias and representative of the task performed. Informing of the stage of the reasoning processing. But the inner mechanisms that rely these cognitive abilities are still not fully understood.
In this study, we conducted a granular analysis of human ocular behavior during a visual reasoning task over features of abstract geometrical figures. We collected human gaze behavior using a video-oculogram while subjects had to read and asses the objects on the image, and its relations, to answer the question presented.
Our results so far reveal that human exploration during this visual reasoning task is sparse, efficient, and strictly object-centric—targeting only task-relevant semantic entities. Since the relevance of the entities depended on the question presented, these results indicate that visual scanpath during this task depends almost exclusively on image semantic value and their affordance in relation to the current question. The sparsity and efficiency during scanning suggest a relevant role of gist processing to deploy a gaze strategy to solve the task.
Moreover, we contrasted human scanpath with attention maps generated by a classical deep learning model on the same task. Our analysis reveals a fundamental dichotomy in information sampling. Despite achieving similar outputs, the underlying "visual evidence gathering" processes are distinct between humans and the artificial intelligence model. Altogether, these findings underscore the semantic efficiency of human visual strategies.
CENIA FB210017; ANID FONDECYT 11241551