SIMILARLY NONIDEAL: SHARED STRATEGIES IN HUMAN AND ARTIFICIAL VISUAL INFERENCE

Most work in Neuro-AI focuses on matching the performance of biological systems in behaviorally relevant tasks while comparing how computations are implemented in biological and artificial agents. Less explored are the strategies emerging from such implementations, which can reveal the algorithmic fit of a learning system to specific task demands.
We use a visual version of the planted clique problem (PCP), a theoretical computer science problem with known theoretical performance bounds. We tested humans, deep neural networks (DNNs), and a family of Bayesian ideal observers (IOs) that solve the task by attending to the graph’s degree distribution or the spatial correlations of its adjacency matrix. Comparing humans, DNNs and IOs with the problem’s bounds and analyzing their behavioral patterns reveals three performance regimes. Among DNNs, the most flexible and structurally agnostic system performs worst. In contrast, agents biased towards local spatial correlations – such as humans and certain DNNs – achieve intermediate performance, matched by the correlation observer, confirming a shared visual-correlation strategy. Only IOs that attend to the degree distribution approach the computational bound defined by the performance of the best known algorithm (Approximate Message Passing, AMP).
These results show how structural biases and priors shape the algorithmic capacities of humans and DNNs in visual inference and how some classes of problems remain inherently difficult to tackle with machine learning approaches. The PCP provides a rigorous framework for studying visual inference from an algorithmic perspective and, going forward, could serve as a testbed for aligning biological and artificial systems.