A neural model for hierarchical and counterfactual information processing inspired by human behavior

The human brain finds solutions to complex multi-stage decision problems that are far more flexible than those learned by artificial systems. Cognitive theories attribute this flexibility to specific algorithms such as hierarchical information processing and counterfactual reasoning. We first developed a moderately challenging multi-stage decision-making task as a baseline paradigm for evaluating the computational basis of cognitive strategies in humans. Comparing behavior to inference models implementing various cognitive algorithms, we found that humans use a hierarchical strategy to solve the task sequentially, and when uncertain, revise their decisions by considering counterfactuals. Next, we used a set of experiments to reverse engineer the constraints from which these algorithms derive. Experiment 1 indicated that the hierarchical strategy results from a processing bottleneck associated with processing parallel streams of evidence. Experiment 2 indicated that compensatory counterfactuals are imperfect due to working memory (WM) limits. Experiment 3 indicated that humans are computationally rational in that the degree to which they rely on counterfactuals depends on how good their working memory is. To test the causal role of these computational constraints, we next asked whether imposing such constraints on self-supervised artificial recurrent neural network (RNN) models would lead to human-like solutions in RNNs. Confronted with the original maze task, RNNs established solutions identical to humans: sequential decisions followed by suboptimal counterfactuals. To test for causality, we developed a battery of RNNs in which we relaxed one or more of these constraints. Analysis of these lesioned models indicated that processing bottlenecks, working memory noise, and self-supervision were crucial constraints for the emergence of human-like strategies in RNNs. Finally, we found that distinct cognitive algorithms (e.g., hierarchical and counterfactual) are mere subdivisions in a strategy continuum including optimal, counterfactual, postdictive, and hierarchical, that the RNN may adopt depending on the level of noise in working memory.