Cortically motivated recurrence enables visual task extrapolation

Biological neural networks use an abundance of “recurrent” connections, yet state-of-the-art deep neural network based computer vision models are predominantly feedforward. Why does biological vision evolve to employ recurrence pervasively? In this paper, we aim to address this question from a machine learning and modeling framework highlighting a critical functional advantage of employing recurrent computations: we show that a recurrent network is naturally able to adapt its computational budget flexibly during inference and generalizes in a zero-shot manner within-task across difficulties. We contribute LocRNN, a recurrent neural network module that is designed based on a prior computational model of biological recurrent intracortical connections (Li, 1998). LocRNN learns highly accurate solutions to Mazes and PathFinder -- two challenging vision benchmarks inspired by prior psychophysics tasks to test DNNs' ability to integrate long-range spatial dependencies. LocRNN is able to flexibly use more recurrent iterations during inference to zero-shot generalize to more difficult instantiations of each task (compared to training difficulty level) without requiring extra training data, a potential functional advantage of recurrence that biological visual systems capitalize on. Our ablation study of LocRNN highlights the fundamental importance of interneurons, piecewise linear activation, and recurrent gating. Our work encourages further study of the role of recurrence as an important biological mechanism underlying domain generalization, task extrapolation, and the incorporation of biological insights to design DNNs.