Statistical learning yields generalization and naturalistic behaviors in transitive inference

A hallmark of intelligence is the ability to infer how stimuli in the world are interrelated. Transitive inference (TI) tasks test if subjects can generalize transitively (A">"B and B">"C implies A">"C), an important case of such relational inference. Despite longstanding study, it remains an open question how the brain, or even statistical learning systems more generally, implements TI. Here we show that a wide range of statistical and biological learning models generalizes transitively and replicates key behavioral patterns observed in humans and animals performing TI. Further, we characterize analytically how specific model components give rise to these behaviors. From both formal analyses and simulations, we find that (i) models with factorized input representations necessarily perform TI as these models are constrained to express an orderly internal representation of the items in the task (a "rank" representation), (ii) models implementing standard statistical inductive biases (namely margin maximization, gradient descent on standard loss functions, and kernel ridge regression) not only perform TI, but also emergently express a rank representation, and (iii) over the course of learning, these models systematically combine memorization and the rank representation. Our account can parsimoniously explain three empirically observed behavioral patterns. Further, we (iv) find analytically that it explains transitive generalization and TI behaviors in neural networks (NNs) in the infinite-width limit and, through simulations, confirm this for slim NNs as well. Our findings show systematically how minimal statistical learning principles can explain the rich behaviors empirically observed in TI, a classic paradigm for investigating relational cognition.