Equality reasoning in neural networks is modulated by learning richness

Equality reasoning is ubiquitous and purely abstract -- sameness or difference may be evaluated no matter the nature of the underlying objects. As a result, same-different tasks (SD) have been extensively studied as a starting point for understanding abstract reasoning in humans, and comparatively across animal species. With the rise of neural networks (NN) that exhibit a striking apparent proficiency for abstractions, equality reasoning in NNs has also gained interest. Yet despite the extensive study, conclusions about equality reasoning vary widely. Some authors claim that SD can be learned only with great difficulty through extensive language experience or specialized training (in great apes and young children). Others demonstrate that SD can be easily learned from a small number of examples (in bees and pigeons). In neural networks, most claim that NNs fail to generalize on equality tasks, though a minority suggest otherwise. To clarify the underlying principles in learning SD, we develop a theory of equality reasoning in multi-layer perceptrons (MLP), the simplest neural network architecture. We identify a normative solution to SD expressible by MLPs that generalizes perfectly to unseen inputs. Crucially, we demonstrate that MLPs indeed learn this ideal solution but only with sufficiently *rich* feature learning, corresponding to adaptive, task-specific representations. Rich feature learning is in contrast to *lazy* learning, typically characterized by high-dimensional, fixed, and task-agnostic representations. By interpolating the MLP between rich and lazy learning, we demonstrate that SD generalization breaks down in the lazy regime. We validate these findings in vision SD tasks where neural networks have been documented to struggle, instead showing that rich feature learning promotes success. Overall, our work identifies feature learning richness as a key parameter modulating equality reasoning, and suggest that equality reasoning in humans and animals may similarly depend on learning richness in neural circuits.