Violations of transitivity disrupt relational inference in humans and reinforcement learning models

Humans are capable of generalising local pairwise relations to transitively infer global relationships between items, states or events. Recent research has shown that asymmetric learning, where beliefs about only the winners (or losers) of local comparisons are updated, is well-suited to allow human and artificial agents to infer relational structures from sparse feedback. However, less is known about how already-acquired knowledge and subsequent learning policies are impacted by experiences that violate the inferred monotonic ordering of items, such as when agents learn that A is greater than B and B is greater than C, but then are later told that C is greater than A. Here, we show that introducing a single such ‘uroboros’ relation into a monotonically ordered set of items disrupted humans’ inferred relational knowledge about the remaining items. Fitting simple reinforcement learning (RL) models to human participant data replicated the asymmetries previously observed in inferential learning, but also captured the uroboros-induced ‘unlearning’ of transitive relations. Despite this disruption to the learned value structure, direct (non-transitive) relational learning of neighbouring items was preserved. This effect was captured by augmenting asymmetric RL with an episodic memory-like process that learns directly experienced pair relations. In explorative analyses, we find no conclusive evidence for a learning policy change in response to the uroboros relation, e.g. in terms of learning rate changes. Our results indicate that asymmetric RL of item values, complemented by direct pair-level memory, not only accounts for efficient inference of latent relational structure, but also for rapid unlearning of inferred relations in response to surprising feedback.