ePoster, FENS Forum 2024
GPT-4 can recognize Theory of Mind in natural conversations: fMRI evidence
- Messe Wien Exhibition & Congress Center, Vienna, Austria
- Neuroscience, Artificial Intelligence and
Abstract
The success of Large Language Models (LLM) has led to controversial hypotheses about emergent intelligence beyond language competence, amongst which Theory of Mind (ToM) has become the most debated. While some reported gpt-4 to pass the Sally-Anne test, others reported limitations. Here we aim to validate LLM ability to recognise ToM occurrences in natural human dialogue using the activity of ToM-associated brain regions.The current analysis uses an existing corpus of 25 participants recorded with functional MRI while carrying conversations in 4 sessions x 6 trials alternating a Human and a Robot interlocutor. The conversation is considered natural as participants are provided with a cover story hiding the actual objective of the experiment to study social interactions, and is known to elicit ToM phenomena.We analysed the resulting 100 hours of transcribed conversations synchronised with fMRI recordings by a bespoke gpt-4-turbo prompt. Each conversation turn from the participant and its interlocutors were tagged as containing (ToM+) or not (ToM-) references to mental states by the LLM. Speakers (2) and ToM category (2) defined the four conditions used in a general linear model analysis of whole-brain response.Contrast between conversation turns tagged ToM+ versus ToM- by gpt-4 revealed significant activations (p<0.1 FWE-corrected at the cluster-level ) in the left TemporoParietal Junction, Medial PreFrontal Cortex and lateral OrbitoFrontal Cortex, aligning with the implementation of mentalizing mechanisms in the human brain. Our contribution differs from previous work in the ecological nature of the task, which contrasts with studies based on dedicated ToM tests.
Topics
- Large Language Models, search World Wide
- Theory of Mind, search World Wide
- gpt-4-turbo, search World Wide
- Sally-Anne test, search World Wide
- natural human dialogue, search World Wide
- functional MRI, search World Wide
- TemporoParietal Junction, search World Wide
- Medial PreFrontal Cortex, search World Wide
- mentalizing, search World Wide
- general linear model, search World Wide