Skip to contentSkip to filters

We use cookies for analytics.

Seeing Social Interactions

Leyla Isik

John Hopkins University

Recording

1 h 5 min

Abstract

Humans see the world in rich social detail, recognizing not only objects and people but also the social interactions between people. Perceiving and understanding others' interactions is critical for functioning in a social world, yet the underlying neural computations remain poorly understood. This talk argues that social interaction perception should be studied with the same computational vision tools now widely applied to scene and object recognition, and presents research using a large-scale naturalistic video dataset and a condition-rich fMRI experiment showing that social interaction information is extracted hierarchically along the recently proposed lateral visual pathway.

Ongoing work with the same dataset finds that, unlike static scene and object vision, current AI vision models match human behavior and neural responses to dynamic social scenes poorly. To help close this gap, the group developed SocialGNN, a graph neural network that instantiates insights from cognitive (neuro)science. SocialGNN reproduces human judgments of social interactions in controlled and natural videos using only visual information, without an explicit model of agents' minds or the physical world, and its relational, graph-based structure and processing are required for accurate recognition. The research suggests that social interaction recognition is a core human ability relying on structured visual representations, pointing towards more human-aligned AI.

Topics