1 h 5 min
Seminar, Hosted by Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University
Seeing Social Interactions
John Hopkins University
- Cognition, Computer Vision and
Recording
Abstract
Humans see the world in rich social detail, recognizing not only objects and people but also the social interactions between people. Perceiving and understanding others' interactions is critical for functioning in a social world, yet the underlying neural computations remain poorly understood. This talk argues that social interaction perception should be studied with the same computational vision tools now widely applied to scene and object recognition, and presents research using a large-scale naturalistic video dataset and a condition-rich fMRI experiment showing that social interaction information is extracted hierarchically along the recently proposed lateral visual pathway.
Ongoing work with the same dataset finds that, unlike static scene and object vision, current AI vision models match human behavior and neural responses to dynamic social scenes poorly. To help close this gap, the group developed SocialGNN, a graph neural network that instantiates insights from cognitive (neuro)science. SocialGNN reproduces human judgments of social interactions in controlled and natural videos using only visual information, without an explicit model of agents' minds or the physical world, and its relational, graph-based structure and processing are required for accurate recognition. The research suggests that social interaction recognition is a core human ability relying on structured visual representations, pointing towards more human-aligned AI.
Topics
- fMRI
- social interaction perception, broader topic: perception
- Computational Vision
- lateral visual pathway, broader topic: visual pathway
- naturalistic video, broader topic: video
- dynamic social scenes, broader topic: scenes
- AI vision models, search World Wide
- SocialGNN, search World Wide
- graph neural network
- structured visual representations, broader topic: visual representations