POSTER DETAILS RELATED POSTERS

POSTER DETAILS

Predicting Local Brain Activity from Conversational Behaviours: A new Experimental Approach to Investigate the Neural Bases of Natural Social Interactions

Thierry Chaminade, Youssef Hmamouche, Magalie Ochs, Laurent Prevot

Date / Location: Monday, 11 July 2022 / S04-014
Add this poster event to your 🗓
Aims: “Second-person neuroscience” (Schilbach, 2013) requires new paradigms to study the neurophysiology of natural social interactions. fMRI was recorded while participants (n=24) discussed freely with a robot or a human. Acquired audio, video, eye tracking and physiological time-series synchronized with BOLD time-series (Rauchbauer, 2019) were used to identify which behavioural features are associated with brain responses across individually unique trials. Methods: Discretized local brain activity is predicted based on the behavioural recordings of each trial. Analysis included two steps, a feature extraction step to derive high-level features from raw signals, and the application of a dynamic prediction model to predict brain activity from the extracted features. 9-fold-cross-validation is performed on training data comprising eighteen participants to find the appropriate parameters of the Random Forest classifier based on the F-score, then tested on the six remaining participants. Results: We focused on a limited set of brain regions from the Brainnetome atlas (Fan, 2016) to validate the analysis, using behavioural features automatically extracted from linguistic exchanges, participants’ eye movements and conversational agents’ head and facial movements. Significantly accurate (p<0.001) predictions were found for the superior temporal sulcus, temporoparietal junction and precuneus bilaterally for Human-Human Interactions, while prediction was not significant for a white matter control region. Speech features dominated predictions for the temporal areas, while the interlocutor’s head and facial movements played an important role for prediction of the precuneus activity. Conclusions: This analysis demonstrates the possibility of associating complex combinations of behavioural features to local brain activity in unconstrained social interactions.

POSTERS YOU MIGHT BE INTERESTED IN

📃 Characterization of brain networks using functional ultrasound imaging
Flora Nelissen, Lorenzo De Angelis, ... , Valeria Gazzola
📃 M/EEG networks integration to elicit patterns of motor imagery-based BCI training
Marie-Constance Corsi, Mario Chavez, ... , Fabrizio De Vico Fallani