POSTER DETAILS
Decoding the Neural Correlates of Musical Predictions with Deep Learning
Mathieu Pham Van Cang, Keith B. Doelling, Luc Arnal
Date / Location: Monday, 11 July 2022 / S04-077
Add this poster event to your 🗓
Add this poster event to your 🗓
Our acoustic environment is full of statistical regularities. According to the predictive coding (PC) theory, the human brain uses such predictable features to generate expectations about future stimuli to optimize perception. Music is a complex naturalistic stimulus containing rhythmic and harmonic regularities and constitutes an ideal framework for studying the neural underpinnings of PC. To study neural correlates of temporal predictions in music listening, we capitalize on deep learning models to extract relevant features of predictive processing. We collected magnetoencephalography (MEG) data on 27 normal-hearing subjects listening to piano performances repeated throughout the experiment. We then assessed the same musical pieces in terms of continuous predictive features, namely surprisal and entropy. Those mathematical quantities were estimated by a recurrent neural network originally designed to generate piano performances. This model is particularly promising as compared with methods commonly used in the field: (i) it relies on minimal prior musical knowledge and (ii) it efficiently works on polyphonic music. Validating this approach on behavioral data confirms that this model efficiently predicts continuous human surprisal ratings to the same music. Using a temporal response function – a linear model mapping the continuous stimulus features to the MEG data – we quantify the relative influence of exogenous (stimulus driven) and endogenous (prediction driven) neural signals underlying processing of musical sequences. Finally, comparing the signatures of ongoing (probabilistic) versus memory-based (experiential) expectations, we distinguish two categories of commonly confounded priors to further delineate the spatial and spectrotemporal architecture of predictive processing in the human brain.