Bayesian integration of audiovisual speech by DNN models is similar to human observers
Haotian Ma, Xiang Zhang, Zhengjia Wang, John F. Magnotti, Michael S. Beauchamp · COSYNE 2025
We investigated whether transformer-based machine learning (ML) models could have similar behaviors to humans and predict electrophysiological data in audiovisual speech recognition tasks. Human audiovisual speech recognition follows principles of Bayes-optimal integration, with each modality weighted by its reliability (Author et al., 2009). To determine whether ML models of audiovisual speech recognition follow similar Bayesian principles, we applied the technique used to estimate modality weights: assessing recognition of stimuli containing a mismatch between modalities (Author \& Author, 2
ePoster · Neuroscience