EYES IN THE LOOP: OCULAR AND NEURAL INTERPLAY IN AUDITORY SPEECH PROCESSING

“Ocular Speech Tracking” is a phenomenon describing the alignment between a heard audio envelope and a listener’s eye movements. Earlier experiments conducted by our group provided evidence of this mechanism and revealed links to the degree of speech understanding. In the present study, we aimed to both replicate earlier findings in an information theoretic framework and develop a deeper understanding of the relationship between ocular speech tracking and the longer-established neural speech tracking; the latter referring to an alignment of neural activity with heard speech. A dataset containing MEG and Eye Tracking data of 114 subjects between the ages of 18 and 79 was used. Our first step was to investigate Ocular Speech Tracking through temporal mutual information; here used to measure the portion of the envelope present in the ocular signal over time. We found tracking in all individuals; moreover, the bimodal age distribution of the dataset allowed us to compare age groups. In a second step we used Partial Information Decomposition to identify any unique or synergistic information about the speech envelope by neural and ocular activity. Synergy here describes the information contributed only by the interplay of neural and ocular data. Our findings support a facilitative role of the oculomotor system in speech processing beyond visually-guided functions like lip reading and attention. Future research on this process may yield new insights into speech processing and guide the development of targeted rehabilitation strategies.