A DEEP ENCODER FOR SENSORY RESTORATION BY NEUROMIMETIC CORTICAL STIMULATION

The primary approach to restoring profound deafness is cochlear implants, which stimulate the inner ear. However, resolution constraints limit cochlear implants to about ten independent stimulation channels leading to imprecise perception. An alternative is targeting the auditory cortex—the final stage of the auditory system—where electrical stimulation also produces perception and which is large enough for hundreds to thousands of electrodes to be implanted. Yet, this requires an efficient coding scheme. Here, we introduce Braincodec, a deep network that compresses sound information with minimal loss while aligning with the auditory cortex’s neural code. Braincodec uses an autoencoder to optimize compression into a latent space corresponding to a desired number of electrodes and to reconstruct sound, assessing restoration quality. By co-optimizing the latent space to enable a classifier to identify human-defined sound categories from millions of samples, Braincodec also ensures a realistic neural code matching human brain representations and auditory cortex response dynamics. With just a few hundred electrodes, Braincodec-reconstructed sounds reach normal hearing quality, far surpassing cochlear implants. In mice with auditory cortical implants, Braincodec also delivers more precise perception than cochlea-like encoders. Thus, Braincodec enables new high-efficiency auditory restoration strategies via direct brain stimulation, offering a computational framework adaptable to any sensory modality.