DEEP LEARNING OF MESOSCALE CORTICAL DYNAMICS ENABLES INTUITIVE TWO-DIMENSIONAL CONTROL OF A FORELIMB NEUROPROSTHESIS IN MICE

Recent studies suggest that mesoscale cortical dynamics encode movement-related information such as direction and speed, making them promising signals for neuroprosthetic control. We developed a deep learning approach to decode forelimb movements from mesoscale cortical activity in mice and demonstrated real-time control of a forelimb neuroprosthesis with two-dimensional movement capability. Using wide-field GCaMP6f imaging in head-fixed mice, we simultaneously recorded cortical activity over primary motor and somatosensory cortices and 3D kinematics of body landmarks during naturalistic forelimb movements on a sliding treadmill. We built a dataset linking 450 ms windows of cortical activity to movement categories (forward/backward, medial/lateral, and still). Two 3D convolutional neural networks, one for each spatial dimension, were trained on this dataset to classify movements along the anteroposterior axis (forward/still/backward) and mediolateral axis (lateral/still/medial), enabling real-time decoding at 100 Hz with sub-10 ms latency. These decoders were integrated into a system controlling a custom-built mouse forelimb prosthesis. Mice were trained to use the prosthesis in a water-reaching task, where decoded cortical activity patterns directly translated into two-dimensional prosthetic movements. Over an 8-day training period, mice significantly improved their performance, increasing both the time spent with the prosthesis at the mouth location and the number of successful water retrievals, demonstrating that mice learned to control the neuroprosthesis. These results establish the feasibility of decoding naturally occurring wide-field cortical patterns for intuitive neuroprosthetic control. This approach offers a less invasive and potentially more stable alternative to traditional electrophysiological recording methods, with promising translational potential through emerging microECoG technologies.