Simultaneous brain control of two cursors enabled by online-robust neural networks

Advancements in machine learning and artificial intelligence have given rise to promising neural network-based models of the complex nonlinear relationships between movement intention and neural activity. In particular, recurrent neural networks (RNNs) are one commonly used algorithm for brain-computer interface (BCI) control, due to their ability to learn temporal dependencies within the training data. However, most BCI training data contains highly stereotyped temporal and behavioral structure to which RNNs can overfit, causing poor performance in online settings when behavior and temporal characteristics differ from the offline task used for RNN calibration.

To solve this problem, we developed a data augmentation technique that dilates/compresses small snippets of training data in time and randomizes the order of these snippets, helping RNN decoders trained on temporally stereotyped open-loop data generalize to the closed-loop setting. We demonstrate this technique in the context of simultaneous two-cursor control via RNN decoders, in a person with paralysis enrolled in the BrainGate2 pilot clinical trial. We show that decoders trained with our data augmentation technique outperform linear decoders and non-augmented RNNs, even though augmented RNNs perform worse on the training data. This suggests that prior work demonstrating very high performance on stereotyped data may not be representative of how performance will translate to online settings.

Our approach of hardening neural networks to online settings makes it possible to achieve high-performance control of multiple effectors at the same time, since multi-effector movement is coded in a nonlinear way that benefits from nonlinear decoding approaches. Insights gained developing neural decoders to accurately restore this multi-effector motion may help future BCIs to restore physical whole-body motion using exoskeletons or muscle stimulation to carry out acts of daily living.