Neuroformer: A Transformer Framework for Multimodal Neural Data Analysis
Antonis Antoniades, Yiyi Yu, Spencer LaVere Smith · COSYNE 2023
Systems neuroscience experiments generate multimodal data streams, including neuronal activity, stimuli, movement, and behavior events. In analysis, it can be beneficial to avoid hypothesis-driven models and instead take a data-driven approach. For this, we need a general purpose framework to handle the diverse data, and identify relationships across space, time, and modalities in a computationally efficient manner. In this work, we introduce such a framework. Our model, called the Neuroformer, can simulate neuronal populations, and jointly process an arbitrary number of additional modalities
Fri, Mar 10, 2023