ePoster, COSYNE 2023
Neuroformer: A Transformer Framework for Multimodal Neural Data Analysis
Antonis Antoniades, Yiyi Yu and Spencer LaVere Smith
University of California Santa Barbara; Computer Science, Dynamical Neuroscience; University of California Santa Barbara
- Montreal, Canada
- Neuroscience, Computational Neuroscience and
Abstract
Systems neuroscience experiments generate multimodal data streams, including neuronal activity, stimuli, movement, and behavior events. In analysis, it can be beneficial to avoid hypothesis-driven models and instead take a data-driven approach. For this, we need a general purpose framework to handle the diverse data, and identify relationships across space, time, and modalities in a computationally efficient manner. In this work, we introduce such a framework. Our model, called the Neuroformer, can simulate neuronal populations, and jointly process an arbitrary number of additional modalities (stimuli, movement, etc.). To validate its performance, we analyzed a simulated neuronal circuit and showed that the inferred network was accurate in structure and directionality of connections. To demonstrate its utility, we used the Neuroformer to analyze large scale, multimodal data from a 2-photon calcium imaging experiment and inferred relationships between visual stimuli and neuronal activity. The Neuroformer is a flexible data-driven tool for modeling and analysis in systems neuroscience. We provide an open-source codebase for replicating and extending this work.
Topics
- Systems neuroscience, search World Wide
- multimodal data, search World Wide
- neuronal activity, search World Wide
- stimuli, search World Wide
- movement, search World Wide
- behavior events, search World Wide
- Neuroformer, search World Wide
- neuronal populations, search World Wide
- 2-photon calcium imaging, search World Wide
Details
- Session
- Poster Session I