Trends in NeuroAI is a reading group hosted by the MedARC Neuroimaging & AI lab (https://medarc.ai/fmri). This will be an informal journal club presentation, we do not have an author of the paper joining us. Title: Brain decoding: toward real-time reconstruction of visual perception Abstract: In the past five years, the use of generative and foundational AI systems has greatly improved the decoding of brain activity. Visual perception, in particular, can now be decoded from functional Magnetic Resonance Imaging (fMRI) with remarkable fidelity. This neuroimaging technique, however, suffers from a limited temporal resolution (≈0.5 Hz) and thus fundamentally constrains its real-time usage. Here, we propose an alternative approach based on magnetoencephalography (MEG), a neuroimaging device capable of measuring brain activity with high temporal resolution (≈5,000 Hz). For this, we develop an MEG decoding model trained with both contrastive and regression objectives and consisting of three modules: i) pretrained embeddings obtained from the image, ii) an MEG module trained end-to-end and iii) a pretrained image generator. Our results are threefold: Firstly, our MEG decoder shows a 7X improvement of image-retrieval over classic linear decoders. Second, late brain responses to images are best decoded with DINOv2, a recent foundational image model. Third, image retrievals and generations both suggest that MEG signals primarily contain high-level visual features, whereas the same approach applied to 7T fMRI also recovers low-level features. Overall, these results provide an important step towards the decoding - in real time - of the visual processes continuously unfolding within the human brain. Speaker: Dr. Paul Scotti (Stability AI, MedARC) Paper link: https://arxiv.org/abs/2310.19812

Advanced noninvasive neuroimaging methods provide valuable information on the brain function, but they have obvious pros and cons in terms of temporal and spatial resolution. Functional magnetic resonance imaging (fMRI) using blood-oxygenation-level-dependent (BOLD) effect provides good spatial resolution in the order of millimeters, but has a poor temporal resolution in the order of seconds due to slow hemodynamic responses to neuronal activation, providing indirect information on neuronal activity. In contrast, electroencephalography (EEG) and magnetoencephalography (MEG) provide excellent temporal resolution in the millisecond range, but spatial information is limited to centimeter scales. Therefore, there has been a longstanding demand for noninvasive brain imaging methods capable of detecting neuronal activity at both high temporal and spatial resolution. In this talk, I will introduce a novel approach that enables Direct Imaging of Neuronal Activity (DIANA) using MRI that can dynamically image neuronal spiking activity in milliseconds precision, achieved by data acquisition scheme of rapid 2D line scan synchronized with periodically applied functional stimuli. DIANA was demonstrated through in vivo mouse brain imaging on a 9.4T animal scanner during electrical whisker-pad stimulation. DIANA with milliseconds temporal resolution had high correlations with neuronal spike activities, which could also be applied in capturing the sequential propagation of neuronal activity along the thalamocortical pathway of brain networks. In terms of the contrast mechanism, DIANA was almost unaffected by hemodynamic responses, but was subject to changes in membrane potential-associated tissue relaxation times such as T2 relaxation time. DIANA is expected to break new ground in brain science by providing an in-depth understanding of the hierarchical functional organization of the brain, including the spatiotemporal dynamics of neural networks.

Here, we present a protocol for using a versatile treadmill system to measure locomotion and neural activity at high temporal resolution in head-fixed mice. We first describe the assembly of the treadmill system. We then detail surgical implantation of the headplate on the mouse skull, followed by habituation of mice to locomotion on the treadmill system. The system is compact, movable, and simple to synchronize with other data streams, making it ideal for monitoring brain activity in diverse behavioral frameworks. https://dx.doi.org/10.1016/j.xpro.2022.101701

The dream of a systems neuroscientist is to be able to unravel neural mechanisms that give rise to behavior. It is increasingly appreciated that behavior involves the concerted distributed activity of multiple brain regions so the focus on single or few brain areas might hinder our understanding. There have been quite a few technological advancements in this domain. Functional ultrasound imaging (fUSi) is an emerging technique that allows us to measure neural activity from medial frontal regions down to subcortical structures up to a depth of 20 mm. It is a method for imaging transient changes in cerebral blood volume (CBV), which are proportional to neural activity changes. It has excellent spatial resolution (~100 μm X 100 μm X 400 μm); its temporal resolution can go down to 100 milliseconds. In this talk, I will present its use in two model systems: marmoset monkeys and rats. In marmoset monkeys, we used it to delineate a social – vocal network involved in vocal communication while in rats, we used it to gain insights into brain wide networks involved in evidence accumulation based decision making. fUSi has the potential to provide an unprecedented access to brain wide dynamics in freely moving animals performing complex behavioral tasks.

Electro- and magneto-encephalography (EEG/MEG) are the leading methods to non-invasively record human neural dynamics with millisecond temporal resolution. However, it can be extremely difficult to infer the underlying cellular and circuit level origins of these macro-scale signals without simultaneous invasive recordings. This limits the translation of E/MEG into novel principles of information processing, or into new treatment modalities for neural pathologies. To address this need, we developed the Human Neocortical Neurosolver (HNN: https://hnn.brown/edu ), a new user-friendly neural modeling tool designed to help researchers and clinicians interpret human imaging data. A unique feature of HNN’s model is that it accounts for the biophysics generating the primary electric currents underlying such data, so simulation results are directly comparable to source localized data. HNN is being constructed with workflows of use to study some of the most commonly measured E/MEG signals including event related potentials, and low frequency brain rhythms. In this talk, I will give an overview of this new tool and describe an application to study the origin and meaning of 15-29Hz beta frequency oscillations, known to be important for sensory and motor function. Our data showed that in primary somatosensory cortex these oscillations emerge as transient high power ‘events’. Functionally relevant differences in averaged power reflected a difference in the number of high-power beta events per trial (“rate”), as opposed to changes in event amplitude or duration. These findings were consistent across detection and attention tasks in human MEG, and in local field potentials from mice performing a detection task. HNN modeling led to a new theory on the circuit origin of such beta events and suggested beta causally impacts perception through layer specific recruitment of cortical inhibition, with support from invasive recordings in animal models and high-resolution MEG in humans. In total, HNN provides an unpresented biophysically principled tool to link mechanism to meaning of human E/MEG signals.

PiVR is a system that allows experimenters to immerse small animals into virtual realities. The system tracks the position of the animal and presents light stimulation according to predefined rules, thus creating a virtual landscape in which the animal can behave. By using optogenetics, we have used PiVR to present fruit fly larvae with virtual olfactory realities, adult fruit flies with a virtual gustatory reality and zebrafish larvae with a virtual light gradient. PiVR operates at high temporal resolution (70Hz) with low latencies (<30 milliseconds) while being affordable (<US$500) and easy to build (<6 hours). Through extensive documentation (www.PiVR.org), this tool was designed to be accessible to a wide public, from high school students to professional researchers studying systems neuroscience in academia.

Powerful new ways to image the internal structures and complex dynamics of cells are revolutionizing cell biology and bio-medical research. In this talk, I will focus on how emerging fluorescent technologies are increasing spatio-temporal resolution dramatically, permitting simultaneous multispectral imaging of multiple cellular components. In addition, results will be discussed from whole cell milling using Focused Ion Beam Electron Microscopy (FIB-SEM), which reconstructs the entire cell volume at 4 voxel resolution. Using these tools, it is now possible to begin constructing an “organelle interactome”, describing the interrelationships of different cellular organelles as they carry out critical functions. The same tools are also revealing new properties of organelles and their trafficking pathways, and how disruptions of their normal functions due to genetic mutations may contribute to important diseases.

Genetic targeting of neuronal cells with activity reporters (calcium or voltage indicators) has initiated the paradigmatic transition whereby photons have replaced electrons for reading large-scale brain activities at cellular resolution. This has alleviated the limitations of single cell or extracellular electrophysiological probing, which only give access to the activity of at best a few neurons simultaneously and to population activity of unresolved cellular origin, respectively. In parallel, optogenetics has demonstrated that targeting neuronal cells with photosensitive microbial opsins, enables the transduction of photons into electrical currents of opposite polarities thus writing, through activation or inhibition, neuronal signals in a non-invasive way. These progresses have in turn stimulated the development of sophisticated optical methods to increase spatial and temporal resolution, light penetration depth and imaging volume. Today, nonlinear microscopy, combined with spatio-temporal wave front shaping, endoscopic probes engineering or multi scan heads design, enable in vivo in depth, simultaneous recording of thousands of cells in mm 3 volumes at single-spike precision and single-cell resolution. Joint progress in opsin engineering, wave front shaping and laser development have provided the methodology, that we named circuits optogenetics, to control single or multiple target activity independently in space and time with single- neuron and single-spike precision, at large depths. Here, we will review the most significant breakthroughs of the past years, which enable reading and writing neuronal activity at the relevant spatiotemporal scale for brain circuits manipulation, with particular emphasis on the most recent advances in circuit optogenetics.

Powerful new ways to image the internal structures and complex dynamics of cells are revolutionizing cell biology and bio-medical research. In this talk, I will focus on how emerging fluorescent technologies are increasing spatio-temporal resolution dramatically, permitting simultaneous multispectral imaging of multiple cellular components. In addition, results will be discussed from whole cell milling using Focused Ion Beam Electron Microscopy (FIB-SEM), which reconstructs the entire cell volume at 4 voxel resolution. Using these tools, it is now possible to begin constructing an “organelle interactome”, describing the interrelationships of different cellular organelles as they carry out critical functions. The same tools are also revealing new properties of organelles and their trafficking pathways, and how disruptions of their normal functions due to genetic mutations may contribute to important diseases.