Sensorimotor
sensorimotor
sensorimotor control, mouvement, touch, EEG
Traditionally, touch is associated with exteroception and is rarely considered a relevant sensory cue for controlling movements in space, unlike vision. We developed a technique to isolate and measure tactile involvement in controlling sliding finger movements over a surface. Young adults traced a 2D shape with their index finger under direct or mirror-reversed visual feedback to create a conflict between visual and somatosensory inputs. In this context, increased reliance on somatosensory input compromises movement accuracy. Based on the hypothesis that tactile cues contribute to guiding hand movements when in contact with a surface, we predicted poorer performance when the participants traced with their bare finger compared to when their tactile sensation was dampened by a smooth, rigid finger splint. The results supported this prediction. EEG source analyses revealed smaller current in the source-localized somatosensory cortex during sensory conflict when the finger directly touched the surface. This finding supports the hypothesis that, in response to mirror-reversed visual feedback, the central nervous system selectively gated task-irrelevant somatosensory inputs, thereby mitigating, though not entirely resolving, the visuo-somatosensory conflict. Together, our results emphasize touch’s involvement in movement control over a surface, challenging the notion that vision predominantly governs goal-directed hand or finger movements.
Prof. KongFatt Wong-Lin
Postdoctoral Research Associate Position in Computational Neuroscience (Computational Modelling of Decision Making) Applications are invited for an externally funded Postdoctoral Research Associate position at the Intelligent Systems Research Centre (ISRC) in Ulster University, UK. The successful candidate will develop and apply computational modelling, and theoretical and analytical techniques to understand brain and behavioural data across primate species, and to apply biologically based neural network modelling to elucidate mechanisms underlying perceptual decision-making. The duration of the position is 24 months, from January 2024 till end of 2025. The personnel will be based at the ISRC in Ulster University, working with Prof. KongFatt Wong-Lin and his team, while collaborating closely with international collaborators in the USA and the Republic of Ireland, namely, Prof. Michael Shadlen at Columbia University (USA), Prof. Stephan Bickel at Northwell-Hofstra School of Medicine (USA), Prof. Redmond O'Connell at Trinity College Dublin (Ireland), Prof. Simon Kelly at University College Dublin (Ireland), and Prof. S. Shushruth at University of Pittsburgh (USA). The ISRC is dedicated to developing a bio-inspired computational basis for AI to power future cognitive technologies. This is achieved through understanding how the brain works at multiple levels, from cells to cognition and apply that understanding to create models and technologies that solve complex issues that face people and society. All applicants should hold a degree in in Computational Neuroscience, Computational Biology, Neuroscience, Computing, Engineering, Mathematics, Data Science, Physical Sciences, Biology, or a cognate area. Apply online: https://my.corehr.com/pls/coreportal_ulsp/erq_jobspec_version_4.display_form?p_company=1&p_internal_external=E&p_display_in_irish=N&p_applicant_no=&p_recruitment_id=023762&p_process_type=&p_form_profile_detail=&p_display_apply_ind=Y&p_refresh_search=Y Closing date for receipt of completed applications: 8th November 2023. Job Ref: 023762. For any informal enquiries regarding this position, please contact KongFatt Wong-Lin; email: k.wong-lin@ulster.ac.uk ; website: https://www.ulster.ac.uk/staff/k-wong-lin
Immune and metabolic regulation of sensorimotor physiology and repair
Visual mechanisms for flexible behavior
Perhaps the most impressive aspect of the way the brain enables us to act on the sensory world is its flexibility. We can make a general inference about many sensory features (rating the ripeness of mangoes or avocados) and map a single stimulus onto many choices (slicing or blending mangoes). These can be thought of as flexibly mapping many (features) to one (inference) and one (feature) to many (choices) sensory inputs to actions. Both theoretical and experimental investigations of this sort of flexible sensorimotor mapping tend to treat sensory areas as relatively static. Models typically instantiate flexibility through changing interactions (or weights) between units that encode sensory features and those that plan actions. Experimental investigations often focus on association areas involved in decision-making that show pronounced modulations by cognitive processes. I will present evidence that the flexible formatting of visual information in visual cortex can support both generalized inference and choice mapping. Our results suggest that visual cortex mediates many forms of cognitive flexibility that have traditionally been ascribed to other areas or mechanisms. Further, we find that a primary difference between visual and putative decision areas is not what information they encode, but how that information is formatted in the responses of neural populations, which is related to difference in the impact of causally manipulating different areas on behavior. This scenario allows for flexibility in the mapping between stimuli and behavior while maintaining stability in the information encoded in each area and in the mappings between groups of neurons.
Sensory Consequences of Visual Actions
We use rapid eye, head, and body movements to extract information from a new part of the visual scene upon each new gaze fixation. But the consequences of such visual actions go beyond their intended sensory outcomes. On the one hand, intrinsic consequences accompany movement preparation as covert internal processes (e.g., predictive changes in the deployment of visual attention). On the other hand, visual actions have incidental consequences, side effects of moving the sensory surface to its intended goal (e.g., global motion of the retinal image during saccades). In this talk, I will present studies in which we investigated intrinsic and incidental sensory consequences of visual actions and their sensorimotor functions. Our results provide insights into continuously interacting top-down and bottom-up sensory processes, and they reify the necessity to study perception in connection to motor behavior that shapes its fundamental processes.
Event-related frequency adjustment (ERFA): A methodology for investigating neural entrainment
Neural entrainment has become a phenomenon of exceptional interest to neuroscience, given its involvement in rhythm perception, production, and overt synchronized behavior. Yet, traditional methods fail to quantify neural entrainment due to a misalignment with its fundamental definition (e.g., see Novembre and Iannetti, 2018; Rajandran and Schupp, 2019). The definition of entrainment assumes that endogenous oscillatory brain activity undergoes dynamic frequency adjustments to synchronize with environmental rhythms (Lakatos et al., 2019). Following this definition, we recently developed a method sensitive to this process. Our aim was to isolate from the electroencephalographic (EEG) signal an oscillatory component that is attuned to the frequency of a rhythmic stimulation, hypothesizing that the oscillation would adaptively speed up and slow down to achieve stable synchronization over time. To induce and measure these adaptive changes in a controlled fashion, we developed the event-related frequency adjustment (ERFA) paradigm (Rosso et al., 2023). A total of twenty healthy participants took part in our study. They were instructed to tap their finger synchronously with an isochronous auditory metronome, which was unpredictably perturbed by phase-shifts and tempo-changes in both positive and negative directions across different experimental conditions. EEG was recorded during the task, and ERFA responses were quantified as changes in instantaneous frequency of the entrained component. Our results indicate that ERFAs track the stimulus dynamics in accordance with the perturbation type and direction, preferentially for a sensorimotor component. The clear and consistent patterns confirm that our method is sensitive to the process of frequency adjustment that defines neural entrainment. In this Virtual Journal Club, the discussion of our findings will be complemented by methodological insights beneficial to researchers in the fields of rhythm perception and production, as well as timing in general. We discuss the dos and don’ts of using instantaneous frequency to quantify oscillatory dynamics, the advantages of adopting a multivariate approach to source separation, the robustness against the confounder of responses evoked by periodic stimulation, and provide an overview of domains and concrete examples where the methodological framework can be applied.
Predictive processing in older adults: How does it shape perception and sensorimotor control?
Microstructural Features of the Human Sensorimotor Cortex in Health and Disease
Effect of Different Influences on Temporal Error Monitoring
Metacognition has long been defined as “cognition about cognition”. One of its aspects is the error monitoring ability, which includes being aware of one’s own errors without external feedback. This ability is mostly investigated in two-alternative forced choice tasks, where the performance has all or none nature in terms of accuracy. The previous literature documents the effect of different influences on the error monitoring ability, such as working memory, feedback and sensorimotor involvement. However, these demonstrations fall short of generalizing to the real life scenarios where the errors often have a magnitude and a direction. To bridge this gap, recent studies showed that humans could keep track of the magnitude and the direction of their errors in temporal, spatial and numerical domains in two metrics: confidence and short-long/few-more judgements. This talk will cover how the documented effects that are obtained in the two alternative forced choices tasks apply to the temporal error monitoring ability. Finally, how magnitude and direction monitoring (i.e., confidence and short-long judgements) can be differentiated as the two indices of temporal error monitoring ability will be discussed.
Somatotopic reorganization of the macaque sensorimotor cortex after accidental arm amputation
Transcriptional controls over projection neuron fate diversity
The cerebral cortex is the most evolved structure of the brain and the site for higher cognitive functions. It consists of 6 layers, each composed of specific types of neurons. Interconnectivity between cortical areas is critical for sensory integration and sensorimotor transformation. Inter-areal cortical projection neurons are located in all cortical layers and form a heterogeneous population, which send their axon across cortical areas, both within and across hemispheres. How this diversity emerges during development remains largely unknown. Here, we address this question by linking the connectome and transcriptome of developing cortical projection neurons and show distinct maturation paces in neurons with distinct projections, which correlates with the sequential development of sensory and motor functions during postnatal period.
Visualization and manipulation of our perception and imagery by BCI
We have been developing Brain-Computer Interface (BCI) using electrocorticography (ECoG) [1] , which is recorded by electrodes implanted on brain surface, and magnetoencephalography (MEG) [2] , which records the cortical activities non-invasively, for the clinical applications. The invasive BCI using ECoG has been applied for severely paralyzed patient to restore the communication and motor function. The non-invasive BCI using MEG has been applied as a neurofeedback tool to modulate some pathological neural activities to treat some neuropsychiatric disorders. Although these techniques have been developed for clinical application, BCI is also an important tool to investigate neural function. For example, motor BCI records some neural activities in a part of the motor cortex to generate some movements of external devices. Although our motor system consists of complex system including motor cortex, basal ganglia, cerebellum, spinal cord and muscles, the BCI affords us to simplify the motor system with exactly known inputs, outputs and the relation of them. We can investigate the motor system by manipulating the parameters in BCI system. Recently, we are developing some BCIs to visualize and manipulate our perception and mental imagery. Although these BCI has been developed for clinical application, the BCI will be useful to understand our neural system to generate the perception and imagery. In this talk, I will introduce our study of phantom limb pain [3] , that is controlled by MEG-BCI, and the development of a communication BCI using ECoG [4] , that enable the subject to visualize the contents of their mental imagery. And I would like to discuss how much we can control our cortical activities that represent our perception and mental imagery. These examples demonstrate that BCI is a promising tool to visualize and manipulate the perception and imagery and to understand our consciousness. References 1. Yanagisawa, T., Hirata, M., Saitoh, Y., Kishima, H., Matsushita, K., Goto, T., Fukuma, R., Yokoi, H., Kamitani, Y., and Yoshimine, T. (2012). Electrocorticographic control of a prosthetic arm in paralyzed patients. AnnNeurol 71, 353-361. 2. Yanagisawa, T., Fukuma, R., Seymour, B., Hosomi, K., Kishima, H., Shimizu, T., Yokoi, H., Hirata, M., Yoshimine, T., Kamitani, Y., et al. (2016). Induced sensorimotor brain plasticity controls pain in phantom limb patients. Nature communications 7, 13209. 3. Yanagisawa, T., Fukuma, R., Seymour, B., Tanaka, M., Hosomi, K., Yamashita, O., Kishima, H., Kamitani, Y., and Saitoh, Y. (2020). BCI training to move a virtual hand reduces phantom limb pain: A randomized crossover trial. Neurology 95, e417-e426. 4. Ryohei Fukuma, Takufumi Yanagisawa, Shinji Nishimoto, Hidenori Sugano, Kentaro Tamura, Shota Yamamoto, Yasushi Iimura, Yuya Fujita, Satoru Oshino, Naoki Tani, Naoko Koide-Majima, Yukiyasu Kamitani, Haruhiko Kishima (2022). Voluntary control of semantic neural representations by imagery with conflicting visual stimulation. arXiv arXiv:2112.01223.
Measuring and modeling behavior to decode sensorimotor control
Mice identify subgoals locations through an action-driven mapping process
Mammals instinctively explore and form mental maps of their spatial environments. Models of cognitive mapping in neuroscience mostly depict map-learning as a process of random or biased diffusion. In practice, however, animals explore spaces using structured, purposeful, sensory-guided actions. We have used threat-evoked escape behavior in mice to probe the relationship between ethological exploratory behavior and abstract spatial cognition. First, we show that in arenas with obstacles and a shelter, mice spontaneously learn efficient multi-step escape routes by memorizing allocentric subgoal locations. Using closed-loop neural manipulations to interrupt running movements during exploration, we next found that blocking runs targeting an obstacle edge abolished subgoal learning. We conclude that mice use an action-driven learning process to identify subgoals, and these subgoals are then integrated into an allocentric map-like representation. We suggest a conceptual framework for spatial learning that is compatible with the successor representation from reinforcement learning and sensorimotor enactivism from cognitive science.
Brain circuit dynamics in Action and Sleep
Our group focuses on brain computation, physiology and evolution, with a particular focus on network dynamics, sleep (evolution and mechanistic underpinnings), cortical computation (through the study of ancestral cortices), and sensorimotor processing. This talk will describe our recent results on the remarkable camouflage behavior of cuttlefish (action) and on brain activity in REM and NonREM in lizards (sleep). Both topics will focus on aspects of circuit dynamics.
NMC4 Short Talk: Brain-inspired spiking neural network controller for a neurorobotic whisker system
It is common for animals to use self-generated movements to actively sense the surrounding environment. For instance, rodents rhythmically move their whiskers to explore the space close to their body. The mouse whisker system has become a standard model to study active sensing and sensorimotor integration through feedback loops. In this work, we developed a bioinspired spiking neural network model of the sensorimotor peripheral whisker system, modelling trigeminal ganglion, trigeminal nuclei, facial nuclei, and central pattern generator neuronal populations. This network was embedded in a virtual mouse robot, exploiting the Neurorobotics Platform, a simulation platform offering a virtual environment to develop and test robots driven by brain-inspired controllers. Eventually, the peripheral whisker system was properly connected to an adaptive cerebellar network controller. The whole system was able to drive active whisking with learning capability, matching neural correlates of behaviour experimentally recorded in mice.
Brain circuit dynamics in Action and Sleep
Our group focuses on brain computation, physiology and evolution, with a particular focus on network dynamics, sleep (evolution and mechanistic underpinnings), cortical computation (through the study of ancestral cortices), and sensorimotor processing. This talk will describe our recent results on the remarkable camouflage behavior of cuttlefish (action) and on brain activity in REM and NonREM in lizards (sleep). Both topics will focus on aspects of circuit dynamics.
NMC4 Short Talk: What can deep reinforcement learning tell us about human motor learning and vice-versa ?
In the deep reinforcement learning (RL) community, motor control problems are usually approached from a reward-based learning perspective. However, humans are often believed to learn motor control through directed error-based learning. Within this learning setting, the control system is assumed to have access to exact error signals and their gradients with respect to the control signal. This is unlike reward-based learning, in which errors are assumed to be unsigned, encoding relative successes and failures. Here, we try to understand the relation between these two approaches, reward- and error- based learning, and ballistic arm reaches. To do so, we test canonical (deep) RL algorithms on a well-known sensorimotor perturbation in neuroscience: mirror-reversal of visual feedback during arm reaching. This test leads us to propose a potentially novel RL algorithm, denoted as model-based deterministic policy gradient (MB-DPG). This RL algorithm draws inspiration from error-based learning to qualitatively reproduce human reaching performance under mirror-reversal. Next, we show MB-DPG outperforms the other canonical (deep) RL algorithms on a single- and a multi- target ballistic reaching task, based on a biomechanical model of the human arm. Finally, we propose MB-DPG may provide an efficient computational framework to help explain error-based learning in neuroscience.
NMC4 Short Talk: Directly interfacing brain and deep networks exposes non-hierarchical visual processing
A recent approach to understanding the mammalian visual system is to show correspondence between the sequential stages of processing in the ventral stream with layers in a deep convolutional neural network (DCNN), providing evidence that visual information is processed hierarchically, with successive stages containing ever higher-level information. However, correspondence is usually defined as shared variance between brain region and model layer. We propose that task-relevant variance is a stricter test: If a DCNN layer corresponds to a brain region, then substituting the model’s activity with brain activity should successfully drive the model’s object recognition decision. Using this approach on three datasets (human fMRI and macaque neuron firing rates) we found that in contrast to the hierarchical view, all ventral stream regions corresponded best to later model layers. That is, all regions contain high-level information about object category. We hypothesised that this is due to recurrent connections propagating high-level visual information from later regions back to early regions, in contrast to the exclusively feed-forward connectivity of DCNNs. Using task-relevant correspondence with a late DCNN layer akin to a tracer, we used Granger causal modelling to show late-DCNN correspondence in IT drives correspondence in V4. Our analysis suggests, effectively, that no ventral stream region can be appropriately characterised as ‘early’ beyond 70ms after stimulus presentation, challenging hierarchical models. More broadly, we ask what it means for a model component and brain region to correspond: beyond quantifying shared variance, we must consider the functional role in the computation. We also demonstrate that using a DCNN to decode high-level conceptual information from ventral stream produces a general mapping from brain to model activation space, which generalises to novel classes held-out from training data. This suggests future possibilities for brain-machine interface with high-level conceptual information, beyond current designs that interface with the sensorimotor periphery.
Neural dynamics of probabilistic information processing in humans and recurrent neural networks
In nature, sensory inputs are often highly structured, and statistical regularities of these signals can be extracted to form expectation about future sensorimotor associations, thereby optimizing behavior. One of the fundamental questions in neuroscience concerns the neural computations that underlie these probabilistic sensorimotor processing. Through a recurrent neural network (RNN) model and human psychophysics and electroencephalography (EEG), the present study investigates circuit mechanisms for processing probabilistic structures of sensory signals to guide behavior. We first constructed and trained a biophysically constrained RNN model to perform a series of probabilistic decision-making tasks similar to paradigms designed for humans. Specifically, the training environment was probabilistic such that one stimulus was more probable than the others. We show that both humans and the RNN model successfully extract information about stimulus probability and integrate this knowledge into their decisions and task strategy in a new environment. Specifically, performance of both humans and the RNN model varied with the degree to which the stimulus probability of the new environment matched the formed expectation. In both cases, this expectation effect was more prominent when the strength of sensory evidence was low, suggesting that like humans, our RNNs placed more emphasis on prior expectation (top-down signals) when the available sensory information (bottom-up signals) was limited, thereby optimizing task performance. Finally, by dissecting the trained RNN model, we demonstrate how competitive inhibition and recurrent excitation form the basis for neural circuitry optimized to perform probabilistic information processing.
Striatal circuits underlying sensorimotor functions
Interpreting the Mechanisms and Meaning of Sensorimotor Beta Rhythms with the Human Neocortical Neurosolver (HNN) Neural Modeling Software
Electro- and magneto-encephalography (EEG/MEG) are the leading methods to non-invasively record human neural dynamics with millisecond temporal resolution. However, it can be extremely difficult to infer the underlying cellular and circuit level origins of these macro-scale signals without simultaneous invasive recordings. This limits the translation of E/MEG into novel principles of information processing, or into new treatment modalities for neural pathologies. To address this need, we developed the Human Neocortical Neurosolver (HNN: https://hnn.brown/edu ), a new user-friendly neural modeling tool designed to help researchers and clinicians interpret human imaging data. A unique feature of HNN’s model is that it accounts for the biophysics generating the primary electric currents underlying such data, so simulation results are directly comparable to source localized data. HNN is being constructed with workflows of use to study some of the most commonly measured E/MEG signals including event related potentials, and low frequency brain rhythms. In this talk, I will give an overview of this new tool and describe an application to study the origin and meaning of 15-29Hz beta frequency oscillations, known to be important for sensory and motor function. Our data showed that in primary somatosensory cortex these oscillations emerge as transient high power ‘events’. Functionally relevant differences in averaged power reflected a difference in the number of high-power beta events per trial (“rate”), as opposed to changes in event amplitude or duration. These findings were consistent across detection and attention tasks in human MEG, and in local field potentials from mice performing a detection task. HNN modeling led to a new theory on the circuit origin of such beta events and suggested beta causally impacts perception through layer specific recruitment of cortical inhibition, with support from invasive recordings in animal models and high-resolution MEG in humans. In total, HNN provides an unpresented biophysically principled tool to link mechanism to meaning of human E/MEG signals.
PiVR: An affordable and versatile closed-loop platform to study unrestrained sensorimotor behavior
PiVR is a system that allows experimenters to immerse small animals into virtual realities. The system tracks the position of the animal and presents light stimulation according to predefined rules, thus creating a virtual landscape in which the animal can behave. By using optogenetics, we have used PiVR to present fruit fly larvae with virtual olfactory realities, adult fruit flies with a virtual gustatory reality and zebrafish larvae with a virtual light gradient. PiVR operates at high temporal resolution (70Hz) with low latencies (<30 milliseconds) while being affordable (<US$500) and easy to build (<6 hours). Through extensive documentation (www.PiVR.org), this tool was designed to be accessible to a wide public, from high school students to professional researchers studying systems neuroscience in academia.
Flexible codes and loci of visual working memory
Neural correlates of visual working memory have been found in early visual, parietal, and prefrontal regions. These findings have spurred fruitful debate over how and where in the brain memories might be represented. Here, I will present data from multiple experiments to demonstrate how a focus on behavioral requirements can unveil a more comprehensive understanding of the visual working memory system. Specifically, items in working memory must be maintained in a highly robust manner, resilient to interference. At the same time, storage mechanisms must preserve a high degree of flexibility in case of changing behavioral goals. Several examples will be explored in which visual memory representations are shown to undergo transformations, and even shift their cortical locus alongside their coding format based on specifics of the task.
Synchrony and Synaptic Signaling in Cerebellar Circuits
The cerebellum permits a wide range of behaviors that involve sensorimotor integration. We have been investigating how specific cellular and synaptic specializations of cerebellar neurons measured in vitro, give rise to circuit activity in vivo. We have investigated these issues by studying Purkinje neurons as well as the large neurons of the mouse cerebellar nuclei, which form the major excitatory premotor projection from the cerebellum. Large CbN cells have ion channels that favor spontaneous action potential firing and GABAA receptors that generate ultra-fast inhibitory synaptic currents, raising the possibility that these biophysical attributes may permit CbN cells to respond differently to the degree of temporal coherence of their Purkinje cell inputs. In vivo, self-initiated motor programs associated with whisking correlates with asynchronous changes in Purkinje cell simple spiking that are asynchronous across the population. The resulting inhibition converges with mossy fiber excitation to yield little change in CbN cell firing, such that cerebellar output is low or cancelled. In contrast, externally applied sensory stimuli elicits a transient, synchronous inhibition of Purkinje cell simple spiking. During the resulting strong disinhibition of CbN cells, sensory-induced excitation from mossy fibers effectively drives cerebellar outputs that increase the magnitude of reflexive whisking. Purkinje cell synchrony, therefore, may be a key variable contributing to the “positive effort” hypothesized by David Marr in 1969 to be necessary for cerebellar control of movement.
Tapping the beat of four subdivisions: Neural entrainment, musical training and the binary advantage
The subdivision benefit refers to the positive effect of subdividing a beat on sensorimotor synchronization. We recorded electroencephalograms of musicians and non-musicians to study how they listened or finger-tapped to a beat, subdivided into four distinct subdivisions. Musicians showed more consistent tapping responses than non-musicians, and enhanced neural entrainment during the tapping task than in the listening task. In both groups, there was a neural enhancement of the beat frequency and its first harmonic (related to duplets) after listening to the four subdivisions. Furthermore, non-musicians tapped more consistently to the beat of duplets than other subdivisions. Altogether, this suggests a neural and behavioral advantage for binary subdivisions, that can be modulated with formal training in music.
Learning Speech Perception and Action through Sensorimotor Interactions
Sensorimotor -independent brain representations in association cortices
How flexible are association cortices? I will present a series of fMRI experiments addressing this question by investigating individuals born without hands, who use their feet as effectors to perform everyday actions. These results suggest that computations in association cortices are abstracted from visuomotor features and experience, similarly to the visual -independence of the association networks in people born blind, highlighting these regions’ ability to compensate for experience in any specific modality. These findings also open new avenues to utilize effector-independence in the action system for motor rehabilitation.
Understanding sensorimotor control at global and local scales
The brain is remarkably flexible, and appears to instantly reconfigure its processing depending on what’s needed to solve a task at hand: fMRI studies indicate that distal brain areas appear to fluidly couple and decouple with one another depending on behavioral context. But the structural architecture of the brain is comprised of long-range axonal projections that are relatively fixed by adulthood. How does the global dynamism evident in fMRI recordings manifest at a cellular level? To bridge the gap between the activity of single neurons and cortex-wide networks, we correlated electrophysiological recordings of individual neurons in primary visual (V1) and retrosplenial (RSP) associational cortex with activity across dorsal cortex, recorded simultaneously using widefield calcium imaging. We found that individual neurons in both cortical areas independently engaged in different distributed cortical networks depending on the animal’s behavioral state, suggesting that locomotion puts cortex into a more sensory driven mode relevant for navigation.
Restless engrams: the origin of continually reconfiguring neural representations
During learning, populations of neurons alter their connectivity and activity patterns, enabling the brain to construct a model of the external world. Conventional wisdom holds that the durability of a such a model is reflected in the stability of neural responses and the stability of synaptic connections that form memory engrams. However, recent experimental findings have challenged this idea, revealing that neural population activity in circuits involved in sensory perception, motor planning and spatial memory continually change over time during familiar behavioural tasks. This continual change suggests significant redundancy in neural representations, with many circuit configurations providing equivalent function. I will describe recent work that explores the consequences of such redundancy for learning and for task representation. Despite large changes in neural activity, we find cortical responses in sensorimotor tasks admit a relatively stable readout at the population level. Furthermore, we find that redundancy in circuit connectivity can make a task easier to learn and compensate for deficiencies in biological learning rules. Finally, if neuronal connections are subject to an unavoidable level of turnover, the level of plasticity required to optimally maintain a memory is generally lower than the total change due to turnover itself, predicting continual reconfiguration of an engram.
Sensorimotor circuitry for shape discrimination in mice
Neural circuits for goal-directed sensorimotor transformation
Basal ganglia circuits underlying sensorimotor function
Corticothalamic cells: a critical link in forebrain sensorimotor loops
Molecular controls over corticospinal neuron axon branching at specific spinal segments
Corticospinal neurons (CSN) are the cortical projection neurons that innervate the spinal cord and some brainstem targets with segmental precision to control voluntary movement of specific functional motor groups, limb sections, or individual digits, yet molecular regulation over CSN segmental target specificity is essentially unknown. CSN subpopulations exhibit striking axon targeting specificity from development into maturity: Evolutionarily newer rostrolateral CSN exclusively innervate bulbar-cervical targets (CSNBC-lat), while evolutionarily older caudomedial CSN (CSNmed) are more heterogeneous, with distinct subpopulations extending axons to either bulbar-cervical or thoraco-lumbar segments. The cervical cord, with its evolutionarily enhanced precision of forelimb movement, is innervated by multiple CSN subpopulations, suggesting inter-neuronal interactions in establishing corticospinal connectivity. I identify that Lumican, previously unrecognized in axon development, controls the specificity of cervical spinal cord innervation by CSN. Remarkably, Lumican, an extracellular matrix protein expressed by CSNBC-lat, non-cell-autonomously suppresses axon collateralization in the cervical cord by CSNmed. Intersectional viral labeling and mouse genetics further identify that Lumican controls axon collateralization by multiple subpopulations in caudomedial sensorimotor cortex. These results identify inter-axonal molecular crosstalk between CSN subpopulations as a novel mechanism controlling corticospinal connectivity and competitive specificity. Further, this mechanism has potential implications for evolutionary diversification of corticospinal circuitry with finer scale precision. "" Complementing this work, to comprehensively elucidate related axon projection mechanisms functioning at tips of growing CSN axons in vivo, I am currently applying experimental and analytic approaches recently developed in my postdoc lab (Poulopoulos*, Murphy*, Nature, 2019) to quantitatively and subcellularly “map” RNA and protein molecular machinery of subtype-specific growth cones, in parallel to their parent somata, isolated directly in vivo from developing subcerebral projection neurons (SCPN; the broader cortical output neuron population targeting both brainstem and spinal cord; includes CSN). I am investigating both normal development and GC-soma dysregulation with mutation of central CSN-SCPN transcriptional regulator Ctip2/Bcl11b.
Contextual inference underlies the learning of sensorimotor repertoires
Humans spend a lifetime learning, storing and refining a repertoire of motor memories. However, it is unknown what principle underlies the way our continuous stream of sensori-motor experience is segmented into separate memories and how we adapt and use this growing repertoire. Here we develop a principled theory of motor learning based on the key insight that memory creation, updating, and expression are all controlled by a single computation – contextual inference. Unlike dominant theories of single-context learning, our repertoire-learning model accounts for key features of motor learning that had no unified explanation and predicts novel phenomena, which we confirm experimentally. These results suggest that contextual inference is the key principle underlying how a diverse set of experiences is reflected in motor behavior.
An evolutionarily conserved hindwing circuit mediates Drosophila flight control
My research at the interface of neurobiology, biomechanics, and behavior seeks to understand how the timing precision of sensory input structures locomotor output. My lab studies the flight behavior of the fruit fly, Drosophila melanogaster, combining powerful genetic tools available for labeling and manipulating neural circuits with cutting-edge imaging in awake, behaving animals. This work has the potential to fundamentally reshape understanding of the evolution of insect flight, as well as highlight the tremendous importance of timing in the context of locomotion. Timing is crucial to the nervous system. The ability to rapidly detect and process subtle disturbances in the environment determines whether an animal can attain its next meal or successfully navigate complex, unpredictable terrain. While previous work on various animals has made tremendous strides uncovering the specialized neural circuits used to resolve timing differences with sub-microsecond resolution, it has focused on the detection of timing differences in sensory systems. Understanding of how the timing of motor output is structured by precise sensory input remains poor. My research focuses on an organ unique to fruit flies, called the haltere, that serves as a bridge for detecting and acting on subtle timing differences, helping flies execute rapid maneuvers. Understanding how this relatively simple insect canperform such impressive aerial feats demands an integrative approach that combines physics, muscle mechanics, neuroscience, and behavior. This unique, powerful approach will reveal the general principles that govern sensorimotor processing.
Motor BMIs for probing sensorimotor control and parsing distributed learning
Brain-machine interfaces (BMIs) change how the brain sends and receives information from the environment, opening new ways to probe brain function. For instance, motor BMIs allow us to precisely define and manipulate the sensorimotor loop which has enabled new insights into motor control and learning. In this talk, I’ll first present an example study where sensory-motor loop manipulations in BMI allowed us to probe feed-forward and feedback control mechanisms in ways that are not possible in the natural motor system. This study shed light on sensorimotor processing, and in turn led to state-of-the-art neural interface performance. I’ll then survey recent work that highlights the likelihood that BMIs, much like natural motor learning, engages multiple distributed learning mechanisms that can be carefully interrogated with BMI.
Understanding sensorimotor control at global and local scales
The brain is remarkably flexible, and appears to instantly reconfigure its processing depending on what’s needed to solve a task at hand: fMRI studies indicate that distal brain areas appear to fluidly couple and decouple with one another depending on behavioral context. We investigated how the brain coordinates its activity across areas to inform complex, top-down control behaviors. Animals were trained to perform a novel brain machine interface task to guide a visual cursor to a reward zone, using activity recorded with widefield calcium imaging. This allowed us to screen for cortical areas implicated in causal neural control of the visual object. Animals could decorrelate normally highly-correlated areas to perform the task, and used an explore-exploit search in neural activity space to discover successful strategies. Higher visual and parietal areas were more active during the task in expert animals. Single unit recordings targeted to these areas indicated that the sensory representation of an object was sensitive to an animal’s subjective sense of controlling it.
Motor Cortical Control of Vocal Interactions in a Neotropical Singing Mouse
Using sounds for social interactions is common across many taxa. Humans engaged in conversation, for example, take rapid turns to go back and forth. This ability to act upon sensory information to generate a desired motor output is a fundamental feature of animal behavior. How the brain enables such flexible sensorimotor transformations, for example during vocal interactions, is a central question in neuroscience. Seeking a rodent model to fill this niche, we are investigating neural mechanisms of vocal interaction in Alston’s singing mouse (Scotinomys teguina) – a neotropical rodent native to the cloud forests of Central America. We discovered sub-second temporal coordination of advertisement songs (counter-singing) between males of this species – a behavior that requires the rapid modification of motor outputs in response to auditory cues. We leveraged this natural behavior to probe the neural mechanisms that generate and allow fast and flexible vocal communication. Using causal manipulations, we recently showed that an orofacial motor cortical area (OMC) in this rodent is required for vocal interactions (Okobi*, Banerjee* et. al, 2019). Subsequently, in electrophysiological recordings, I find neurons in OMC that track initiation, termination and relative timing of songs. Interestingly, persistent neural dynamics during song progression stretches or compresses on every trial to match the total song duration (Banerjee et al, in preparation). These results demonstrate robust cortical control of vocal timing in a rodent and upends the current dogma that motor cortical control of vocal output is evolutionarily restricted to the primate lineage.
Food Mind Control: Regulation of Sensory Behaviors by Gut-Brain Signaling
How does the presence or absence of food shape and prioritize behavioral decisions? When is food more than just food? As in other animals, prolonged food deprivation dramatically alters sensory behaviors in C. elegans. For instance, it has been known since the mid-1970s that hungry worms no longer respond to temperature changes in their environment, but the underlying mechanisms have been unclear. I will describe unpublished work showing that insulin signaling from the gut regulates thermosensory behaviors as a function of feeding state by engaging a modulatory sensorimotor circuit that gates the output of the core thermosensory network. C. elegans is associated with, and consumes, diverse bacteria in the wild. I will also discuss a recent story in which we find that in addition to providing nutrition, a bacterial strain in the worm gut alters the hosts’ olfactory behavior and drives food choice decisions by producing a neurotransmitter that targets the hosts’ sensory neurons. These results add to our growing body of knowledge of how signaling from the gut modulates peripheral and central neuron properties and drives sensory behavioral plasticity.
Information and Decision-Making
In recent years it has become increasingly clear that (Shannon) information is a central resource for organisms, akin in importance to energy. Any decision that an organism or a subsystem of an organism takes involves the acquisition, selection, and processing of information and ultimately its concentration and enaction. It is the consequences of this balance that will occupy us in this talk. This perception-action loop picture of an agent's life cycle is well established and expounded especially in the context of Fuster's sensorimotor hierarchies. Nevertheless, the information-theoretic perspective drastically expands the potential and predictive power of the perception-action loop perspective. On the one hand information can be treated - to a significant extent - as a resource that is being sought and utilized by an organism. On the other hand, unlike energy, information is not additive. The intrinsic structure and dynamics of information can be exceedingly complex and subtle; in the last two decades one has discovered that Shannon information possesses a rich and nontrivial intrinsic structure that must be taken into account when informational contributions, information flow or causal interactions of processes are investigated, whether in the brain or in other complex processes. In addition, strong parallels between information and control theory have emerged. This parallelism between the theories allows one to obtain unexpected insights into the nature and properties of the perception-action loop. Through the lens of information theory, one can not only come up with novel hypotheses about necessary conditions for the organization of information processing in a brain, but also with constructive conjectures and predictions about what behaviours, brain structure and dynamics and even evolutionary pressures one can expect to operate on biological organisms, induced purely by informational considerations.
Untangling the web of behaviours used to produce spider orb webs
Many innate behaviours are the result of multiple sensorimotor programs that are dynamically coordinated to produce higher-order behaviours such as courtship or architecture construction. Extendend phenotypes such as architecture are especially useful for ethological study because the structure itself is a physical record of behavioural intent. A particularly elegant and easily quantifiable structure is the spider orb-web. The geometric symmetry and regularity of these webs have long generated interest in their behavioural origin. However, quantitative analyses of this behaviour have been sparse due to the difficulty of recording web-making in real-time. To address this, we have developed a novel assay enabling real-time, high-resolution tracking of limb movements and web structure produced by the hackled orb-weaver Uloborus diversus. With its small brain size of approximately 100,000 neurons, the spider U. diversus offers a tractable model organism for the study of complex behaviours. Using deep learning frameworks for limb tracking, and unsupervised behavioural clustering methods, we have developed an atlas of stereotyped movement motifs and are investigating the behavioural state transitions of which the geometry of the web is an emergent property. In addition to tracking limb movements, we have developed algorithms to track the web’s dynamic graph structure. We aim to model the relationship between the spider’s sensory experience on the web and its motor decisions, thereby identifying the sensory and internal states contributing to this sensorimotor transformation. Parallel efforts in our group are establishing 2-photon in vivo calcium imaging protocols in this spider, eventually facilitating a search for neural correlates underlying the internal and sensory state variables identified by our behavioural models. In addition, we have assembled a genome, and are developing genetic perturbation methods to investigate the genetic underpinnings of orb-weaving behaviour. Together, we aim to understand how complex innate behaviours are coordinated by underlying neuronal and genetic mechanisms.
Neural and computational principles of the processing of dynamic faces and bodies
Body motion is a fundamental signal of social communication. This includes facial as well as full-body movements. Combining advanced methods from computer animation with motion capture in humans and monkeys, we synthesized highly-realistic monkey avatar models. Our face avatar is perceived by monkeys as almost equivalent to a real animal, and does not induce an ‘uncanny valley effect’, unlike all other previously used avatar models in studies with monkeys. Applying machine-learning methods for the control of motion style, we were able to investigate how species-specific shape and dynamic cues influence the perception of human and monkey facial expressions. Human observers showed very fast learning of monkey expressions, and a perceptual encoding of expression dynamics that was largely independent of facial shape. This result is in line with the fact that facial shape evolved faster than the neuromuscular control in primate phylogenesis. At the same time, it challenges popular neural network models of the recognition of dynamic faces that assume a joint encoding of facial shape and dynamics. We propose an alternative physiologically-inspired neural model that realizes such an orthogonal encoding of facial shape and expression from video sequences. As second example, we investigated the perception of social interactions from abstract stimuli, similar to the ones by Heider & Simmel (1944), and also from more realistic stimuli. We developed and validated a new generative model for the synthesis of such social interaction, which is based on a modification of human navigation model. We demonstrate that the recognition of such stimuli, including the perception of agency, can be accounted for by a relatively elementary physiologically-inspired hierarchical neural recognition model, that does not require the assumption of sophisticated inference mechanisms, as postulated by some cognitive theories of social recognition. Summarizing, this suggests that essential phenomena in social cognition might be accounted for by a small set of simple neural principles that can be easily implemented by cortical circuits. The developed technologies for stimulus control form the basis of electrophysiological studies that can verify specific neural circuits, as the ones proposed by our theoretical models.
Cortical population coding of consumption decisions
The moment that a tasty substance enters an animal’s mouth, the clock starts ticking. Taste information transduced on the tongue signals whether a potential food will nourish or poison, and the animal must therefore use this information quickly if it is to decide whether the food should be swallowed or expelled. The system tasked with computing this important decision is rife with cross-talk and feedback—circuitry that all but ensures dynamics and between-neuron coupling in neural responses to tastes. In fact, cortical taste responses, rather than simply reporting individual taste identities, do contain characterizable dynamics: taste-driven firing first reflects the substance’s presence on the tongue, and then broadly codes taste quality, and then shifts again to correlate with the taste’s current palatability—the basis of consumption decisions—all across the 1-1.5 seconds after taste administration. Ensemble analyses reveal the onset of palatability-related firing to be a sudden, nonlinear transition happening in many neurons simultaneously, such that it can be reliably detected in single trials. This transition faithfully predicts both the nature and timing of consumption behaviours, despite the huge trial-to-trial variability in both; furthermore, perturbations of this transition interfere with production of the behaviours. These results demonstrate the specific importance of ensemble dynamics in the generation of behaviour, and reveal the taste system to be akin to a range of other integrated sensorimotor systems.
The active modulation of sound and vibration perception
The dominant view of perception right now is that information travels from the environment to the sensory system, then to the nervous systems which processes it to generate a percept and behaviour. Ongoing behaviour is thought to occur largely through simple iterations of this process. However, this linear view, where information flows only in one direction and the properties of the environment and the sensory system remain static and unaffected by behaviour, is slowly fading. Many of us are beginning to appreciate that perception is largely active, i.e. that information flows back and forth between the three systems modulating their respective properties. In other words, in the real world, the environment and sensorimotor loop is pretty much always closed. I study the loop; in particular I study how the reverse arm of the loop affects sound and vibration perception. I will present two examples of motor modulation of perception at two very different temporal and spatial scales. First, in crickets, I will present data on how high-speed molecular motor activity enhances hearing via the well-studied phenomenon of active amplification. Second, in spiders I will present data on how body posture, a slow macroscopic feature, which can barely be called ‘active’, can nonetheless modulate vibration perception. I hope these results will motivate a conversation about whether ‘active’ perception is an optional feature observed in some sensory systems, or something that is ultimately necessitated by both evolution and physics.
Auditory cortex represents an abstract sensorimotor rule
COSYNE 2022
Differential encoding of innate and learned behaviors in the sensorimotor striatum
COSYNE 2022
Dual pathway architecture in songbirds boosts sensorimotor learning
COSYNE 2022
Inferring implicit sensorimotor costs by inverse optimal control with signal dependent noise
COSYNE 2022
Inferring implicit sensorimotor costs by inverse optimal control with signal dependent noise
COSYNE 2022
Multi-region Poisson GPFA isolates shared and independent latent structure in sensorimotor tasks
COSYNE 2022
Multi-region Poisson GPFA isolates shared and independent latent structure in sensorimotor tasks
COSYNE 2022
Widespread representations of sensory evidence with distinct temporal dynamics across the sensorimotor axis
COSYNE 2022
Widespread representations of sensory evidence with distinct temporal dynamics across the sensorimotor axis
COSYNE 2022
Abstract structure and generalization in sensorimotor networks configured with semantic-based instruction embeddings
COSYNE 2023
Dynamical Neural Computation in Predictive Sensorimotor Control
COSYNE 2023
Sensorimotor prediction errors in the mouse olfactory cortex
COSYNE 2023
Brain-wide neural dynamics accompanying fast goal-directed sensorimotor learning
COSYNE 2025
Complementary goal and prediction-driven learning systems in a model of mammalian sensorimotor areas
COSYNE 2025
An encoding model to study the sensorimotor cortex of freely-behaving monkeys
COSYNE 2025
How internal states shape sensorimotor mapping in zebrafish larvae
COSYNE 2025
A model linking neural population activity to flexibility in sensorimotor control
COSYNE 2025
Reach-to-grasp activity is organized along an abstract-to-detailed gradient in mouse sensorimotor cortex
COSYNE 2025
An all-optical approach to disentangle the role of intrinsic and synaptic plasticity in sensorimotor learning
FENS Forum 2024
Brain-wide neuronal dynamics underlying rapid goal-directed sensorimotor learning
FENS Forum 2024
Two distinct inhibitory neuronal classes govern acquisition and recall of spinal sensorimotor learning
FENS Forum 2024
Effect of high-intensity interval and moderate-intensity continuous training on neuroplasticity, cognition, and sensorimotor performance in aged rats
FENS Forum 2024
Exercise partially prevents motor alteration induced by early sensorimotor restriction in rats
FENS Forum 2024
Fast pathway-specific reorganization of barrel cortex underlying rapid goal-directed sensorimotor learning
FENS Forum 2024
Frequency tagging in the sensorimotor cortex is enhanced by footstep sounds compared to visual information movement in a walking movement integration task
FENS Forum 2024
Machine learning approach applied to exploration of neuronal sensorimotor processing during a visuomotor rule-based task performed by a monkey
FENS Forum 2024
Mapping sensorimotor circuit underlying defensive actions in Drosophila larva
FENS Forum 2024
Mapping the sensorimotor connectome underlying protein-specific appetites in Drosophila melanogaster
FENS Forum 2024
Modeling the sensorimotor system with deep reinforcement learning
FENS Forum 2024
Retrieval inhibition during sensorimotor consolidation modulates memory retention
FENS Forum 2024
Role of uncertainty about grasp type in sensorimotor integration during dexterous object manipulation
FENS Forum 2024
Sensorimotor integration in the zebrafish inferior olive during motor adaptation
FENS Forum 2024
Sensorimotor representations of stimulus velocity in larval zebrafish
FENS Forum 2024
Sparse and unique functional innervation of barrel cortex onto single projection neurons in dorsal striatum and its plasticity after sensorimotor learning
FENS Forum 2024
Superior colliculus as a key player in Huntington’s disease sensorimotor coordination deficits: From circuits to behaviour
FENS Forum 2024
Targeted recombination in active populations to study sensorimotor learning
FENS Forum 2024