Single-neuron correlates of perception and memory in the human medial temporal lobe

The human medial temporal lobe contains neurons that respond selectively to the semantic contents of a presented stimulus. These "concept cells" may respond to very different pictures of a given person and even to their written or spoken name. Their response latency is far longer than necessary for object recognition, they follow subjective, conscious perception, and they are found in brain regions that are crucial for declarative memory formation. It has thus been hypothesized that they may represent the semantic "building blocks" of episodic memories. In this talk I will present data from single unit recordings in the hippocampus, entorhinal cortex, parahippocampal cortex, and amygdala during paradigms involving object recognition and conscious perception as well as encoding of episodic memories in order to characterize the role of concept cells in these cognitive functions.

Contentopic mapping and object dimensionality - a novel understanding on the organization of object knowledge

Our ability to recognize an object amongst many others is one of the most important features of the human mind. However, object recognition requires tremendous computational effort, as we need to solve a complex and recursive environment with ease and proficiency. This challenging feat is dependent on the implementation of an effective organization of knowledge in the brain. Here I put forth a novel understanding of how object knowledge is organized in the brain, by proposing that the organization of object knowledge follows key object-related dimensions, analogously to how sensory information is organized in the brain. Moreover, I will also put forth that this knowledge is topographically laid out in the cortical surface according to these object-related dimensions that code for different types of representational content – I call this contentopic mapping. I will show a combination of fMRI and behavioral data to support these hypotheses and present a principled way to explore the multidimensionality of object processing.

Principles of Cognitive Control over Task Focus and Task

2024 BACN Mid-Career Prize Lecture Adaptive behavior requires the ability to focus on a current task and protect it from distraction (cognitive stability), and to rapidly switch tasks when circumstances change (cognitive flexibility). How people control task focus and switch-readiness has therefore been the target of burgeoning research literatures. Here, I review and integrate these literatures to derive a cognitive architecture and functional rules underlying the regulation of stability and flexibility. I propose that task focus and switch-readiness are supported by independent mechanisms whose strategic regulation is nevertheless governed by shared principles: both stability and flexibility are matched to anticipated challenges via an incremental, online learner that nudges control up or down based on the recent history of task demands (a recency heuristic), as well as via episodic reinstatement when the current context matches a past experience (a recognition heuristic).

The immunopathogenesis of autoimmune seizure disorders

Immune-mediated mechanisms are increasingly recognised as a cause of epilepsy even in the absence of an immune response against a specifical neuronal antigen. In some cases, these autoimmune processes are clearly pathogenic, for example acute seizures in autoimmune encephalitis, whereas in others this is less clear, for example autoimmune-associated epilepsy. Recent research has provided novel insights into the clinical, paraclinical and immunopathogenetic mechanisms in these conditions. I will provide an overview of clinical and paraclinical features of immune-associated seizures. Furthermore, I will describe specific immunopathogenic examples implicating lymphoid follicular autoimmunisation and intrathecal B cells in these conditions. These insights into immunopathogenesis may help to explain the role of current and immunotherapies in these conditions.

Of glia and macrophages, signaling hubs in development and homeostasis

We are interested in the biology of macrophages, which represent the first line of defense against pathogens. In Drosophila, the embryonic hemocytes arise from the mesoderm whereas glial cells arise from multipotent precursors in the neurogenic region. These cell types represent, respectively, the macrophages located outside and within the nervous system (similar to vertebrate microglia). Thus, despite their different origin, hemocytes and glia display common functions. In addition, both cell types express the Glide/Gcm transcription factor, which plays an evolutionarily conserved role as an anti-inflammatory factor. Moreover, embryonic hemocytes play an evolutionarily conserved and fundamental role in development. The ability to migrate and to contact different tissues/organs most likely allow macrophages to function as signaling hubs. The function of macrophages beyond the recognition of the non-self calls for revisiting the biology of these heterogeneous and plastic cells in physiological and pathological conditions across evolution.

Deepfake Detection in Super-Recognizers and Police Officers

Using videos from the Deepfake Detection Challenge (cf. Groh et al., 2021), we investigated human deepfake detection performance (DDP) in two unique observer groups: Super-Recognizers (SRs) and "normal" officers from within the 18K members of the Berlin Police. SRs were identified either via previously proposed lab-based procedures (Ramon, 2021) or the only existing tool for SR identification involving increasingly challenging, authentic forensic material: beSure® (Berlin Test For Super-Recognizer Identification; Ramon & Rjosk, 2022). Across two experiments we examined deepfake detection performance (DDP) in participants who judged single videos and pairs of videos in a 2AFC decision setting. We explored speed-accuracy trade-offs in DDP, compared DDP between lab-identified SRs and non-SRs, and police officers whose face identity processing skills had been extensively tested using challenging. In this talk I will discuss our surprising findings and argue that further work is needed too determine whether face identity processing is related to DDP or not.

Recognizing Faces: Insights from Group and Individual Differences

Decoding mental conflict between reward and curiosity in decision-making

Humans and animals are not always rational. They not only rationally exploit rewards but also explore an environment owing to their curiosity. However, the mechanism of such curiosity-driven irrational behavior is largely unknown. Here, we developed a decision-making model for a two-choice task based on the free energy principle, which is a theory integrating recognition and action selection. The model describes irrational behaviors depending on the curiosity level. We also proposed a machine learning method to decode temporal curiosity from behavioral data. By applying it to rat behavioral data, we found that the rat had negative curiosity, reflecting conservative selection sticking to more certain options and that the level of curiosity was upregulated by the expected future information obtained from an uncertain environment. Our decoding approach can be a fundamental tool for identifying the neural basis for reward–curiosity conflicts. Furthermore, it could be effective in diagnosing mental disorders.

Vision Unveiled: Understanding Face Perception in Children Treated for Congenital Blindness

Despite her still poor visual acuity and minimal visual experience, a 2-3 month old baby will reliably respond to facial expressions, smiling back at her caretaker or older sibling. But what if that same baby had been deprived of her early visual experience? Will she be able to appropriately respond to seemingly mundane interactions, such as a peer’s facial expression, if she begins seeing at the age of 10? My work is part of Project Prakash, a dual humanitarian/scientific mission to identify and treat curably blind children in India and then study how their brain learns to make sense of the visual world when their visual journey begins late in life. In my talk, I will give a brief overview of Project Prakash, and present findings from one of my primary lines of research: plasticity of face perception with late sight onset. Specifically, I will discuss a mixed methods effort to probe and explain the differential windows of plasticity that we find across different aspects of distributed face recognition, from distinguishing a face from a nonface early in the developmental trajectory, to recognizing facial expressions, identifying individuals, and even identifying one’s own caretaker. I will draw connections between our empirical findings and our recent theoretical work hypothesizing that children with late sight onset may suffer persistent face identification difficulties because of the unusual acuity progression they experience relative to typically developing infants. Finally, time permitting, I will point to potential implications of our findings in supporting newly-sighted children as they transition back into society and school, given that their needs and possibilities significantly change upon the introduction of vision into their lives.

Microbial modulation of zebrafish behavior and brain development

There is growing recognition that host-associated microbiotas modulate intrinsic neurodevelopmental programs including those underlying human social behavior. Despite this awareness, the fundamental processes are generally not understood. We discovered that the zebrafish microbiota is necessary for normal social behavior. By examining neuronal correlates of behavior, we found that the microbiota restrains neurite complexity and targeting of key forebrain neurons within the social behavior circuitry. The microbiota is also necessary for both localization and molecular functions of forebrain microglia, brain-resident phagocytes that remodel neuronal arbors. In particular, the microbiota promotes expression of complement signaling pathway components important for synapse remodeling. Our work provides evidence that the microbiota modulates zebrafish social behavior by stimulating microglial remodeling of forebrain circuits during early neurodevelopment and suggests molecular pathways for therapeutic interventions during atypical neurodevelopment.

From cells to systems: multiscale studies of the epileptic brain

It is increasingly recognized that epilepsy affects human brain organization across multiple scales, ranging from cellular alterations in specific regions towards macroscale network imbalances. My talk will overview an emerging paradigm that integrates cellular, neuroimaging, and network modelling approaches to faithful characterize the extent of structural and functional alterations in the common epilepsies. I will also discuss how multiscale framework can help to derive clinically useful biomarkers of dysfunction, and how these methods may guide surgical planning and prognostics.

Off the rails - how pathological patterns of whole brain activity emerge in epileptic seizures

In most brains across the animal kingdom, brain dynamics can enter pathological states that are recognisable as epileptic seizures. Yet usually, brain operate within certain constraints given through neuronal function and synaptic coupling, that will prevent epileptic seizure dynamics from emerging. In this talk, I will bring together different approaches to identifying how networks in the broadest sense shape brain dynamics. Using illustrative examples from intracranial EEG recordings, disorders characterised by molecular disruption of a single neurotransmitter receptor type, to single-cell recordings of whole-brain activity in the larval zebrafish, I will address three key questions - (1) how does the regionally specific composition of synaptic receptors shape ongoing physiological brain activity; (2) how can disruption of this regionally specific balance result in abnormal brain dynamics; and (3) which cellular patterns underly the transition into an epileptic seizure.

Learning to see stuff

Humans are very good at visually recognizing materials and inferring their properties. Without touching surfaces, we can usually tell what they would feel like, and we enjoy vivid visual intuitions about how they typically behave. This is impressive because the retinal image that the visual system receives as input is the result of complex interactions between many physical processes. Somehow the brain has to disentangle these different factors. I will present some recent work in which we show that an unsupervised neural network trained on images of surfaces spontaneously learns to disentangle reflectance, lighting and shape. However, the disentanglement is not perfect, and we find that as a result the network not only predicts the broad successes of human gloss perception, but also the specific pattern of errors that humans exhibit on an image-by-image basis. I will argue this has important implications for thinking about appearance and vision more broadly.

Analyzing artificial neural networks to understand the brain

In the first part of this talk I will present work showing that recurrent neural networks can replicate broad behavioral patterns associated with dynamic visual object recognition in humans. An analysis of these networks shows that different types of recurrence use different strategies to solve the object recognition problem. The similarities between artificial neural networks and the brain presents another opportunity, beyond using them just as models of biological processing. In the second part of this talk, I will discuss—and solicit feedback on—a proposed research plan for testing a wide range of analysis tools frequently applied to neural data on artificial neural networks. I will present the motivation for this approach as well as the form the results could take and how this would benefit neuroscience.

Representations of people in the brain

Faces and voices convey much of the non-verbal information that we use when communicating with other people. We look at faces and listen to voices to recognize others, understand how they are feeling, and decide how to act. Recent research in my lab aims to investigate whether there are similar coding mechanisms to represent faces and voices, and whether there are brain regions that integrate information across the visual and auditory modalities. In the first part of my talk, I will focus on an fMRI study in which we found that a region of the posterior STS exhibits modality-general representations of familiar people that can be similarly driven by someone’s face and their voice (Tsantani et al. 2019). In the second part of the talk, I will describe our recent attempts to shed light on the type of information that is represented in different face-responsive brain regions (Tsantani et al., 2021).

Training Dynamic Spiking Neural Network via Forward Propagation Through Time

With recent advances in learning algorithms, recurrent networks of spiking neurons are achieving performance competitive with standard recurrent neural networks. Still, these learning algorithms are limited to small networks of simple spiking neurons and modest-length temporal sequences, as they impose high memory requirements, have difficulty training complex neuron models, and are incompatible with online learning.Taking inspiration from the concept of Liquid Time-Constant (LTCs), we introduce a novel class of spiking neurons, the Liquid Time-Constant Spiking Neuron (LTC-SN), resulting in functionality similar to the gating operation in LSTMs. We integrate these neurons in SNNs that are trained with FPTT and demonstrate that thus trained LTC-SNNs outperform various SNNs trained with BPTT on long sequences while enabling online learning and drastically reducing memory complexity. We show this for several classical benchmarks that can easily be varied in sequence length, like the Add Task and the DVS-gesture benchmark. We also show how FPTT-trained LTC-SNNs can be applied to large convolutional SNNs, where we demonstrate novel state-of-the-art for online learning in SNNs on a number of standard benchmarks (S-MNIST, R-MNIST, DVS-GESTURE) and also show that large feedforward SNNs can be trained successfully in an online manner to near (Fashion-MNIST, DVS-CIFAR10) or exceeding (PS-MNIST, R-MNIST) state-of-the-art performance as obtained with offline BPTT. Finally, the training and memory efficiency of FPTT enables us to directly train SNNs in an end-to-end manner at network sizes and complexity that was previously infeasible: we demonstrate this by training in an end-to-end fashion the first deep and performant spiking neural network for object localization and recognition. Taken together, we out contribution enable for the first time training large-scale complex spiking neural network architectures online and on long temporal sequences.

Behavioral Timescale Synaptic Plasticity (BTSP) for biologically plausible credit assignment across multiple layers via top-down gating of dendritic plasticity

A central problem in biological learning is how information about the outcome of a decision or behavior can be used to reliably guide learning across distributed neural circuits while obeying biological constraints. This “credit assignment” problem is commonly solved in artificial neural networks through supervised gradient descent and the backpropagation algorithm. In contrast, biological learning is typically modelled using unsupervised Hebbian learning rules. While these rules only use local information to update synaptic weights, and are sometimes combined with weight constraints to reflect a diversity of excitatory (only positive weights) and inhibitory (only negative weights) cell types, they do not prescribe a clear mechanism for how to coordinate learning across multiple layers and propagate error information accurately across the network. In recent years, several groups have drawn inspiration from the known dendritic non-linearities of pyramidal neurons to propose new learning rules and network architectures that enable biologically plausible multi-layer learning by processing error information in segregated dendrites. Meanwhile, recent experimental results from the hippocampus have revealed a new form of plasticity—Behavioral Timescale Synaptic Plasticity (BTSP)—in which large dendritic depolarizations rapidly reshape synaptic weights and stimulus selectivity with as little as a single stimulus presentation (“one-shot learning”). Here we explore the implications of this new learning rule through a biologically plausible implementation in a rate neuron network. We demonstrate that regulation of dendritic spiking and BTSP by top-down feedback signals can effectively coordinate plasticity across multiple network layers in a simple pattern recognition task. By analyzing hidden feature representations and weight trajectories during learning, we show the differences between networks trained with standard backpropagation, Hebbian learning rules, and BTSP.

Beyond Biologically Plausible Spiking Networks for Neuromorphic Computing

Biologically plausible spiking neural networks (SNNs) are an emerging architecture for deep learning tasks due to their energy efficiency when implemented on neuromorphic hardware. However, many of the biological features are at best irrelevant and at worst counterproductive when evaluated in the context of task performance and suitability for neuromorphic hardware. In this talk, I will present an alternative paradigm to design deep learning architectures with good task performance in real-world benchmarks while maintaining all the advantages of SNNs. We do this by focusing on two main features – event-based computation and activity sparsity. Starting from the performant gated recurrent unit (GRU) deep learning architecture, we modify it to make it event-based and activity-sparse. The resulting event-based GRU (EGRU) is extremely efficient for both training and inference. At the same time, it achieves performance close to conventional deep learning architectures in challenging tasks such as language modelling, gesture recognition and sequential MNIST.

Shallow networks run deep: How peripheral preprocessing facilitates odor classification

Drosophila olfactory sensory hairs ("sensilla") typically house two olfactory receptor neurons (ORNs) which can laterally inhibit each other via electrical ("ephaptic") coupling. ORN pairing is highly stereotyped and genetically determined. Thus, olfactory signals arriving in the Antennal Lobe (AL) have been pre-processed by a fixed and shallow network at the periphery. To uncover the functional significance of this organization, we developed a nonlinear phenomenological model of asymmetrically coupled ORNs responding to odor mixture stimuli. We derived an analytical solution to the ORNs’ dynamics, which shows that the peripheral network can extract the valence of specific odor mixtures via transient amplification. Our model predicts that for efficient read-out of the amplified valence signal there must exist specific patterns of downstream connectivity that reflect the organization at the periphery. Analysis of AL→Lateral Horn (LH) fly connectomic data reveals evidence directly supporting this prediction. We further studied the effect of ephaptic coupling on olfactory processing in the AL→Mushroom Body (MB) pathway. We show that stereotyped ephaptic interactions between ORNs lead to a clustered odor representation of glomerular responses. Such clustering in the AL is an essential assumption of theoretical studies on odor recognition in the MB. Together our work shows that preprocessing of olfactory stimuli by a fixed and shallow network increases sensitivity to specific odor mixtures, and aids in the learning of novel olfactory stimuli. Work led by Palka Puri, in collaboration with Chih-Ying Su and Shiuan-Tze Wu.

Real-world scene perception and search from foveal to peripheral vision

A high-resolution central fovea is a prominent design feature of human vision. But how important is the fovea for information processing and gaze guidance in everyday visual-cognitive tasks? Following on from classic findings for sentence reading, I will present key results from a series of eye-tracking experiments in which observers had to search for a target object within static or dynamic images of real-world scenes. Gaze-contingent scotomas were used to selectively deny information processing in the fovea, parafovea, or periphery. Overall, the results suggest that foveal vision is less important and peripheral vision is more important for scene perception and search than previously thought. The importance of foveal vision was found to depend on the specific requirements of the task. Moreover, the data support a central-peripheral dichotomy in which peripheral vision selects and central vision recognizes.

General purpose event-based architectures for deep learning

Biologically plausible spiking neural networks (SNNs) are an emerging architecture for deep learning tasks due to their energy efficiency when implemented on neuromorphic hardware. However, many of the biological features are at best irrelevant and at worst counterproductive when evaluated in the context of task performance and suitability for neuromorphic hardware. In this talk, I will present an alternative paradigm to design deep learning architectures with good task performance in real-world benchmarks while maintaining all the advantages of SNNs. We do this by focusing on two main features -- event-based computation and activity sparsity. Starting from the performant gated recurrent unit (GRU) deep learning architecture, we modify it to make it event-based and activity-sparse. The resulting event-based GRU (EGRU) is extremely efficient for both training and inference. At the same time, it achieves performance close to conventional deep learning architectures in challenging tasks such as language modelling, gesture recognition and sequential MNIST

Neuroscience of socioeconomic status and poverty: Is it actionable?

SES neuroscience, using imaging and other methods, has revealed generalizations of interest for population neuroscience and the study of individual differences. But beyond its scientific interest, SES is a topic of societal importance. Does neuroscience offer any useful insights for promoting socioeconomic justice and reducing the harms of poverty? In this talk I will use research from my own lab and others’ to argue that SES neuroscience has the potential to contribute to policy in this area, although its application is premature at present. I will also attempt to forecast the ways in which practical solutions to the problems of poverty may emerge from SES neuroscience. Bio: Martha Farah has conducted groundbreaking research on face and object recognition, visual attention, mental imagery, and semantic memory and - in more recent times - has been at the forefront of interdisciplinary research into neuroscience and society. This deals with topics such as using fMRI for lie detection, ethics of cognitive enhancement, and effects of social deprivation on brain development.

New Insights into the Neural Machinery of Face Recognition

Don't forget the gametes: Neurodevelopmental pathogenesis starts in the sperm and egg

Proper development of the nervous system depends not only on the inherited DNA sequence, but also on proper regulation of gene expression, as controlled in part by epigenetic mechanisms present in the parental gametes. In this presentation an internationally recognized research advocate explains why researchers concerned about the origins of increasingly prevalent neurodevelopmental disorders such as autism and attention deficit hyperactivity disorder should look beyond genetics in probing the origins of dysregulated transcription of brain-related genes. The culprit for a subset of cases, she contends, may lie in the exposure history of the parents, and thus their germ cells. To illustrate how environmentally informed, nongenetic dysfunction may occur, she focuses on the example of parents' histories of exposure to common agents of modern inhalational anesthesia, a highly toxic exposure that in mammalian models has been seen to induce heritable neurodevelopmental abnormality in offspring born of exposed germline.

Feedforward and feedback processes in visual recognition

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching – and sometimes even surpassing – human accuracy on a variety of visual recognition tasks. In this talk, however, I will show that these neural networks and their recent extensions exhibit a limited ability to solve seemingly simple visual reasoning problems involving incremental grouping, similarity, and spatial relation judgments. Our group has developed a recurrent network model of classical and extra-classical receptive field circuits that is constrained by the anatomy and physiology of the visual cortex. The model was shown to account for diverse visual illusions providing computational evidence for a novel canonical circuit that is shared across visual modalities. I will show that this computational neuroscience model can be turned into a modern end-to-end trainable deep recurrent network architecture that addresses some of the shortcomings exhibited by state-of-the-art feedforward networks for solving complex visual reasoning tasks. This suggests that neuroscience may contribute powerful new ideas and approaches to computer science and artificial intelligence.

Unchanging and changing: hardwired taste circuits and their top-down control

The taste system detects 5 major categories of ethologically relevant stimuli (sweet, bitter, umami, sour and salt) and accordingly elicits acceptance or avoidance responses. While these taste responses are innate, the taste system retains a remarkable flexibility in response to changing external and internal contexts. Taste chemicals are first recognized by dedicated taste receptor cells (TRCs) and then transmitted to the cortex via a multi-station relay. I reasoned that if I could identify taste neural substrates along this pathway, it would provide an entry to decipher how taste signals are encoded to drive innate response and modulated to facilitate adaptive response. Given the innate nature of taste responses, these neural substrates should be genetically identifiable. I therefore exploited single-cell RNA sequencing to isolate molecular markers defining taste qualities in the taste ganglion and the nucleus of the solitary tract (NST) in the brainstem, the two stations transmitting taste signals from TRCs to the brain. How taste information propagates from the ganglion to the brain is highly debated (i.e., does taste information travel in labeled-lines?). Leveraging these genetic handles, I demonstrated one-to-one correspondence between ganglion and NST neurons coding for the same taste. Importantly, inactivating one ‘line’ did not affect responses to any other taste stimuli. These results clearly showed that taste information is transmitted to the brain via labeled lines. But are these labeled lines aptly adapted to the internal state and external environment? I studied the modulation of taste signals by conflicting taste qualities in the concurrence of sweet and bitter to understand how adaptive taste responses emerge from hardwired taste circuits. Using functional imaging, anatomical tracing and circuit mapping, I found that bitter signals suppress sweet signals in the NST via top-down modulation by taste cortex and amygdala of NST taste signals. While the bitter cortical field provides direct feedback onto the NST to amplify incoming bitter signals, it exerts negative feedback via amygdala onto the incoming sweet signal in the NST. By manipulating this feedback circuit, I showed that this top-down control is functionally required for bitter evoked suppression of sweet taste. These results illustrate how the taste system uses dedicated feedback lines to finely regulate innate behavioral responses and may have implications for the context-dependent modulation of hardwired circuits in general.

The evolution and development of visual complexity: insights from stomatopod visual anatomy, physiology, behavior, and molecules

Bioluminescence, which is rare on land, is extremely common in the deep sea, being found in 80% of the animals living between 200 and 1000 m. These animals rely on bioluminescence for communication, feeding, and/or defense, so the generation and detection of light is essential to their survival. Our present knowledge of this phenomenon has been limited due to the difficulty in bringing up live deep-sea animals to the surface, and the lack of proper techniques needed to study this complex system. However, new genomic techniques are now available, and a team with extensive experience in deep-sea biology, vision, and genomics has been assembled to lead this project. This project is aimed to study three questions 1) What are the evolutionary patterns of different types of bioluminescence in deep-sea shrimp? 2) How are deep-sea organisms’ eyes adapted to detect bioluminescence? 3) Can bioluminescent organs (called photophores) detect light in addition to emitting light? Findings from this study will provide valuable insight into a complex system vital to communication, defense, camouflage, and species recognition. This study will bring monumental contributions to the fields of deep sea and evolutionary biology, and immediately improve our understanding of bioluminescence and light detection in the marine environment. In addition to scientific advancement, this project will reach K-college aged students through the development and dissemination of educational tools, a series of molecular and organismal-based workshops, museum exhibits, public seminars, and biodiversity initiatives.

Brain and behavioural impacts of early life adversity

Abuse, neglect, and other forms of uncontrollable stress during childhood and early adolescence can lead to adverse outcomes later in life, including especially perturbations in the regulation of mood and emotional states, and specifically anxiety disorders and depression. However, stress experiences vary from one individual to the next, meaning that causal relationships and mechanistic accounts are often difficult to establish in humans. This interdisciplinary talk considers the value of research in experimental animals where stressor experiences can be tightly controlled and detailed investigations of molecular, cellular, and circuit-level mechanisms can be carried out. The talk will focus on the widely used repeated maternal separation procedure in rats where rat offspring are repeatedly separated from maternal care during early postnatal life. This early life stress has remarkably persistent effects on behaviour with a general recognition that maternally-deprived animals are susceptible to depressive-like phenotypes. The validity of this conclusion will be critically appraised with convergent insights from a recent longitudinal study in maternally separated rats involving translational brain imaging, transcriptomics, and behavioural assessment.

Functional segregation of rostral and caudal hippocampus in associative memory

It has long been established that the hippocampus plays a crucial role for episodic memory. As opposed to the modular approach, now it is generally assumed that being a complex structure, the HC performs multiplex interconnected functions, whose hierarchical organization provides basis for the higher cognitive functions such as semantics-based encoding and retrieval. However, the «where, when and how» properties of distinct memory aspects within and outside the HC are still under debate. Here we used a visual associative memory task as a probe to test the hypothesis about the differential involvement of the rostral and caudal portions of the human hippocampus in memory encoding, recognition and associative recall. In epilepsy patients implanted with stereo-EEG, we show that at retrieval the rostral HC is selectively active for recognition memory, whereas the caudal HC is selectively active for the associative memory. Low frequency desynchronization and high frequency synchronization characterize the temporal dynamic in encoding and retrieval. Therefore, we describe here anatomical segregation in the hippocampal contributions to associative and recognition memory.

Biopsychosocial pathways in dementia inequalities

In the United States, racial/ethnic inequalities in Alzheimer's disease and related dementias persist even after controlling for socioeconomic factors and physical health. These persistent and unexplained disparities suggest: (1) there are unrecognized dementia risk factors that are socially patterned and/or (2) known dementia risk factors exhibit differential impact across social groups. Pursuing these research directions with data from multiple longitudinal studies of brain and cognitive aging has revealed several challenges to the study of late-life health inequalities, highlighted evidence for both risk and resilience within marginalized communities, and inspired new data collection efforts to advance the field.

Object recognition by touch and other senses

Cross-modality imaging of the neural systems that support executive functions

Executive functions refer to a collection of mental processes such as attention, planning and problem solving, supported by a frontoparietal distributed brain network. These functions are essential for everyday life. Specifically in the context of patients with brain tumours there is a need to preserve them in order to enable good quality of life for patients. During surgeries for the removal of a brain tumour, the aim is to remove as much as possible of the tumour and at the same time prevent damage to the areas around it to preserve function and enable good quality of life for patients. In many cases, functional mapping is conducted during an awake surgery in order to identify areas critical for certain functions and avoid their surgical resection. While mapping is routinely done for functions such as movement and language, mapping executive functions is more challenging. Despite growing recognition in the importance of these functions for patient well-being in recent years, only a handful of studies addressed their intraoperative mapping. In the talk, I will present our new approach for mapping executive function areas using electrocorticography during awake brain surgery. These results will be complemented by neuroimaging data from healthy volunteers, directed at reliably localizing executive function regions in individuals using fMRI. I will also discuss more broadly challenges ofß using neuroimaging for neurosurgical applications. We aim to advance cross-modality neuroimaging of cognitive function which is pivotal to patient-tailored surgical interventions, and will ultimately lead to improved clinical outcomes.

A biological model system for studying predictive processing

Despite the increasing recognition of predictive processing in circuit neuroscience, little is known about how it may be implemented in cortical circuits. We set out to develop and characterise a biological model system with layer 5 pyramidal cells in the centre. We aim to gain access to prediction and internal model generating processes by controlling, understanding or monitoring everything else: the sensory environment, feed-forward and feed-back inputs, integrative properties, their spiking activity and output. I’ll show recent work from the lab establishing such a model system both in terms of biology as well as tool development.

Why is the suprachiasmatic nucleus such a brilliant circadian time-keeper?

Circadian clocks dominate our lives. By creating and distributing an internal representation of 24-hour solar time, they prepare us, and thereby adapt us, to the daily and seasonal world. Jet-lag is an obvious indicator of what can go wrong when such adaptation is disrupted acutely. More seriously, the growing prevalence of rotational shift-work which runs counter to our circadian life, is a significant chronic challenge to health, presenting as increased incidence of systemic conditions such as metabolic and cardiovascular disease. Added to this, circadian and sleep disturbances are a recognised feature of various neurological and psychiatric conditions, and in some cases may contribute to disease progression. The “head ganglion” of the circadian system is the suprachiasmatic nucleus (SCN) of the hypothalamus. It synchronises the, literally, innumerable cellular clocks across the body, to each other and to solar time. Isolated in organotypic slice culture, it can maintain precise, high-amplitude circadian cycles of neural activity, effectively, indefinitely, just as it does in vivo. How is this achieved: how does this clock in a dish work? This presentation will consider SCN time-keeping at the level of molecular feedback loops, neuropeptidergic networks and neuron-astrocyte interactions.

Multimodal framework and fusion of EEG, graph theory and sentiment analysis for the prediction and interpretation of consumer decision

The application of neuroimaging methods to marketing has recently gained lots of attention. In analyzing consumer behaviors, the inclusion of neuroimaging tools and methods is improving our understanding of consumer’s preferences. Human emotions play a significant role in decision making and critical thinking. Emotion classification using EEG data and machine learning techniques has been on the rise in the recent past. We evaluate different feature extraction techniques, feature selection techniques and propose the optimal set of features and electrodes for emotion recognition.Affective neuroscience research can help in detecting emotions when a consumer responds to an advertisement. Successful emotional elicitation is a verification of the effectiveness of an advertisement. EEG provides a cost effective alternative to measure advertisement effectiveness while eliminating several drawbacks of the existing market research tools which depend on self-reporting. We used Graph theoretical principles to differentiate brain connectivity graphs when a consumer likes a logo versus a consumer disliking a logo. The fusion of EEG and sentiment analysis can be a real game changer and this combination has the power and potential to provide innovative tools for market research.

Hearing in an acoustically varied world

In order for animals to thrive in their complex environments, their sensory systems must form representations of objects that are invariant to changes in some dimensions of their physical cues. For example, we can recognize a friend’s speech in a forest, a small office, and a cathedral, even though the sound reaching our ears will be very different in these three environments. I will discuss our recent experiments into how neurons in auditory cortex can form stable representations of sounds in this acoustically varied world. We began by using a normative computational model of hearing to examine how the brain may recognize a sound source across rooms with different levels of reverberation. The model predicted that reverberations can be removed from the original sound by delaying the inhibitory component of spectrotemporal receptive fields in the presence of stronger reverberation. Our electrophysiological recordings then confirmed that neurons in ferret auditory cortex apply this algorithm to adapt to different room sizes. Our results demonstrate that this neural process is dynamic and adaptive. These studies provide new insights into how we can recognize auditory objects even in highly reverberant environments, and direct further research questions about how reverb adaptation is implemented in the cortical circuit.

What does the primary visual cortex tell us about object recognition?

Object recognition relies on the complex visual representations in cortical areas at the top of the ventral stream hierarchy. While these are thought to be derived from low-level stages of visual processing, this has not been shown, yet. Here, I describe the results of two projects exploring the contributions of primary visual cortex (V1) processing to object recognition using artificial neural networks (ANNs). First, we developed hundreds of ANN-based V1 models and evaluated how their single neurons approximate those in the macaque V1. We found that, for some models, single neurons in intermediate layers are similar to their biological counterparts, and that the distributions of their response properties approximately match those in V1. Furthermore, we observed that models that better matched macaque V1 were also more aligned with human behavior, suggesting that object recognition is derived from low-level. Motivated by these results, we then studied how an ANN’s robustness to image perturbations relates to its ability to predict V1 responses. Despite their high performance in object recognition tasks, ANNs can be fooled by imperceptibly small, explicitly crafted perturbations. We observed that ANNs that better predicted V1 neuronal activity were also more robust to adversarial attacks. Inspired by this, we developed VOneNets, a new class of hybrid ANN vision models. Each VOneNet contains a fixed neural network front-end that simulates primate V1 followed by a neural network back-end adapted from current computer vision models. After training, VOneNets were substantially more robust, outperforming state-of-the-art methods on a set of perturbations. While current neural network architectures are arguably brain-inspired, these results demonstrate that more precisely mimicking just one stage of the primate visual system leads to new gains in computer vision applications and results in better models of the primate ventral stream and object recognition behavior.

From bench to clinic – Translating fundamental neuroscience into real-life healthcare practices, and developing nationally recognised life science companies

Dr. Ryan C.N. D’Arcy is a Canadian neuroscientist, researcher, innovator and entrepreneur. Dr. D'Arcy co-founded HealthTech Connex Inc. and serves as President and Chief Scientific Officer. HealthTech Connex translates neuroscience advances into health technology breakthroughs. D'Arcy is most known for coining the term "brain vital signs" and for leading the research and development of the brain vital signs framework. Dr. D’Arcy also holds a BC Leadership Chair in Medical Technology, is a full Professor at Simon Fraser University, and a member of the DM Centre for Brain Health at the University of British Columbia. He has published more than 260 academic works, attracted more than $85 Million CAD in competitive research and innovation funding, and been recognized through numerous awards and distinctions. Please join us for an exciting virtual talk with Dr. D'Arcy who will speak on some of the current research he is involved in, how he is translating this research into real-life applications, and the development of HealthTech Connects Inc.

If we can make computers play chess, why can't we make them see?

If we can make computers play chess and even Jeopardy and Go, then why can't we make them see like us? How does our brain solve the problem of seeing? I will describe some of our recent insights into understanding object recognition in the brain using behavioral, neuronal and computational methods.

Molecular recognition and the assembly of feature-selective retinal circuits

NMC4 Short Talk: Novel population of synchronously active pyramidal cells in hippocampal area CA1

Hippocampal pyramidal cells have been widely studied during locomotion, when theta oscillations are present, and during short wave ripples at rest, when replay takes place. However, we find a subset of pyramidal cells that are preferably active during rest, in the absence of theta oscillations and short wave ripples. We recorded these cells using two-photon imaging in dorsal CA1 of the hippocampus of mice, during a virtual reality object location recognition task. During locomotion, the cells show a similar level of activity as control cells, but their activity increases during rest, when this population of cells shows highly synchronous, oscillatory activity at a low frequency (0.1-0.4 Hz). In addition, during both locomotion and rest these cells show place coding, suggesting they may play a role in maintaining a representation of the current location, even when the animal is not moving. We performed simultaneous electrophysiological and calcium recordings, which showed a higher correlation of activity between the LFO and the hippocampal cells in the 0.1-0.4 Hz low frequency band during rest than during locomotion. However, the relationship between the LFO and calcium signals varied between electrodes, suggesting a localized effect. We used the Allen Brain Observatory Neuropixels Visual Coding dataset to further explore this. These data revealed localised low frequency oscillations in CA1 and DG during rest. Overall, we show a novel population of hippocampal cells, and a novel oscillatory band of activity in hippocampus during rest.

NMC4 Short Talk: Directly interfacing brain and deep networks exposes non-hierarchical visual processing

A recent approach to understanding the mammalian visual system is to show correspondence between the sequential stages of processing in the ventral stream with layers in a deep convolutional neural network (DCNN), providing evidence that visual information is processed hierarchically, with successive stages containing ever higher-level information. However, correspondence is usually defined as shared variance between brain region and model layer. We propose that task-relevant variance is a stricter test: If a DCNN layer corresponds to a brain region, then substituting the model’s activity with brain activity should successfully drive the model’s object recognition decision. Using this approach on three datasets (human fMRI and macaque neuron firing rates) we found that in contrast to the hierarchical view, all ventral stream regions corresponded best to later model layers. That is, all regions contain high-level information about object category. We hypothesised that this is due to recurrent connections propagating high-level visual information from later regions back to early regions, in contrast to the exclusively feed-forward connectivity of DCNNs. Using task-relevant correspondence with a late DCNN layer akin to a tracer, we used Granger causal modelling to show late-DCNN correspondence in IT drives correspondence in V4. Our analysis suggests, effectively, that no ventral stream region can be appropriately characterised as ‘early’ beyond 70ms after stimulus presentation, challenging hierarchical models. More broadly, we ask what it means for a model component and brain region to correspond: beyond quantifying shared variance, we must consider the functional role in the computation. We also demonstrate that using a DCNN to decode high-level conceptual information from ventral stream produces a general mapping from brain to model activation space, which generalises to novel classes held-out from training data. This suggests future possibilities for brain-machine interface with high-level conceptual information, beyond current designs that interface with the sensorimotor periphery.

Aesthetic preference for art can be predicted from a mixture of low- and high-level visual features

It is an open question whether preferences for visual art can be lawfully predicted from the basic constituent elements of a visual image. Here, we developed and tested a computational framework to investigate how aesthetic values are formed. We show that it is possible to explain human preferences for a visual art piece based on a mixture of low- and high-level features of the image. Subjective value ratings could be predicted not only within but also across individuals, using a regression model with a common set of interpretable features. We also show that the features predicting aesthetic preference can emerge hierarchically within a deep convolutional neural network trained only for object recognition. Our findings suggest that human preferences for art can be explained at least in part as a systematic integration over the underlying visual features of an image.

What neural oscillations can(not) do for syntactic structure building

The question of how syntactic structure can be built at the neural level has come to the forefront of cognitive neuroscience in the last decade. Neural oscillations have been widely recognised as playing an important role in building syntactic representations. In this talk I will review existing oscillatory approaches to syntactic structure building and assess their functionality in light of basic properties of a hierarchical syntactic structure, such as varied length of syntactic phrases, nesting of constituents, overlap in length between different levels of the syntactic hierarchy and others. I will also briefly discuss key requirements on neural structure building mechanisms from the perspective of a real-time parser.

How do we find what we are looking for? The Guided Search 6.0 model

The talk will give a tour of Guided Search 6.0 (GS6), the latest evolution of the Guided Search model of visual search. Part 1 describes The Mechanics of Search. Because we cannot recognize more than a few items at a time, selective attention is used to prioritize items for processing. Selective attention to an item allows its features to be bound together into a representation that can be matched to a target template in memory or rejected as a distractor. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 msec/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid serial and parallel model. If a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Part 2 elaborates on the five sources of Guidance that are combined into a spatial “priority map” to guide the deployment of attention (hence “guided search”). These are (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g. priming), (4) reward, and (5) scene syntax and semantics. Finally, in Part 3, we will consider the internal representation of what we are searching for; what is often called “the search template”. That search template is really two templates: a guiding template (probably in working memory) and a target template (in long term memory). Put these pieces together and you have GS6.

Towards a Theory of Human Visual Reasoning

Many tasks that are easy for humans are difficult for machines. In particular, while humans excel at tasks that require generalising across problems, machine systems notably struggle. One such task that has received a good amount of attention is the Synthetic Visual Reasoning Test (SVRT). The SVRT consists of a range of problems where simple visual stimuli must be categorised into one of two categories based on an unknown rule that must be induced. Conventional machine learning approaches perform well only when trained to categorise based on a single rule and are unable to generalise without extensive additional training to tasks with any additional rules. Multiple theories of higher-level cognition posit that humans solve such tasks using structured relational representations. Specifically, people learn rules based on structured representations that generalise to novel instances quickly and easily. We believe it is possible to model this approach in a single system which learns all the required relational representations from scratch and performs tasks such as SVRT in a single run. Here, we present a system which expands the DORA/LISA architecture and augments the existing model with principally novel components, namely a) visual reasoning based on the established theories of recognition by components; b) the process of learning complex relational representations by synthesis (in addition to learning by analysis). The proposed augmented model matches human behaviour on SVRT problems. Moreover, the proposed system stands as perhaps a more realistic account of human cognition, wherein rather than using tools that has been shown successful in the machine learning field to inform psychological theorising, we use established psychological theories to inform developing a machine system.

Encoding and perceiving the texture of sounds: auditory midbrain codes for recognizing and categorizing auditory texture and for listening in noise

Natural soundscapes such as from a forest, a busy restaurant, or a busy intersection are generally composed of a cacophony of sounds that the brain needs to interpret either independently or collectively. In certain instances sounds - such as from moving cars, sirens, and people talking - are perceived in unison and are recognized collectively as single sound (e.g., city noise). In other instances, such as for the cocktail party problem, multiple sounds compete for attention so that the surrounding background noise (e.g., speech babble) interferes with the perception of a single sound source (e.g., a single talker). I will describe results from my lab on the perception and neural representation of auditory textures. Textures, such as a from a babbling brook, restaurant noise, or speech babble are stationary sounds consisting of multiple independent sound sources that can be quantitatively defined by summary statistics of an auditory model (McDermott & Simoncelli 2011). How and where in the auditory system are summary statistics represented and the neural codes that potentially contribute towards their perception, however, are largely unknown. Using high-density multi-channel recordings from the auditory midbrain of unanesthetized rabbits and complementary perceptual studies on human listeners, I will first describe neural and perceptual strategies for encoding and perceiving auditory textures. I will demonstrate how distinct statistics of sounds, including the sound spectrum and high-order statistics related to the temporal and spectral correlation structure of sounds, contribute to texture perception and are reflected in neural activity. Using decoding methods I will then demonstrate how various low and high-order neural response statistics can differentially contribute towards a variety of auditory tasks including texture recognition, discrimination, and categorization. Finally, I will show examples from our recent studies on how high-order sound statistics and accompanying neural activity underlie difficulties for recognizing speech in background noise.

Cortex-wide high density ECoG recordings from rat reveal diverse generators of sleep-spindles with characteristic anatomical topographies and non-stationary subcycle dynamics

Bernstein Conference 2024

Non-Human Recognition of Orthography: How is it implemented and how does it differ from Human orthographic processing

Bernstein Conference 2024

Recognizing relevant information in neural activity

Bernstein Conference 2024

Action recognition best explains neural activity in cuneate nucleus

COSYNE 2022

Do better object recognition models improve the generalization gap in neural predictivity?

COSYNE 2022

Linking neural dynamics across macaque V4, IT, and PFC to trial-by-trial object recognition behavior

COSYNE 2022

Linking neural dynamics across macaque V4, IT, and PFC to trial-by-trial object recognition behavior

COSYNE 2022

How many objects can be recognized under all possible views?

COSYNE 2022

How many objects can be recognized under all possible views?

COSYNE 2022

Distinct roles of excitatory and inhibitory neurons in the macaque IT cortex in object recognition

COSYNE 2023

Leveraging computational and animal models of vision to probe atypical emotion recognition in autism

COSYNE 2023

On-line SEUDO for real-time cell recognition in Calcium Imaging

COSYNE 2023

Spatial-frequency channels for object recognition by neural networks are twice as wide as those of humans

COSYNE 2023

Temporal pattern recognition in retinal ganglion cells is mediated by dynamical inhibitory synapses

COSYNE 2023

Geometric Signatures of Speech Recognition: Insights from Deep Neural Networks to the Brain

COSYNE 2025

The analysis of the OXT-DA interaction causing social recognition deficit in Syntaxin1A KO

FENS Forum 2024

Behavioral impacts of simulated microgravity on male mice: Locomotion, social interactions and memory in a novel object recognition task

FENS Forum 2024

Cortex-wide high-density ECoG and translaminar local field potential recordings reveal rich broad-band spatio-temporal dynamics

FENS Forum 2024

The cortical amygdala mediates individual recognition in mice

FENS Forum 2024

A deep learning approach for the recognition of behaviors in the forced swim test

FENS Forum 2024

Direct electrical stimulation of the human amygdala enhances recognition memory for objects but not scenes

FENS Forum 2024

Two distinct ways to form long-term object recognition memory during sleep and wakefulness

FENS Forum 2024

Early disruption in social recognition and its impact on episodic memory in triple transgenic mice model of Alzheimer’s disease

FENS Forum 2024

ECoG-based functional mapping of the motor cortex in rhesus monkeys

FENS Forum 2024

Evaluation of novel object recognition test results of rats injected with intracerebroventricular streptozocin to develop Alzheimer's disease models

FENS Forum 2024

GPT-4 can recognize Theory of Mind in natural conversations: fMRI evidence

FENS Forum 2024

HBK-15 rescues recognition memory in MK-801- and stress-induced cognitive impairments in female mice

FENS Forum 2024

Homecage-based unsupervised novel object recognition in mice

FENS Forum 2024

Interaction of sex and sleep on performance at the novel object recognition task in mice

FENS Forum 2024

Investigating the recruitment of parvalbumin and somatostatin interneurons into engrams for associative recognition memory

FENS Forum 2024

Mouse can recognize other individuals: Maternal exposure to dioxin does not affect identification but perturbs the recognition ability of other individuals

FENS Forum 2024

Myoelectric gesture recognition in patients with spinal cord injury using a medium-density EMG system

FENS Forum 2024

Noradrenergic modulation of recognition memory in male and female mice

FENS Forum 2024

The processing of spatial frequencies through time in visual word recognition

FENS Forum 2024

Recognition of complex spatial environments showed dimorphic patterns of theta (4-8 Hz) activity

FENS Forum 2024

Resonant song recognition in crickets

FENS Forum 2024

Robustness and evolvability in a model of a pattern recognition network

FENS Forum 2024

Scent of a memory: Dissecting the vomeronasal-hippocampal axis in social recognition

FENS Forum 2024

Sex-dependent effects of voluntary physical exercise on object recognition memory restoration after traumatic brain injury in middle-aged rats

FENS Forum 2024

Sleepless nights, vanishing faces: The effect of sleep deprivation on long-term social recognition memory in mice

FENS Forum 2024