Imitation
imitation
The SIMple microscope: Development of a fibre-based platform for accessible SIM imaging in unconventional environments
Advancements in imaging speed, depth and resolution have made structured illumination microscopy (SIM) an increasingly powerful optical sectioning (OS) and super-resolution (SR) technique, but these developments remain inaccessible to many life science researchers due to the cost, optical complexity and delicacy of these instruments. We address these limitations by redesigning the optical path using in-line fibre components that are compact, lightweight and easily assembled in a “Plug & Play” modality, without compromising imaging performance. They can be integrated into an existing widefield microscope with a minimum of optical components and alignment, making OS-SIM more accessible to researchers with less optics experience. We also demonstrate a complete SR-SIM imaging system with dimensions 300 mm × 300 mm × 450 mm. We propose to enable accessible SIM imaging by utilising its compact, lightweight and robust design to transport it where it is needed, and image in “unconventional” environments where factors such as temperature and biosafety considerations currently limit imaging experiments.
FLUXSynID: High-Resolution Synthetic Face Generation for Document and Live Capture Images
Synthetic face datasets are increasingly used to overcome the limitations of real-world biometric data, including privacy concerns, demographic imbalance, and high collection costs. However, many existing methods lack fine-grained control over identity attributes and fail to produce paired, identity-consistent images under structured capture conditions. In this talk, I will present FLUXSynID, a framework for generating high-resolution synthetic face datasets with user-defined identity attribute distributions and paired document-style and trusted live capture images. The dataset generated using FLUXSynID shows improved alignment with real-world identity distributions and greater diversity compared to prior work. I will also discuss how FLUXSynID’s dataset and generation tools can support research in face recognition and morphing attack detection (MAD), enhancing model robustness in both academic and practical applications.
An Ecological and Objective Neural Marker of Implicit Unfamiliar Identity Recognition
We developed a novel paradigm measuring implicit identity recognition using Fast Periodic Visual Stimulation (FPVS) with EEG among 16 students and 12 police officers with normal face processing abilities. Participants' neural responses to a 1-Hz tagged oddball identity embedded within a 6-Hz image stream revealed implicit recognition with high-quality mugshots but not CCTV-like images, suggesting optimal resolution requirements. Our findings extend previous research by demonstrating that even unfamiliar identities can elicit robust neural recognition signatures through brief, repeated passive exposure. This approach offers potential for objective validation of face processing abilities in forensic applications, including assessment of facial examiners, Super-Recognisers, and eyewitnesses, potentially overcoming limitations of traditional behavioral assessment methods.
Pharmacological exploitation of neurotrophins and their receptors to develop novel therapeutic approaches against neurodegenerative diseases and brain trauma
Neurotrophins (NGF, BDNF, NT-3) are endogenous growth factors that exert neuroprotective effects by preventing neuronal death and promoting neurogenesis. They act by binding to their respective high-affinity, pro-survival receptors TrkA, TrkB or TrkC, as well as to p75NTR death receptor. While these molecules have been shown to significantly slow or prevent neurodegeneration, their reduced bioavailability and inability to penetrate the blood-brain-barrier limit their use as potential therapeutics. To bypass these limitations, our research team has developed and patented small-sized, lipophilic compounds which selectively resemble neurotrophins’ effects, presenting preferable pharmacological properties and promoting neuroprotection and repair against neurodegeneration. In addition, the combination of these molecules with 3D cultured human neuronal cells, and their targeted delivery in the brain ventricles through soft robotic systems, could offer novel therapeutic approaches against neurodegenerative diseases and brain trauma.
“Open Raman Microscopy (ORM): A modular Raman spectroscopy setup with an open-source controller”
Raman spectroscopy is a powerful technique for identifying chemical species by probing their vibrational energy levels, offering exceptional specificity with a relatively simple setup involving a laser source, spectrometer, and microscope/probe. However, the high cost of Raman systems lacking modularity often limits exploratory research hindering broader adoption. To address the need for an affordable, modular microscopy platform for multimodal imaging, we present a customizable confocal Raman spectroscopy setup alongside an open-source acquisition software, ORM (Open Raman Microscopy) Controller, developed in Python. This solution bridges the gap between expensive commercial systems and complex, custom-built setups used by specialist research groups. In this presentation, we will cover the components of the setup, the design rationale, assembly methods, limitations, and its modular potential for expanding functionality. Additionally, we will demonstrate ORM’s capabilities for instrument control, 2D and 3D Raman mapping, region-of-interest selection, and its adaptability to various instrument configurations. We will conclude by showcasing practical applications of this setup across different research fields.
Localisation of Seizure Onset Zone in Epilepsy Using Time Series Analysis of Intracranial Data
There are over 30 million people with drug-resistant epilepsy worldwide. When neuroimaging and non-invasive neural recordings fail to localise seizure onset zones (SOZ), intracranial recordings become the best chance for localisation and seizure-freedom in those patients. However, intracranial neural activities remain hard to visually discriminate across recording channels, which limits the success of intracranial visual investigations. In this presentation, I present methods which quantify intracranial neural time series and combine them with explainable machine learning algorithms to localise the SOZ in the epileptic brain. I present the potentials and limitations of our methods in the localisation of SOZ in epilepsy providing insights for future research in this area.
Trackoscope: A low-cost, open, autonomous tracking microscope for long-term observations of microscale organisms
Cells and microorganisms are motile, yet the stationary nature of conventional microscopes impedes comprehensive, long-term behavioral and biomechanical analysis. The limitations are twofold: a narrow focus permits high-resolution imaging but sacrifices the broader context of organism behavior, while a wider focus compromises microscopic detail. This trade-off is especially problematic when investigating rapidly motile ciliates, which often have to be confined to small volumes between coverslips affecting their natural behavior. To address this challenge, we introduce Trackoscope, an 2-axis autonomous tracking microscope designed to follow swimming organisms ranging from 10μm to 2mm across a 325 square centimeter area for extended durations—ranging from hours to days—at high resolution. Utilizing Trackoscope, we captured a diverse array of behaviors, from the air-water swimming locomotion of Amoeba to bacterial hunting dynamics in Actinosphaerium, walking gait in Tardigrada, and binary fission in motile Blepharisma. Trackoscope is a cost-effective solution well-suited for diverse settings, from high school labs to resource-constrained research environments. Its capability to capture diverse behaviors in larger, more realistic ecosystems extends our understanding of the physics of living systems. The low-cost, open architecture democratizes scientific discovery, offering a dynamic window into the lives of previously inaccessible small aquatic organisms.
Generative models for video games (rescheduled)
Developing agents capable of modeling complex environments and human behaviors within them is a key goal of artificial intelligence research. Progress towards this goal has exciting potential for applications in video games, from new tools that empower game developers to realize new creative visions, to enabling new kinds of immersive player experiences. This talk focuses on recent advances of my team at Microsoft Research towards scalable machine learning architectures that effectively capture human gameplay data. In the first part of my talk, I will focus on diffusion models as generative models of human behavior. Previously shown to have impressive image generation capabilities, I present insights that unlock applications to imitation learning for sequential decision making. In the second part of my talk, I discuss a recent project taking ideas from language modeling to build a generative sequence model of an Xbox game.
Generative models for video games
Developing agents capable of modeling complex environments and human behaviors within them is a key goal of artificial intelligence research. Progress towards this goal has exciting potential for applications in video games, from new tools that empower game developers to realize new creative visions, to enabling new kinds of immersive player experiences. This talk focuses on recent advances of my team at Microsoft Research towards scalable machine learning architectures that effectively capture human gameplay data. In the first part of my talk, I will focus on diffusion models as generative models of human behavior. Previously shown to have impressive image generation capabilities, I present insights that unlock applications to imitation learning for sequential decision making. In the second part of my talk, I discuss a recent project taking ideas from language modeling to build a generative sequence model of an Xbox game.
Are integrative, multidisciplinary, and pragmatic models possible? The #PsychMapping experience
This presentation delves into the necessity for simplified models in the field of psychological sciences to cater to a diverse audience of practitioners. We introduce the #PsychMapping model, evaluate its merits and limitations, and discuss its place in contemporary scientific culture. The #PsychMapping model is the product of an extensive literature review, initially within the realm of sport and exercise psychology and subsequently encompassing a broader spectrum of psychological sciences. This model synthesizes the progress made in psychological sciences by categorizing variables into a framework that distinguishes between traits (e.g., body structure and personality) and states (e.g., heart rate and emotions). Furthermore, it delineates internal traits and states from the externalized self, which encompasses behaviour and performance. All three components—traits, states, and the externalized self—are in a continuous interplay with external physical, social, and circumstantial factors. Two core processes elucidate the interactions among these four primary clusters: external perception, encompassing the mechanism through which external stimuli transition into internal events, and self-regulation, which empowers individuals to become autonomous agents capable of exerting control over themselves and their actions. While the model inherently oversimplifies intricate processes, the central question remains: does its pragmatic utility outweigh its limitations, and can it serve as a valuable tool for comprehending human behaviour?
Are integrative, multidisciplinary, and pragmatic models possible? The #PsychMapping experience
This presentation delves into the necessity for simplified models in the field of psychological sciences to cater to a diverse audience of practitioners. We introduce the #PsychMapping model, evaluate its merits and limitations, and discuss its place in contemporary scientific culture. The #PsychMapping model is the product of an extensive literature review, initially within the realm of sport and exercise psychology and subsequently encompassing a broader spectrum of psychological sciences. This model synthesizes the progress made in psychological sciences by categorizing variables into a framework that distinguishes between traits (e.g., body structure and personality) and states (e.g., heart rate and emotions). Furthermore, it delineates internal traits and states from the externalized self, which encompasses behaviour and performance. All three components—traits, states, and the externalized self—are in a continuous interplay with external physical, social, and circumstantial factors. Two core processes elucidate the interactions among these four primary clusters: external perception, encompassing the mechanism through which external stimuli transition into internal events, and self-regulation, which empowers individuals to become autonomous agents capable of exerting control over themselves and their actions. While the model inherently oversimplifies intricate processes, the central question remains: does its pragmatic utility outweigh its limitations, and can it serve as a valuable tool for comprehending human behaviour?
Attending to the ups and downs of Lewy body dementia: An exploration of cognitive fluctuations
Dementia with Lewy bodies (DLB) and Parkinson's disease dementia (PDD) share similarities in pathology and clinical presentation and come under the umbrella term of Lewy body dementias (LBD). Fluctuating cognition is a key symptom in LBD and manifests as altered levels of alertness and attention, with a marked difference between best and worst performance. Cognition and alertness can change over seconds or minutes to hours and days of obtundation. Cognitive fluctuations can have significant impacts on the quality of life of people with LBD as well as potentially contribute to the exacerbation of other transient symptoms including, for example, hallucinations and psychosis as well as making it difficult to measure cognitive effect size benefits in clinical trials of LBD. However, this significant symptom in LBD is poorly understood. In my presentation I will discuss the phenomenology of cognitive fluctuations, how we can measure it clinically and limitations of these approaches. I will then outline the work of our group and others which has been focussed on unpicking the aetiological basis of cognitive fluctuations in LBD using a variety of imaging approaches (e.g. SPECT, sMRI, fMRI and EEG). I will then briefly explore future research directions.
Why robots? A brief introduction to the use of robots in psychological research
Why should psychologists be interested in robots? This talk aims to illustrate how social robots – machines with human-like features and behaviors – can offer interesting insights into the human mind. I will first provide a brief overview of how robots have been used in psychology and cognitive science research focusing on two approaches - Developmental Robotics and Human-Robot Interaction (HRI). We will then delve into recent works in HRI, including my own, in greater detail. We will also address the limitations of research thus far, such as the lack of proper controlled experiments, and discuss how the scientific community should evaluate the use of technology in educational and other social settings.
Dissociating learning-induced effects of meaning and familiarity in visual working memory for Chinese characters
Visual working memory (VWM) is limited in capacity, but memorizing meaningful objects may refine this limitation. However, meaningless and meaningful stimuli usually differ perceptually and an object’s association with meaning is typically already established before the actual experiment. We applied a strict control over these potential confounds by asking observers (N=45) to actively learn associations of (initially) meaningless objects. To this end, a change detection task presented Chinese characters, which were meaningless to our observers. Subsequently, half of the characters were consistently paired with pictures of animals. Then, the initial change detection task was repeated. The results revealed enhanced VWM performance after learning, in particular for meaning-associated characters (though not quite reaching the accuracy level attained by N=20 native Chinese observers). These results thus provide direct experimental evidence that the short-term retention of objects benefits from active learning of an object’s association with meaning in long-term memory.
Spatially-embedded recurrent neural networks reveal widespread links between structural and functional neuroscience findings
Brain networks exist within the confines of resource limitations. As a result, a brain network must overcome metabolic costs of growing and sustaining the network within its physical space, while simultaneously implementing its required information processing. To observe the effect of these processes, we introduce the spatially-embedded recurrent neural network (seRNN). seRNNs learn basic task-related inferences while existing within a 3D Euclidean space, where the communication of constituent neurons is constrained by a sparse connectome. We find that seRNNs, similar to primate cerebral cortices, naturally converge on solving inferences using modular small-world networks, in which functionally similar units spatially configure themselves to utilize an energetically-efficient mixed-selective code. As all these features emerge in unison, seRNNs reveal how many common structural and functional brain motifs are strongly intertwined and can be attributed to basic biological optimization processes. seRNNs can serve as model systems to bridge between structural and functional research communities to move neuroscientific understanding forward.
Meta-learning functional plasticity rules in neural networks
Synaptic plasticity is known to be a key player in the brain’s life-long learning abilities. However, due to experimental limitations, the nature of the local changes at individual synapses and their link with emerging network-level computations remain unclear. I will present a numerical, meta-learning approach to deduce plasticity rules from either neuronal activity data and/or prior knowledge about the network's computation. I will first show how to recover known rules, given a human-designed loss function in rate networks, or directly from data, using an adversarial approach. Then I will present how to scale-up this approach to recurrent spiking networks using simulation-based inference.
Lifelong Learning AI via neuro inspired solutions
AI embedded in real systems, such as in satellites, robots and other autonomous devices, must make fast, safe decisions even when the environment changes, or under limitations on the available power; to do so, such systems must be adaptive in real time. To date, edge computing has no real adaptivity – rather the AI must be trained in advance, typically on a large dataset with much computational power needed; once fielded, the AI is frozen: It is unable to use its experience to operate if environment proves outside its training or to improve its expertise; and worse, since datasets cannot cover all possible real-world situations, systems with such frozen intelligent control are likely to fail. Lifelong Learning is the cutting edge of artificial intelligence - encompassing computational methods that allow systems to learn in runtime and incorporate learning for application in new, unanticipated situations. Until recently, this sort of computation has been found exclusively in nature; thus, Lifelong Learning looks to nature, and in particular neuroscience, for its underlying principles and mechanisms and then translates them to this new technology. Our presentation will introduce a number of state-of-the-art approaches to achieve AI adaptive learning, including from the DARPA’s L2M program and subsequent developments. Many environments are affected by temporal changes, such as the time of day, week, season, etc. A way to create adaptive systems which are both small and robust is by making them aware of time and able to comprehend temporal patterns in the environment. We will describe our current research in temporal AI, while also considering power constraints.
Learning Relational Rules from Rewards
Humans perceive the world in terms of objects and relations between them. In fact, for any given pair of objects, there is a myriad of relations that apply to them. How does the cognitive system learn which relations are useful to characterize the task at hand? And how can it use these representations to build a relational policy to interact effectively with the environment? In this paper we propose that this problem can be understood through the lens of a sub-field of symbolic machine learning called relational reinforcement learning (RRL). To demonstrate the potential of our approach, we build a simple model of relational policy learning based on a function approximator developed in RRL. We trained and tested our model in three Atari games that required to consider an increasingly number of potential relations: Breakout, Pong and Demon Attack. In each game, our model was able to select adequate relational representations and build a relational policy incrementally. We discuss the relationship between our model with models of relational and analogical reasoning, as well as its limitations and future directions of research.
Multi-level theory of neural representations in the era of large-scale neural recordings: Task-efficiency, representation geometry, and single neuron properties
A central goal in neuroscience is to understand how orchestrated computations in the brain arise from the properties of single neurons and networks of such neurons. Answering this question requires theoretical advances that shine light into the ‘black box’ of representations in neural circuits. In this talk, we will demonstrate theoretical approaches that help describe how cognitive and behavioral task implementations emerge from the structure in neural populations and from biologically plausible neural networks. First, we will introduce an analytic theory that connects geometric structures that arise from neural responses (i.e., neural manifolds) to the neural population’s efficiency in implementing a task. In particular, this theory describes a perceptron’s capacity for linearly classifying object categories based on the underlying neural manifolds’ structural properties. Next, we will describe how such methods can, in fact, open the ‘black box’ of distributed neuronal circuits in a range of experimental neural datasets. In particular, our method overcomes the limitations of traditional dimensionality reduction techniques, as it operates directly on the high-dimensional representations, rather than relying on low-dimensionality assumptions for visualization. Furthermore, this method allows for simultaneous multi-level analysis, by measuring geometric properties in neural population data, and estimating the amount of task information embedded in the same population. These geometric frameworks are general and can be used across different brain areas and task modalities, as demonstrated in the work of ours and others, ranging from the visual cortex to parietal cortex to hippocampus, and from calcium imaging to electrophysiology to fMRI datasets. Finally, we will discuss our recent efforts to fully extend this multi-level description of neural populations, by (1) investigating how single neuron properties shape the representation geometry in early sensory areas, and by (2) understanding how task-efficient neural manifolds emerge in biologically-constrained neural networks. By extending our mathematical toolkit for analyzing representations underlying complex neuronal networks, we hope to contribute to the long-term challenge of understanding the neuronal basis of tasks and behaviors.
Computational Imaging: Augmenting Optics with Algorithms for Biomedical Microscopy and Neural Imaging
Computational imaging seeks to achieve novel capabilities and overcome conventional limitations by combining optics and algorithms. In this seminar, I will discuss two computational imaging technologies developed in Boston University Computational Imaging Systems lab, including Intensity Diffraction Tomography and Computational Miniature Mesoscope. In our intensity diffraction tomography system, we demonstrate 3D quantitative phase imaging on a simple LED array microscope. We develop both single-scattering and multiple-scattering models to image complex biological samples. In our Computational Miniature Mesoscope, we demonstrate single-shot 3D high-resolution fluorescence imaging across a wide field-of-view in a miniaturized platform. We develop methods to characterize 3D spatially varying aberrations and physical simulator-based deep learning strategies to achieve fast and accurate reconstructions. Broadly, I will discuss how synergies between novel optical instrumentation, physical modeling, and model- and learning-based computational algorithms can push the limits in biomedical microscopy and neural imaging.
A Framework for a Conscious AI: Viewing Consciousness through a Theoretical Computer Science Lens
We examine consciousness from the perspective of theoretical computer science (TCS), a branch of mathematics concerned with understanding the underlying principles of computation and complexity, including the implications and surprising consequences of resource limitations. We propose a formal TCS model, the Conscious Turing Machine (CTM). The CTM is influenced by Alan Turing's simple yet powerful model of computation, the Turing machine (TM), and by the global workspace theory (GWT) of consciousness originated by cognitive neuroscientist Bernard Baars and further developed by him, Stanislas Dehaene, Jean-Pierre Changeux, George Mashour, and others. However, the CTM is not a standard Turing Machine. It’s not the input-output map that gives the CTM its feeling of consciousness, but what’s under the hood. Nor is the CTM a standard GW model. In addition to its architecture, what gives the CTM its feeling of consciousness is its predictive dynamics (cycles of prediction, feedback and learning), its internal multi-modal language Brainish, and certain special Long Term Memory (LTM) processors, including its Inner Speech and Model of the World processors. Phenomena generally associated with consciousness, such as blindsight, inattentional blindness, change blindness, dream creation, and free will, are considered. Explanations derived from the model draw confirmation from consistencies at a high level, well above the level of neurons, with the cognitive neuroscience literature. Reference. L. Blum and M. Blum, "A theory of consciousness from a theoretical computer science perspective: Insights from the Conscious Turing Machine," PNAS, vol. 119, no. 21, 24 May 2022. https://www.pnas.org/doi/epdf/10.1073/pnas.2115934119
PET imaging in brain diseases
Talk 1. PET based biomarkers of treatment efficacy in temporal lobe epilepsy A critical aspect of drug development involves identifying robust biomarkers of treatment response for use as surrogate endpoints in clinical trials. However, these biomarkers also have the capacity to inform mechanisms of disease pathogenesis and therapeutic efficacy. In this webinar, Dr Bianca Jupp will report on a series of studies using the GABAA PET ligand, [18F]-Flumazenil, to establish biomarkers of treatment response to a novel therapeutic for temporal lobe epilepsy, identifying affinity at this receptor as a key predictor of treatment outcome. Dr Bianca Jupp is a Research Fellow in the Department of Neuroscience, Monash University and Lead PET/CT Scientist at the Alfred Research Alliance–Monash Biomedical Imaging facility. Her research focuses on neuroimaging and its capacity to inform the neurobiology underlying neurological and neuropsychiatric disorders. Talk 2. The development of a PET radiotracer for reparative microglia Imaging of neuroinflammation is currently hindered by the technical limitations associated with TSPO imaging. In this webinar, Dr Lucy Vivash will discuss the development of PET radiotracers that specifically image reparative microglia through targeting the receptor kinase MerTK. This includes medicinal chemistry design and testing, radiochemistry, and in vitro and in vivo testing of lead tracers. Dr Lucy Vivash is a Research Fellow in the Department of Neuroscience, Monash University. Her research focuses on the preclinical development and clinical translation of novel PET radiotracers for the imaging of neurodegenerative diseases.
Learning from others, helping others learn: Cognitive foundations of distinctively human social learning
Learning does not occur in isolation. From parent-child interactions to formal classroom environments, humans explore, learn, and communicate in rich, diverse social contexts. Rather than simply observing and copying their conspecifics, humans engage in a range of epistemic practices that actively recruit those around them. What makes human social learning so distinctive, powerful, and smart? In this talk, I will present a series of studies that reveal the remarkably sophisticated inferential abilities that young children show not only in how they learn from others but also in how they help others learn. Children interact with others as learners and as teachers to learn and communicate about the world, about others, and even about the self. The results collectively paint a picture of human social learning that is far more than copying and imitation: It is active, bidirectional, and cooperative. I will end by discussing ongoing work that extends this picture beyond what we typically call “social learning”, with implications for building better machines that learn from and interact with humans.
Neural circuits of visuospatial working memory
One elementary brain function that underlies many of our cognitive behaviors is the ability to maintain parametric information briefly in mind, in the time scale of seconds, to span delays between sensory information and actions. This component of working memory is fragile and quickly degrades with delay length. Under the assumption that behavioral delay-dependencies mark core functions of the working memory system, our goal is to find a neural circuit model that represents their neural mechanisms and apply it to research on working memory deficits in neuropsychiatric disorders. We have constrained computational models of spatial working memory with delay-dependent behavioral effects and with neural recordings in the prefrontal cortex during visuospatial working memory. I will show that a simple bump attractor model with weak inhomogeneities and short-term plasticity mechanisms can link neural data with fine-grained behavioral output in a trial-by-trial basis and account for the main delay-dependent limitations of working memory: precision, cardinal repulsion biases and serial dependence. I will finally present data from participants with neuropsychiatric disorders that suggest that serial dependence in working memory is specifically altered, and I will use the model to infer the possible neural mechanisms affected.
Probabilistic computation in natural vision
A central goal of vision science is to understand the principles underlying the perception and neural coding of the complex visual environment of our everyday experience. In the visual cortex, foundational work with artificial stimuli, and more recent work combining natural images and deep convolutional neural networks, have revealed much about the tuning of cortical neurons to specific image features. However, a major limitation of this existing work is its focus on single-neuron response strength to isolated images. First, during natural vision, the inputs to cortical neurons are not isolated but rather embedded in a rich spatial and temporal context. Second, the full structure of population activity—including the substantial trial-to-trial variability that is shared among neurons—determines encoded information and, ultimately, perception. In the first part of this talk, I will argue for a normative approach to study encoding of natural images in primary visual cortex (V1), which combines a detailed understanding of the sensory inputs with a theory of how those inputs should be represented. Specifically, we hypothesize that V1 response structure serves to approximate a probabilistic representation optimized to the statistics of natural visual inputs, and that contextual modulation is an integral aspect of achieving this goal. I will present a concrete computational framework that instantiates this hypothesis, and data recorded using multielectrode arrays in macaque V1 to test its predictions. In the second part, I will discuss how we are leveraging this framework to develop deep probabilistic algorithms for natural image and video segmentation.
What is Cognitive Neuropsychology Good For? An Unauthorized Biography
Abstract: There is no doubt that the study of brain damaged individuals has contributed greatly to our understanding of the mind/brain. Within this broad approach, cognitive neuropsychology accentuates the cognitive dimension: it investigates the structure and organization of perceptual, motor, cognitive, and language systems – prerequisites for understanding the functional organization of the brain – through the analysis of their dysfunction following brain damage. Significant insights have come specifically from this paradigm. But progress has been slow and enthusiasm for this approach has waned somewhat in recent years, and the use of existing findings to constrain new theories has also waned. What explains the current diminished status of cognitive neuropsychology? One reason may be failure to calibrate expectations about the effective contribution of different subfields of the study of the mind/brain as these are determined by their natural peculiarities – such factors as the types of available observations and their complexity, opportunity of access to such observations, the possibility of controlled experimentation, and the like. Here, I also explore the merits and limitations of cognitive neuropsychology, with particular focus on the role of intellectual, pragmatic, and societal factors that determine scientific practice within the broader domains of cognitive science/neuroscience. I conclude on an optimistic note about the continuing unique importance of cognitive neuropsychology: although limited to the study of experiments of nature, it offers a privileged window into significant aspects of the mind/brain that are not easily accessible through other approaches. Biography: Alfonso Caramazza's research has focussed extensively on how words and their meanings are represented in the brain. His early pioneering studies helped to reformulate our thinking about Broca's aphasia (not limited to production) and formalised the logic of patient-based neuropsychology. More recently he has been instrumental in reconsidering popular claims about embodied cognition.
From natural scene statistics to multisensory integration: experiments, models and applications
To efficiently process sensory information, the brain relies on statistical regularities in the input. While generally improving the reliability of sensory estimates, this strategy also induces perceptual illusions that help reveal the underlying computational principles. Focusing on auditory and visual perception, in my talk I will describe how the brain exploits statistical regularities within and across the senses for the perception space, time and multisensory integration. In particular, I will show how results from a series of psychophysical experiments can be interpreted in the light of Bayesian Decision Theory, and I will demonstrate how such canonical computations can be implemented into simple and biologically plausible neural circuits. Finally, I will show how such principles of sensory information processing can be leveraged in virtual and augmented reality to overcome display limitations and expand human perception.
“Mind reading” with brain scanners: Facts versus science fiction
Every thought is associated with a unique pattern of brain activity. Thus, in principle, it should be possible to use these activity patterns as "brain fingerprints" for different thoughts and to read out what a person is thinking based on their brain activity alone. Indeed, using machine learning considerable progress has been made in such "brainreading" in recent years. It is now possible to decode which image a person is viewing, which film sequence they are watching, which emotional state they are in or which intentions they hold in mind. This talk will provide an overview of the current state of the art in brain reading. It will also highlight the main challenges and limitations of this research field. For example, mathematical models are needed to cope with the high dimensionality of potential mental states. Furthermore, the ethical concerns raised by (often premature) commercial applications of brain reading will also be discussed.
The bounded rationality of probability distortion
In decision-making under risk (DMR) participants' choices are based on probability values systematically different from those that are objectively correct. Similar systematic distortions are found in tasks involving relative frequency judgments (JRF). These distortions limit performance in a wide variety of tasks and an evident question is, why do we systematically fail in our use of probability and relative frequency information? We propose a Bounded Log-Odds Model (BLO) of probability and relative frequency distortion based on three assumptions: (1) log-odds: probability and relative frequency are mapped to an internal log-odds scale, (2) boundedness: the range of representations of probability and relative frequency are bounded and the bounds change dynamically with task, and (3) variance compensation: the mapping compensates in part for uncertainty in probability and relative frequency values. We compared human performance in both DMR and JRF tasks to the predictions of the BLO model as well as eleven alternative models each missing one or more of the underlying BLO assumptions (factorial model comparison). The BLO model and its assumptions proved to be superior to any of the alternatives. In a separate analysis, we found that BLO accounts for individual participants’ data better than any previous model in the DMR literature. We also found that, subject to the boundedness limitation, participants’ choice of distortion approximately maximized the mutual information between objective task-relevant values and internal values, a form of bounded rationality.
Qualitative Structure, Automorphism Groups and Private Language
It is generally agreed upon that qualities of conscious experience instantiate structural properties, usually called relations. They furnish a representation of qualities (or qualia, in fact) in terms of a mathematical space Q (rather than a set), which is crucial to both modelling and measuring of conscious experience." "What is usually disregarded is that “only such structural properties generalize across individuals” (Austen Clark), but that qualities themselves as differentiated by stimulus specifications, behavior or reports do not. We show that this implies that only the part of Q which is invariant with respect to the automorphism group has a well-defined referent, while individual elements do not. This poses a prima facie limitation of any theory or experiment that aims to address individual qualities. We show how mathematical theories of consciousness can overcome this limitation via symmetry groups and group actions, making accessible to science what is properly called private language.
Norse: A library for gradient-based learning in Spiking Neural Networks
We introduce Norse: An open-source library for gradient-based training of spiking neural networks. In contrast to neuron simulators which mainly target computational neuroscientists, our library seamlessly integrates with the existing PyTorch ecosystem using abstractions familiar to the machine learning community. This has immediate benefits in that it provides a familiar interface, hardware accelerator support and, most importantly, the ability to use gradient-based optimization. While many parallel efforts in this direction exist, Norse emphasizes flexibility and usability in three ways. Users can conveniently specify feed-forward (convolutional) architectures, as well as arbitrarily connected recurrent networks. We strictly adhere to a functional and class-based API such that neuron primitives and, for example, plasticity rules composes. Finally, the functional core API ensures compatibility with the PyTorch JIT and ONNX infrastructure. We have made progress to support network execution on the SpiNNaker platform and plan to support other neuromorphic architectures in the future. While the library is useful in its present state, it also has limitations we will address in ongoing work. In particular, we aim to implement event-based gradient computation, using the EventProp algorithm, which will allow us to support sparse event-based data efficiently, as well as work towards support of more complex neuron models. With this library, we hope to contribute to a joint future of computational neuroscience and neuromorphic computing.
Deriving local synaptic learning rules for efficient representations in networks of spiking neurons
How can neural networks learn to efficiently represent complex and high-dimensional inputs via local plasticity mechanisms? Classical models of representation learning assume that input weights are learned via pairwise Hebbian-like plasticity. Here, we show that pairwise Hebbian-like plasticity only works under specific requirements on neural dynamics and input statistics. To overcome these limitations, we derive from first principles a learning scheme based on voltage-dependent synaptic plasticity rules. Here, inhibition learns to locally balance excitatory input in individual dendritic compartments, and thereby can modulate excitatory synaptic plasticity to learn efficient representations. We demonstrate in simulations that this learning scheme works robustly even for complex, high-dimensional and correlated inputs. It also works in the presence of inhibitory transmission delays, where Hebbian-like plasticity typically fails. Our results draw a direct connection between dendritic excitatory-inhibitory balance and voltage-dependent synaptic plasticity as observed in vivo, and suggest that both are crucial for representation learning.
In vitro bioelectronic models of the gut-brain axis
The human gut microbiome has emerged as a key player in the bidirectional communication of the gut-brain axis, affecting various aspects of homeostasis and pathophysiology. Until recently, the majority of studies that seek to explore the mechanisms underlying the microbiome-gut-brain axis cross-talk relied almost exclusively on animal models, and particularly gnotobiotic mice. Despite the great progress made with these models, various limitations, including ethical considerations and interspecies differences that limit the translatability of data to human systems, pushed researchers to seek for alternatives. Over the past decades, the field of in vitro modelling of tissues has experienced tremendous growth, thanks to advances in 3D cell biology, materials, science and bioengineering, pushing further the borders of our ability to more faithfully emulate the in vivo situation. Organ-on-chip technology and bioengineered tissues have emerged as highly promising alternatives to animal models for a wide range of applications. In this talk I’ll discuss our progress towards generating a complete platform of the human microbiota-gut-brain axis with integrated monitoring and sensing capabilities. Bringing together principles of materials science, tissue engineering, 3D cell biology and bioelectronics, we are building advanced models of the GI and the BBB /NVU, with real-time and label-free monitoring units adapted in the model architecture, towards a robust and more physiologically relevant human in vitro model, aiming to i) elucidate the role of microbiota in the gut-brain axis communication, ii) to study how diet and impaired microbiota profiles affect various (patho-)physiologies, and iii) to test personalised medicine approaches for disease modelling and drug testing.
Categories, language, and visual working memory: how verbal labels change capacity limitations
The limited capacity of visual working memory constrains the quantity and quality of the information we can store in mind for ongoing processing. Research from our lab has demonstrated that verbal labeling/categorization of visual inputs increases its retention and fidelity in visual working memory. In this talk, I will outline the hypotheses that explain the interaction between visual and verbal inputs in working memory, leading to the boosts we observed. I will further show how manipulations of the categorical distinctiveness of the labels, the timing of their occurrence, to which item labels are applied, as well as their validity modulate the benefits one can draw from combining visual and verbal inputs to alleviate capacity limitations. Finally, I will discuss the implications of these results to our understanding of working memory and its interaction with prior knowledge.
Understanding Perceptual Priors with Massive Online Experiments
One of the most important questions in psychology and neuroscience is understanding how the outside world maps to internal representations. Classical psychophysics approaches to this problem have a number of limitations: they mostly study low dimensional perpetual spaces, and are constrained in the number and diversity of participants and experiments. As ecologically valid perception is rich, high dimensional, contextual, and culturally dependent, these impediments severely bias our understanding of perceptual representations. Recent technological advances—the emergence of so-called “Virtual Labs”— can significantly contribute toward overcoming these barriers. Here I present a number of specific strategies that my group has developed in order to probe representations across a number of dimensions. 1) Massive online experiments can increase significantly the amount of participants and experiments that can be carried out in a single study, while also significantly diversifying the participant pool. We have developed a platform, PsyNet, that enables “experiments as code,” whereby the orchestration of computer servers, recruiting, compensation of participants, and data management is fully automated and every experiment can be fully replicated with one command line. I will demonstrate how PsyNet allows us to recruit thousands of participants for each study with a large number of control experimental conditions, significantly increasing our understanding of auditory perception. 2) Virtual lab methods also enable us to run experiments that are nearly impossible in a traditional lab setting. I will demonstrate our development of adaptive sampling, a set of behavioural methods that combine machine learning sampling techniques (Monte Carlo Markov Chains) with human interactions and allow us to create high-dimensional maps of perceptual representations with unprecedented resolution. 3) Finally, I will demonstrate how the aforementioned methods can be applied to the study of perceptual priors in both audition and vision, with a focus on our work in cross-cultural research, which studies how perceptual priors are influenced by experience and culture in diverse samples of participants from around the world.
Multiphoton imaging with next-generation indicators
Two-photon (2P) in vivo functional imaging of genetically encoded fluorescent Ca2+indicators (GECIs) for neuronal activity has become a broadly applied standard tool in modern neuroscience, because it allows simultaneous imaging of the activity of many neurons at high spatial resolution within living animals. Unfortunately, the most commonly used light-sources – tunable femtosecond pulsed ti:sapphire lasers – can be prohibitively expensive for many labs and fall short of delivering sufficient powers for some new ultra-fast 2P microscopy modalities. Inexpensive homebuilt or industrial light sources such as Ytterbium fiber lasers (YbFLs) show great promise to overcome these limitations as they are becoming widely available at costs orders of magnitude lower and power outputs of up to many times higher than conventional ti:sapphire lasers. However, these lasers are typically bound to emitting a single wavelength (i.e., not tunable) centered around 1020-1060 nm, which fails to efficiently excite state of the art green GECIs such as jGCaMP7 or 8. To this end, we designed and characterized spectral variants (yellow CaMP = YCaMP) of the ultrasensitive genetically encoded calcium indicator jGCaMP7, that allows for efficient 2P-excitation at wavelengths above 1010nm. In this talk I will give a brief overview over some of the reasons why using a fiber laser for 2P excitation might be right for you. I will talk about the development of jYCaMP and some exciting new experimental avenues that it has opened while touching on the prospect that shifting biosensors yellow could have for the 2P imaging community. Please join me for an interesting and fun discussion on whether “yellow is the new green” after the talk!
Investigating visual recognition and the temporal lobes using electrophysiology and fast periodic visual stimulation
The ventral visual pathway extends from the occipital to the anterior temporal regions, and is specialized in giving meaning to objects and people that are perceived through vision. Numerous studies in functional magnetic resonance imaging have focused on the cerebral basis of visual recognition. However, this technique is susceptible to magnetic artefacts in ventral anterior temporal regions and it has led to an underestimation of the role of these regions within the ventral visual stream, especially with respect to face recognition and semantic representations. Moreover, there is an increasing need for implicit methods assessing these functions as explicit tasks lack specificity. In this talk, I will present three studies using fast periodic visual stimulation (FPVS) in combination with scalp and/or intracerebral EEG to overcome these limitations and provide high SNR in temporal regions. I will show that, beyond face recognition, FPVS can be extended to investigate semantic representations using a face-name association paradigm and a semantic categorisation paradigm with written words. These results shed new light on the role of temporal regions and demonstrate the high potential of the FPVS approach as a powerful electrophysiological tool to assess various cognitive functions in neurotypical and clinical populations.
SpikeInterface
Much development has been directed toward improving the performance and automation of spike sorting. This continuous development, while essential, has contributed to an over-saturation of new, incompatible tools that hinders rigorous benchmarking and complicates reproducible analysis. To address these limitations, we developed SpikeInterface, a Python framework designed to unify preexisting spike sorting technologies into a single codebase and to facilitate straightforward comparison and adoption of different approaches. With a few lines of code, researchers can reproducibly run, compare, and benchmark most modern spike sorting algorithms; pre-process, post-process, and visualize extracellular datasets; validate, curate, and export sorting outputs; and more. In this presentation, I will provide an overview of SpikeInterface and, with applications to real and simulated datasets, demonstrate how it can be utilized to reduce the burden of manual curation and to more comprehensively benchmark automated spike sorters.
Neural codes in early sensory areas maximize fitness
It has generally been presumed that sensory information encoded by a nervous system should be as accurate as its biological limitations allow. However, perhaps counter intuitively, accurate representations of sensory signals do not necessarily maximize the organism’s chances of survival. We show that neural codes that maximize reward expectation—and not accurate sensory representations—account for retinal responses in insects, and retinotopically-specific adaptive codes in humans. Thus, our results provide evidence that fitness-maximizing rules imposed by the environment are applied at the earliest stages of sensory processing.
Application of Airy beam light sheet microscopy to examine early neurodevelopmental structures in 3D hiPSC-derived human cortical spheroids
The inability to observe relevant biological processes in vivo significantly restricts human neurodevelopmental research. Advances in appropriate in vitro model systems, including patient-specific human brain organoids and human cortical spheroids (hCSs), offer a pragmatic solution to this issue. In particular, hCSs are an accessible method for generating homogenous organoids of dorsal telencephalic fate, which recapitulate key aspects of human corticogenesis, including the formation of neural rosettes—in vitro correlates of the neural tube. These neurogenic niches give rise to neural progenitors that subsequently differentiate into neurons. Studies differentiating induced pluripotent stem cells (hiPSCs) in 2D have linked atypical formation of neural rosettes with neurodevelopmental disorders such as autism spectrum conditions. Thus far, however, conventional methods of tissue preparation in this field limit the ability to image these structures in three-dimensions within intact hCS or other 3D preparations. To overcome this limitation, we have sought to optimise a methodological approach to process hCSs to maximise the utility of a novel Airy-beam light sheet microscope (ALSM) to acquire high resolution volumetric images of internal structures within hCS representative of early developmental time points.
Understanding and treating epilepsy in tuberous sclerosis complex
Tuberous sclerosis complex (TSC) and focal cortical dysplasia type II (FCDII) are caused by mutations in mTOR pathway genes leading to mTOR hyperactivity, focal malformations of cortical development (fMCD), and seizures in 80-90% of the patients. The current definitive treatments for epilepsy are surgical resection or treatment with everolimus, which inhibits mTOR activity (only approved for TSC). Because both options have severe limitations, there is a major need to better understand the mechanisms leading to seizures to improve life-long epilepsy treatment in TSC and FCDII. To investigate such mechanisms, we recently developed a murine model of fMCD-associated epilepsy that recapitulates the human TSC and FCDII disorders. fMCD are defined by the presence of misplaced, dysmorphic cortical neurons expressing hyperactive mTOR – for simplicity we will refer to these as “mutant” neurons. In our model and in human TSC tissue, we made a surprising finding that mutant neurons express HCN4 channels, which are not normally functionally expressed in cortical neurons, and increased levels of filamin A (FLNA). FLNA is an actin-crossing linking molecule that has also multiple binding partners inside cells. These data led us to ask several important questions: (1) As HCN4 channels are responsible for the pacemaking activity of the heart, can HCN4 channel expression lead to repetitive firing of mutant neurons resulting in seizures? (2) HCN4 is the most cAMP-sensitive of the four HCN isoforms. Does increase in cAMP lead to the firing of mutant neurons? (3) Does increase in FLNA contribute to neuronal alterations and seizures? (4) Is the abnormal HCN4 and FLNA expression in mutant neurons due to mTOR? These questions will be discussed and addressed in the lecture.
Exploring Memories of Scenes
State-of-the-art machine vision models can predict human recognition memory for complex scenes with astonishing accuracy. In this talk I present work that investigated how memorable scenes are actually remembered and experienced by human observers. We found that memorable scenes were recognized largely based on recollection of specific episodic details but also based on familiarity for an entire scene. I thus highlight current limitations in machine vision models emulating human recognition memory, with promising opportunities for future research. Moreover, we were interested in what observers specifically remember about complex scenes. We thus considered the functional role of eye-movements as a window into the content of memories, particularly when observers recollected specific information about a scene. We found that when observers formed a memory representation that they later recollected (compared to scenes that only felt familiar), the overall extent of exploration was broader, with a specific subset of fixations clustered around later to-be-recollected scene content, irrespective of the memorability of a scene. I discuss the critical role that our viewing behavior plays in visual memory formation and retrieval and point to potential implications for machine vision models predicting the content of human memories.
A developmental-cognitive perspective on the impact of adolescent social media use
Concerns about the impact of social media use on adolescent well-being and mental health are common. While the amount of research in this area has increased rapidly over the last 5 years, most outputs are still marred by a multitude of limitations. These shortcomings have left our understanding of social media effects severely limited, holding back both scientific discovery and policy interventions. This talk discusses how developmental, cognitive and neuroscientific approaches might provide a new and improved way of studying social media effects. It will detail new studies in support of this idea, and raise potential avenues for collaborative work across the Cambridge Neuroscience community. As the digital world now (re)shapes what it means for us to live, communicate and develop, only an interdisciplinary approach will allow us to truly understand its impacts.
Exploring fine detail: The interplay of attention, oculomotor behavior and visual perception in the fovea
Outside the foveola, visual acuity and other visual functions gradually deteriorate with increasing eccentricity. Humans compensate for these limitations by relying on a tight link between perception and action; rapid gaze shifts (saccades) occur 2-3 times every second, separating brief “fixation” intervals in which visual information is acquired and processed. During fixation, however, the eye is not immobile. Small eye movements incessantly shift the image on the retina even when the attended stimulus is already foveated, suggesting a much deeper coupling between visual functions and oculomotor activity. Thanks to a combination of techniques allowing for high-resolution recordings of eye position, retinal stabilization, and accurate gaze localization, we examined how attention and eye movements are controlled at this scale. We have shown that during fixation, visual exploration of fine spatial detail unfolds following visuomotor strategies similar to those occurring at a larger scale. This behavior compensates for non-homogenous visual capabilities within the foveola and is finely controlled by attention, which facilitates processing at selected foveal locations. Ultimately, the limits of high acuity vision are greatly influenced by the spatiotemporal modulations introduced by fixational eye movements. These findings reveal that, contrary to common intuition, placing a stimulus within the foveola is necessary but not sufficient for high visual acuity; fine spatial vision is the outcome of an orchestrated synergy of motor, cognitive, and attentional factors.
Holographic control of neuronal circuits
Genetic targeting of neuronal cells with activity reporters (calcium or voltage indicators) has initiated the paradigmatic transition whereby photons have replaced electrons for reading large-scale brain activities at cellular resolution. This has alleviated the limitations of single cell or extracellular electrophysiological probing, which only give access to the activity of at best a few neurons simultaneously and to population activity of unresolved cellular origin, respectively. In parallel, optogenetics has demonstrated that targeting neuronal cells with photosensitive microbial opsins, enables the transduction of photons into electrical currents of opposite polarities thus writing, through activation or inhibition, neuronal signals in a non-invasive way. These progresses have in turn stimulated the development of sophisticated optical methods to increase spatial and temporal resolution, light penetration depth and imaging volume. Today, nonlinear microscopy, combined with spatio-temporal wave front shaping, endoscopic probes engineering or multi scan heads design, enable in vivo in depth, simultaneous recording of thousands of cells in mm 3 volumes at single-spike precision and single-cell resolution. Joint progress in opsin engineering, wave front shaping and laser development have provided the methodology, that we named circuits optogenetics, to control single or multiple target activity independently in space and time with single- neuron and single-spike precision, at large depths. Here, we will review the most significant breakthroughs of the past years, which enable reading and writing neuronal activity at the relevant spatiotemporal scale for brain circuits manipulation, with particular emphasis on the most recent advances in circuit optogenetics.
Microneurography And Microstimulation Of Single Tactile Afferents In The Human Hand
Microneurography is a method, invented by Ake Vallbo and Karl-Erik Hagbarth in the late 1960, with which we can record the activity from single, identified nerve fibres in awake human participants. In this talk, I will then discuss the method, its advantages and limitations, and some of the key discoveries regarding coding of tactile events in the signalling from receptors in the human skin. An extension of the method is to stimulate single afferents, and record the resulting tactile sensations reported by the participants, so-called microstimulation. The first experiments were done in the 1980s, but the method has recently seen a revival, and is currently being combined with high-resolution brain imaging in the study of the relationship between tactile nerve signals, sensations, and processing of tactile information in the brain.
E-prop: A biologically inspired paradigm for learning in recurrent networks of spiking neurons
Transformative advances in deep learning, such as deep reinforcement learning, usually rely on gradient-based learning methods such as backpropagation through time (BPTT) as a core learning algorithm. However, BPTT is not argued to be biologically plausible, since it requires to a propagate gradients backwards in time and across neurons. Here, we propose e-prop, a novel gradient-based learning method with local and online weight update rules for recurrent neural networks, and in particular recurrent spiking neural networks (RSNNs). As a result, e-prop has the potential to provide a substantial fraction of the power of deep learning to RSNNs. In this presentation, we will motivate e-prop from the perspective of recent insights in neuroscience and show how these have to be combined to form an algorithm for online gradient descent. The mathematical results will be supported by empirical evidence in supervised and reinforcement learning tasks. We will also discuss how limitations that are inherited from gradient-based learning methods, such as sample-efficiency, can be addressed by considering an evolution-like optimization that enhances learning on particular task families. The emerging learning architecture can be used to learn tasks by a single demonstration, hence enabling one-shot learning.
Swimming in the third domain: archaeal extremophiles
Archaea have evolved to survive in some of the most extreme environments on earth. Life in extreme, nutrient-poor conditions gives the opportunity to probe fundamental energy limitations on movement and response to stimuli, two essential markers of living systems. Here we use three-dimensional holographic microscopy and computer simulations to show that halophilic archaea achieve chemotaxis with power requirements one hundred-fold lower than common eubacterial model systems. Their swimming direction is stabilised by their flagella (archaella), enhancing directional persistence in a manner similar to that displayed by eubacteria, albeit with a different motility apparatus. Our experiments and simulations reveal that the cells are capable of slow but deterministic chemotaxis up a chemical gradient, in a biased random walk at the thermodynamic limit.
The consequences and constraints of functional organization on behavior
In many ways, cognitive neuroscience is the attempt to use physiological observation to clarify the mechanisms that shape behavior. Over the past 25 years, fMRI has provided a system-wide and yet somewhat spatially precise view of the response in human cortex evoked by a wide variety of stimuli and task contexts. The current talk focuses on the other direction of inference; the implications of this observed functional organization for behavior. To begin, we must interrogate the methodological and empirical frameworks underlying our derivation of this organization, partially by exploring its relationship to and predictability from gross neuroanatomy. Next, across a series of studies, the implications of two properties of functional organization for behavior will be explored: 1) the co-localization of visual working memory and perceptual processing and 2) implicit learning in the context of distributed responses. In sum, these results highlight the limitations of our current approach and hint at a new general mechanism for explaining observed behavior in context with the neural substrate.
A New Approach to the Hard Problem of Consciousness
David Chalmers’s (1995) hard problem famously states: “It is widely agreed that experience arises from a physical basis, but we have no good explanation of why and how it so arises.” Thomas Nagel (1974) wrote something similar: “If we acknowledge that a physical theory of mind must account for the subjective character of experience, we must admit that no presently available conception gives us a clue about how this could be done.” This presentation will point the way towards the long-sought “good explanation” -- or at least it will provide “a clue”. I will make three points: (1) It is unfortunate that cognitive science took vision as its model example when looking for a ‘neural correlate of consciousness’ because cortical vision (like most cognitive processes) is not intrinsically conscious. There is not necessarily ‘something it is like’ to see. (2) Affective feeling, by contrast, is conscious by definition. You cannot feel something without feeling it. Moreover, affective feeling, generated in the upper brainstem, is the foundational form of consciousness: prerequisite for all the higher cognitive forms. (3) The functional mechanism of feeling explains why and how it cannot go on ‘in the dark’, free of any inner feel. Affect enables the organism to monitor deviations from its expected self-states in uncertain situations and thereby frees homeostasis from the limitations of automatism. As Nagel says, “An organism has conscious mental states if and only if there is something that it is like to be that organism—something it is like for the organism.” Affect literally constitutes the sentient subject.
High precision coding in visual cortex
Single neurons in visual cortex provide unreliable measurements of visual features due to their high trial-to-trial variability. It is not known if this “noise” extends its effects over large neural populations to impair the global encoding of stimuli. We recorded simultaneously from ∼20,000 neurons in mouse primary visual cortex (V1) and found that the neural populations had discrimination thresholds of ∼0.34° in an orientation decoding task. These thresholds were nearly 100 times smaller than those reported behaviourally in mice. The discrepancy between neural and behavioural discrimination could not be explained by the types of stimuli we used, by behavioural states or by the sequential nature of perceptual learning tasks. Furthermore, higher-order visual areas lateral to V1 could be decoded equally well. These results imply that the limits of sensory perception in mice are not set by neural noise in sensory cortex, but by the limitations of downstream decoders.
Deep imitation learning for neuromechanical control: realistic walking in an embodied fly
COSYNE 2025
Testing the power and limitations of predictive connectomics in the fly visual system
COSYNE 2025
Differences in neural activation patterns within the action observation network during imitation of point-light displays and fully visible manipulative actions
FENS Forum 2024
Brain-like visual surround suppression in generic CNNs: successes and limitations
Neuromatch 5