Transfer
transfer
Computational Mechanisms of Predictive Processing in Brains and Machines
Predictive processing offers a unifying view of neural computation, proposing that brains continuously anticipate sensory input and update internal models based on prediction errors. In this talk, I will present converging evidence for the computational mechanisms underlying this framework across human neuroscience and deep neural networks. I will begin with recent work showing that large-scale distributed prediction-error encoding in the human brain directly predicts how sensory representations reorganize through predictive learning. I will then turn to PredNet, a popular predictive coding inspired deep network that has been widely used to model real-world biological vision systems. Using dynamic stimuli generated with our Spatiotemporal Style Transfer algorithm, we demonstrate that PredNet relies primarily on low-level spatiotemporal structure and remains insensitive to high-level content, revealing limits in its generalization capacity. Finally, I will discuss new recurrent vision models that integrate top-down feedback connections with intrinsic neural variability, uncovering a dual mechanism for robust sensory coding in which neural variability decorrelates unit responses, while top-down feedback stabilizes network dynamics. Together, these results outline how prediction error signaling and top-down feedback pathways shape adaptive sensory processing in biological and artificial systems.
Comparing supervised learning dynamics: Deep neural networks match human data efficiency but show a generalisation lag
Recent research has seen many behavioral comparisons between humans and deep neural networks (DNNs) in the domain of image classification. Often, comparison studies focus on the end-result of the learning process by measuring and comparing the similarities in the representations of object categories once they have been formed. However, the process of how these representations emerge—that is, the behavioral changes and intermediate stages observed during the acquisition—is less often directly and empirically compared. In this talk, I'm going to report a detailed investigation of the learning dynamics in human observers and various classic and state-of-the-art DNNs. We develop a constrained supervised learning environment to align learning-relevant conditions such as starting point, input modality, available input data and the feedback provided. Across the whole learning process we evaluate and compare how well learned representations can be generalized to previously unseen test data. Comparisons across the entire learning process indicate that DNNs demonstrate a level of data efficiency comparable to human learners, challenging some prevailing assumptions in the field. However, our results also reveal representational differences: while DNNs' learning is characterized by a pronounced generalisation lag, humans appear to immediately acquire generalizable representations without a preliminary phase of learning training set-specific information that is only later transferred to novel data.
Improving Language Understanding by Generative Pre Training
Natural language understanding comprises a wide range of diverse tasks such as textual entailment, question answering, semantic similarity assessment, and document classification. Although large unlabeled text corpora are abundant, labeled data for learning these specific tasks is scarce, making it challenging for discriminatively trained models to perform adequately. We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task. In contrast to previous approaches, we make use of task-aware input transformations during fine-tuning to achieve effective transfer while requiring minimal changes to the model architecture. We demonstrate the effectiveness of our approach on a wide range of benchmarks for natural language understanding. Our general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, significantly improving upon the state of the art in 9 out of the 12 tasks studied. For instance, we achieve absolute improvements of 8.9% on commonsense reasoning (Stories Cloze Test), 5.7% on question answering (RACE), and 1.5% on textual entailment (MultiNLI).
Hippocampal sequences in temporal association memory and information transfer
Diffuse coupling in the brain - A temperature dial for computation
The neurobiological mechanisms of arousal and anesthesia remain poorly understood. Recent evidence highlights the key role of interactions between the cerebral cortex and the diffusely projecting matrix thalamic nuclei. Here, we interrogate these processes in a whole-brain corticothalamic neural mass model endowed with targeted and diffusely projecting thalamocortical nuclei inferred from empirical data. This model captures key features seen in propofol anesthesia, including diminished network integration, lowered state diversity, impaired susceptibility to perturbation, and decreased corticocortical coherence. Collectively, these signatures reflect a suppression of information transfer across the cerebral cortex. We recover these signatures of conscious arousal by selectively stimulating the matrix thalamus, recapitulating empirical results in macaque, as well as wake-like information processing states that reflect the thalamic modulation of largescale cortical attractor dynamics. Our results highlight the role of matrix thalamocortical projections in shaping many features of complex cortical dynamics to facilitate the unique communication states supporting conscious awareness.
Representational Connectivity Analysis (RCA): a Method for Investigating Flow of Content-Specific Information in the Brain
Representational Connectivity Analysis (RCA) has gained mounting interest in the past few years. This is because, rather than conventional tracking of signal, RCA allows for the tracking of information across the brain. It can also provide insights into the content and potential transformations of the transferred information. This presentation explains several variations of the method in terms of implementation and how it can be adopted for different modalities (E/MEG and fMRI). I will also present caveats and nuances of the method which should be considered when using the RCA.
Movement planning as a window into hierarchical motor control
The ability to organise one's body for action without having to think about it is taken for granted, whether it is handwriting, typing on a smartphone or computer keyboard, tying a shoelace or playing the piano. When compromised, e.g. in stroke, neurodegenerative and developmental disorders, the individuals’ study, work and day-to-day living are impacted with high societal costs. Until recently, indirect methods such as invasive recordings in animal models, computer simulations, and behavioural markers during sequence execution have been used to study covert motor sequence planning in humans. In this talk, I will demonstrate how multivariate pattern analyses of non-invasive neurophysiological recordings (MEG/EEG), fMRI, and muscular recordings, combined with a new behavioural paradigm, can help us investigate the structure and dynamics of motor sequence control before and after movement execution. Across paradigms, participants learned to retrieve and produce sequences of finger presses from long-term memory. Our findings suggest that sequence planning involves parallel pre-ordering of serial elements of the upcoming sequence, rather than a preparation of a serial trajectory of activation states. Additionally, we observed that the human neocortex automatically reorganizes the order and timing of well-trained movement sequences retrieved from memory into lower and higher-level representations on a trial-by-trial basis. This echoes behavioural transfer across task contexts and flexibility in the final hundreds of milliseconds before movement execution. These findings strongly support a hierarchical and dynamic model of skilled sequence control across the peri-movement phase, which may have implications for clinical interventions.
Diverse applications of artificial intelligence and mathematical approaches in ophthalmology
Ophthalmology is ideally placed to benefit from recent advances in artificial intelligence. It is a highly image-based specialty and provides unique access to the microvascular circulation and the central nervous system. This talk will demonstrate diverse applications of machine learning and deep learning techniques in ophthalmology, including in age-related macular degeneration (AMD), the leading cause of blindness in industrialized countries, and cataract, the leading cause of blindness worldwide. This will include deep learning approaches to automated diagnosis, quantitative severity classification, and prognostic prediction of disease progression, both from images alone and accompanied by demographic and genetic information. The approaches discussed will include deep feature extraction, label transfer, and multi-modal, multi-task training. Cluster analysis, an unsupervised machine learning approach to data classification, will be demonstrated by its application to geographic atrophy in AMD, including exploration of genotype-phenotype relationships. Finally, mediation analysis will be discussed, with the aim of dissecting complex relationships between AMD disease features, genotype, and progression.
Epigenetic rewiring in Schinzel-Giedion syndrome
During life, a variety of specialized cells arise to grant the right and timely corrected functions of tissues and organs. Regulation of chromatin in defining specialized genomic regions (e.g. enhancers) plays a key role in developmental transitions from progenitors into cell lineages. These enhancers, properly topologically positioned in 3D space, ultimately guide the transcriptional programs. It is becoming clear that several pathologies converge in differential enhancer usage with respect to physiological situations. However, why some regulatory regions are physiologically preferred, while some others can emerge in certain conditions, including other fate decisions or diseases, remains obscure. Schinzel-Giedion syndrome (SGS) is a rare disease with symptoms such as severe developmental delay, congenital malformations, progressive brain atrophy, intractable seizures, and infantile death. SGS is caused by mutations in the SETBP1 gene that results in its accumulation further leading to the downstream accumulation of SET. The oncoprotein SET has been found as part of the histone chaperone complex INHAT that blocks the activity of histone acetyltransferases suggesting that SGS may (i) represent a natural model of alternative chromatin regulation and (ii) offer chances to study downstream (mal)adaptive mechanisms. I will present our work on the characterization of SGS in appropriate experimental models including iPSC-derived cultures and mouse.
The Neural Race Reduction: Dynamics of nonlinear representation learning in deep architectures
What is the relationship between task, network architecture, and population activity in nonlinear deep networks? I will describe the Gated Deep Linear Network framework, which schematizes how pathways of information flow impact learning dynamics within an architecture. Because of the gating, these networks can compute nonlinear functions of their input. We derive an exact reduction and, for certain cases, exact solutions to the dynamics of learning. The reduction takes the form of a neural race with an implicit bias towards shared representations, which then govern the model’s ability to systematically generalize, multi-task, and transfer. We show how appropriate network architectures can help factorize and abstract knowledge. Together, these results begin to shed light on the links between architecture, learning dynamics and network performance.
Roots of Analogy
Can nonhuman animals perceive the relation-between-relations? This intriguing question has been studied over the last 40 years; nonetheless, the extent to which nonhuman species can do so remains controversial. Here, I review empirical evidence suggesting that pigeons, parrots, crows, and baboons join humans in reliably acquiring and transferring relational matching-to-sample (RMTS). Many theorists consider that RMTS captures the essence of analogy, because basic to analogy is appreciating the ‘relation between relations.’ Factors affecting RMTS performance include: prior training experience, the entropy of the sample stimulus, and whether the items that serve as sample stimuli can also serve as choice stimuli.
Protocols for the social transfer of pain and analgesia in mice
We provide protocols for the social transfer of pain and analgesia in mice. We describe the steps to induce pain or analgesia (pain relief) in bystander mice with a 1-h social interaction with a partner injected with CFA (complete Freund’s adjuvant) or CFA and morphine, respectively. We detail behavioral tests to assess pain or analgesia in the untreated bystander mice. This protocol has been validated in mice and rats and can be used for investigating mechanisms of empathy. Highlights • A protocol for the rapid social transfer of pain in rodents • Detailed requirements for handling and housing conditions • Procedures for habituation, social interaction, and pain induction and assessment • Adaptable for social transfer of analgesia and may be used to study empathy in rodents https://doi.org/10.1016/j.xpro.2022.101756
Network inference via process motifs for lagged correlation in linear stochastic processes
A major challenge for causal inference from time-series data is the trade-off between computational feasibility and accuracy. Motivated by process motifs for lagged covariance in an autoregressive model with slow mean-reversion, we propose to infer networks of causal relations via pairwise edge measure (PEMs) that one can easily compute from lagged correlation matrices. Motivated by contributions of process motifs to covariance and lagged variance, we formulate two PEMs that correct for confounding factors and for reverse causation. To demonstrate the performance of our PEMs, we consider network interference from simulations of linear stochastic processes, and we show that our proposed PEMs can infer networks accurately and efficiently. Specifically, for slightly autocorrelated time-series data, our approach achieves accuracies higher than or similar to Granger causality, transfer entropy, and convergent crossmapping -- but with much shorter computation time than possible with any of these methods. Our fast and accurate PEMs are easy-to-implement methods for network inference with a clear theoretical underpinning. They provide promising alternatives to current paradigms for the inference of linear models from time-series data, including Granger causality, vector-autoregression, and sparse inverse covariance estimation.
Learning by Analogy in Mathematics
Analogies between old and new concepts are common during classroom instruction. While previous studies of transfer focus on how features of initial learning guide later transfer to new problem solving, less is known about how to best support analogical transfer from previous learning while children are engaged in new learning episodes. Such research may have important implications for teaching and learning in mathematics, which often includes analogies between old and new information. Some existing research promotes supporting learners' explicit connections across old and new information within an analogy. In this talk, I will present evidence that instructors can invite implicit analogical reasoning through warm-up activities designed to activate relevant prior knowledge. Warm-up activities "close the transfer space" between old and new learning without additional direct instruction.
Mouse visual cortex as a limited resource system that self-learns an ecologically-general representation
Studies of the mouse visual system have revealed a variety of visual brain areas in a roughly hierarchical arrangement, together with a multitude of behavioral capacities, ranging from stimulus-reward associations, to goal-directed navigation, and object-centric discriminations. However, an overall understanding of the mouse’s visual cortex organization, and how this organization supports visual behaviors, remains unknown. Here, we take a computational approach to help address these questions, providing a high-fidelity quantitative model of mouse visual cortex. By analyzing factors contributing to model fidelity, we identified key principles underlying the organization of mouse visual cortex. Structurally, we find that comparatively low-resolution and shallow structure were both important for model correctness. Functionally, we find that models trained with task-agnostic, unsupervised objective functions, based on the concept of contrastive embeddings were substantially better than models trained with supervised objectives. Finally, the unsupervised objective builds a general-purpose visual representation that enables the system to achieve better transfer on out-of-distribution visual, scene understanding and reward-based navigation tasks. Our results suggest that mouse visual cortex is a low-resolution, shallow network that makes best use of the mouse’s limited resources to create a light-weight, general-purpose visual system – in contrast to the deep, high-resolution, and more task-specific visual system of primates.
Flexible multitask computation in recurrent networks utilizes shared dynamical motifs
Flexible computation is a hallmark of intelligent behavior. Yet, little is known about how neural networks contextually reconfigure for different computations. Humans are able to perform a new task without extensive training, presumably through the composition of elementary processes that were previously learned. Cognitive scientists have long hypothesized the possibility of a compositional neural code, where complex neural computations are made up of constituent components; however, the neural substrate underlying this structure remains elusive in biological and artificial neural networks. Here we identified an algorithmic neural substrate for compositional computation through the study of multitasking artificial recurrent neural networks. Dynamical systems analyses of networks revealed learned computational strategies that mirrored the modular subtask structure of the task-set used for training. Dynamical motifs such as attractors, decision boundaries and rotations were reused across different task computations. For example, tasks that required memory of a continuous circular variable repurposed the same ring attractor. We show that dynamical motifs are implemented by clusters of units and are reused across different contexts, allowing for flexibility and generalization of previously learned computation. Lesioning these clusters resulted in modular effects on network performance: a lesion that destroyed one dynamical motif only minimally perturbed the structure of other dynamical motifs. Finally, modular dynamical motifs could be reconfigured for fast transfer learning. After slow initial learning of dynamical motifs, a subsequent faster stage of learning reconfigured motifs to perform novel tasks. This work contributes to a more fundamental understanding of compositional computation underlying flexible general intelligence in neural systems. We present a conceptual framework that establishes dynamical motifs as a fundamental unit of computation, intermediate between the neuron and the network. As more whole brain imaging studies record neural activity from multiple specialized systems simultaneously, the framework of dynamical motifs will guide questions about specialization and generalization across brain regions.
Analogical retrieval across disparate task domains
Previous experiments have shown that a comparison of two written narratives highlights their shared relational structure, which in turn facilitates the retrieval of analogous narratives from the past (e.g., Gentner, Loewenstein, Thompson, & Forbus, 2009). However, analogical retrieval occurs across domains that appear more conceptually distant than merely different narratives, and the deepest analogies use matches in higher-order relational structure. The present study investigated whether comparison can facilitate analogical retrieval of higher-order relations across written narratives and abstract symbolic problems. Participants read stories which became retrieval targets after a delay, cued by either analogous stories or letter-strings. In Experiment 1 we replicated Gentner et al. who used narrative retrieval cues, and also found preliminary evidence for retrieval between narrative and symbolic domains. In Experiment 2 we found clear evidence that a comparison of analogous letter-string problems facilitated the retrieval of source stories with analogous higher-order relations. Experiment 3 replicated the retrieval results of Experiment 2 but with a longer delay between encoding and recall, and a greater number of distractor source stories. These experiments offer support for the schema induction account of analogical retrieval (Gentner et al., 2009) and show that the schemas abstracted from comparison of narratives can be transferred to non-semantic symbolic domains.
Exploration-Based Approach for Computationally Supported Design-by-Analogy
Engineering designers practice design-by-analogy (DbA) during concept generation to retrieve knowledge from external sources or memory as inspiration to solve design problems. DbA is a tool for innovation that involves retrieving analogies from a source domain and transferring the knowledge to a target domain. While DbA produces innovative results, designers often come up with analogies by themselves or through serendipitous, random encounters. Computational support systems for searching analogies have been developed to facilitate DbA in systematic design practice. However, many systems have focused on a query-based approach, in which a designer inputs a keyword or a query function and is returned a set of algorithmically determined stimuli. In this presentation, a new analogical retrieval process that leverages a visual interaction technique is introduced. It enables designers to explore a space of analogies, rather than be constrained by what’s retrieved by a query-based algorithm. With an exploration-based DbA tool, designers have the potential to uncover more useful and unexpected inspiration for innovative design solutions.
Trading Off Performance and Energy in Spiking Networks
Many engineered and biological systems must trade off performance and energy use, and the brain is no exception. While there are theories on how activity levels are controlled in biological networks through feedback control (homeostasis), it is not clear what the effects on population coding are, and therefore how performance and energy can be traded off. In this talk we will consider this tradeoff in auto-encoding networks, in which there is a clear definition of performance (the coding loss). We first show how SNNs follow a characteristic trade-off curve between activity levels and coding loss, but that standard networks need to be retrained to achieve different tradeoff points. We next formalize this tradeoff with a joint loss function incorporating coding loss (performance) and activity loss (energy use). From this loss we derive a class of spiking networks which coordinates its spiking to minimize both the activity and coding losses -- and as a result can dynamically adjust its coding precision and energy use. The network utilizes several known activity control mechanisms for this --- threshold adaptation and feedback inhibition --- and elucidates their potential function within neural circuits. Using geometric intuition, we demonstrate how these mechanisms regulate coding precision, and thereby performance. Lastly, we consider how these insights could be transferred to trained SNNs. Overall, this work addresses a key energy-coding trade-off which is often overlooked in network studies, expands on our understanding of homeostasis in biological SNNs, as well as provides a clear framework for considering performance and energy use in artificial SNNs.
In pursuit of a universal, biomimetic iBCI decoder: Exploring the manifold representations of action in the motor cortex
My group pioneered the development of a novel intracortical brain computer interface (iBCI) that decodes muscle activity (EMG) from signals recorded in the motor cortex of animals. We use these synthetic EMG signals to control Functional Electrical Stimulation (FES), which causes the muscles to contract and thereby restores rudimentary voluntary control of the paralyzed limb. In the past few years, there has been much interest in the fact that information from the millions of neurons active during movement can be reduced to a small number of “latent” signals in a low-dimensional manifold computed from the multiple neuron recordings. These signals can be used to provide a stable prediction of the animal’s behavior over many months-long periods, and they may also provide the means to implement methods of transfer learning across individuals, an application that could be of particular importance for paralyzed human users. We have begun to examine the representation within this latent space, of a broad range of behaviors, including well-learned, stereotyped movements in the lab, and more natural movements in the animal’s home cage, meant to better represent a person’s daily activities. We intend to develop an FES-based iBCI that will restore voluntary movement across a broad range of motor tasks without need for intermittent recalibration. However, the nonlinearities and context dependence within this low-dimensional manifold present significant challenges.
A new experimental paradigm to study analogy transfer
Analogical reasoning is one of the most complex cognitive functions in humans that allows abstract thinking, high-level reasoning, and learning. Based on analogical reasoning, one can extract an abstract and general concept (i.e., an analogy schema) from a familiar situation and apply it to a new context or domain (i.e., analogy transfer). These processes allow us to solve problems we never encountered before and generate new ideas. However, the place of analogy transfer in problem solving mechanisms is unclear. This presentation will describe several experiments with three main findings. First, we show how analogy transfer facilitates problem-solving, replicating existing empirical data largely based on the radiation/fortress problems with four new riddles. Second, we propose a new experimental task that allows us to quantify analogy transfer. Finally, using science network methodology, we show how restructuring the mental representation of a problem can predict successful solving of an analogous problem. These results shed new light on the cognitive mechanism underlying solution transfer by analogy and provide a new tool to quantify individual abilities.
Differences between beginning and advanced students using specific analogical stimuli during design-by-analogy
Studies reported the effects of different types and different levels of abstraction of analogical stimuli on designers. However, specific, single visual analogical stimuli on the effects of designers have not been reported. We define this type of stimuli as specific analogical stimuli. We used the extended linkography method to analyze the facilitating and limiting effects of specific analogical stimuli and free association analogical stimuli (nonspecific analogical stimuli) on the students' creativity at different design levels. Through an empirical study, we explored the differences in the effects of specific analogy stimuli on the students at different design levels. It clarifies the use of analogical stimuli in design and the teaching of analogical design methods in design education.
The role of histone methyltransferase SETDB1 on regulating mood behaviors
Scaffolding up from Social Interactions: A proposal of how social interactions might shape learning across development
Social learning and analogical reasoning both provide exponential opportunities for learning. These skills have largely been studied independently, but my future research asks how combining skills across previously independent domains could add up to more than the sum of their parts. Analogical reasoning allows individuals to transfer learning between contexts and opens up infinite opportunities for innovation and knowledge creation. Its origins and development, so far, have largely been studied in purely cognitive domains. Constraining analogical development to non-social domains may mistakenly lead researchers to overlook its early roots and limit ideas about its potential scope. Building a bridge between social learning and analogy could facilitate identification of the origins of analogical reasoning and broaden its far-reaching potential. In this talk, I propose that the early emergence of social learning, its saliency, and its meaningful context for young children provides a springboard for learning. In addition to providing a strong foundation for early analogical reasoning, the social domain provides an avenue for scaling up analogies in order to learn to learn from others via increasingly complex and broad routes.
Representation transfer and signal denoising through topographic modularity
To prevail in a dynamic and noisy environment, the brain must create reliable and meaningful representations from sensory inputs that are often ambiguous or corrupt. Since only information that permeates the cortical hierarchy can influence sensory perception and decision-making, it is critical that noisy external stimuli are encoded and propagated through different processing stages with minimal signal degradation. Here we hypothesize that stimulus-specific pathways akin to cortical topographic maps may provide the structural scaffold for such signal routing. We investigate whether the feature-specific pathways within such maps, characterized by the preservation of the relative organization of cells between distinct populations, can guide and route stimulus information throughout the system while retaining representational fidelity. We demonstrate that, in a large modular circuit of spiking neurons comprising multiple sub-networks, topographic projections are not only necessary for accurate propagation of stimulus representations, but can also help the system reduce sensory and intrinsic noise. Moreover, by regulating the effective connectivity and local E/I balance, modular topographic precision enables the system to gradually improve its internal representations and increase signal-to-noise ratio as the input signal passes through the network. Such a denoising function arises beyond a critical transition point in the sharpness of the feed-forward projections, and is characterized by the emergence of inhibition-dominated regimes where population responses along stimulated maps are amplified and others are weakened. Our results indicate that this is a generalizable and robust structural effect, largely independent of the underlying model specificities. Using mean-field approximations, we gain deeper insight into the mechanisms responsible for the qualitative changes in the system’s behavior and show that these depend only on the modular topographic connectivity and stimulus intensity. The general dynamical principle revealed by the theoretical predictions suggest that such a denoising property may be a universal, system-agnostic feature of topographic maps, and may lead to a wide range of behaviorally relevant regimes observed under various experimental conditions: maintaining stable representations of multiple stimuli across cortical circuits; amplifying certain features while suppressing others (winner-take-all circuits); and endow circuits with metastable dynamics (winnerless competition), assumed to be fundamental in a variety of tasks.
Improving Communication With the Brain Through Electrode Technologies
Over the past 30 years bionic devices such as cochlear implants and pacemakers, have used a small number of metal electrodes to restore function and monitor activity in patients following disease or injury of excitable tissues. Growing interest in neurotechnologies, facilitated by ventures such as BrainGate, Neuralink and the European Human Brain Project, has increased public awareness of electrotherapeutics and led to both new applications for bioelectronics and a growing demand for less invasive devices with improved performance. Coupled with the rapid miniaturisation of electronic chips, bionic devices are now being developed to diagnose and treat a wide variety of neural and muscular disorders. Of particular interest is the area of high resolution devices that require smaller, more densely packed electrodes. Due to poor integration and communication with body tissue, conventional metallic electrodes cannot meet these size and spatial requirements. We have developed a range of polymer based electronic materials including conductive hydrogels (CHs), conductive elastomers (CEs) and living electrodes (LEs). These technologies provide synergy between low impedance charge transfer, reduced stiffness and an ability to be provide a biologically active interface. A range of electrode approaches are presented spanning wearables, implantables and drug delivery devices. This talk outlines the materials development and characterisation of both in vitro properties and translational in vivo performance. The challenges for translation and commercial uptake of novel technologies will also be discussed.
Swarms for people
As tiny robots become individually more sophisticated, and larger robots easier to mass produce, a breakdown of conventional disciplinary silos is enabling swarm engineering to be adopted across scales and applications, from nanomedicine to treat cancer, to cm-sized robots for large-scale environmental monitoring or intralogistics. This convergence of capabilities is facilitating the transfer of lessons learned from one scale to the other. Cm-sized robots that work in the 1000s may operate in a way similar to reaction-diffusion systems at the nanoscale, while sophisticated microrobots may have individual capabilities that allow them to achieve swarm behaviour reminiscent of larger robots with memory, computation, and communication. Although the physics of these systems are fundamentally different, much of their emergent swarm behaviours can be abstracted to their ability to move and react to their local environment. This presents an opportunity to build a unified framework for the engineering of swarms across scales that makes use of machine learning to automatically discover suitable agent designs and behaviours, digital twins to seamlessly move between the digital and physical world, and user studies to explore how to make swarms safe and trustworthy. Such a framework would push the envelope of swarm capabilities, towards making swarms for people.
Analogy and ethics: opportunities at the intersection
Analogy offers a new interpretation of a common concern in ethics: whether decision making includes or excludes a consideration of moral issues. This is often discussed as the moral awareness of decision makers and considered a motivational concern. The possible new interpretation is that moral awareness is in part a matter of expertise. Some failures of moral awareness can then be understood as stemming from novicehood. Studies of analogical transfer are consistent with the possibility that moral awareness is in part a matter of expertise, that as a result motivation is less helpful than some prior theorizing would predict, and that many adults are not as expert in the domain of ethics as one might hope. The possibility of expert knowledge of ethical principles leads to new questions and opportunities.
Music training effects on multisensory and cross-sensory transfer processing: from cross-sectional to RCT studies
Integration of „environmental“ information in the neuronal epigenome
The inhibitory actions of the heterogeneous collection of GABAergic interneurons tremendously influence cortical information processing, which is reflected by diseases like autism, epilepsy and schizophrenia that involve defects in cortical inhibition. Apart from the regulation of physiological processes like synaptic transmission, proper interneuron function also relies on their correct development. Hence, decrypting regulatory networks that direct proper cortical interneuron development as well as adult functionality is of great interest, as this helps to identify critical events implicated in the etiology of the aforementioned diseases. Thereby, extrinsic factors modulate these processes and act on cell- and stage-specific transcriptional programs. Herein, epigenetic mechanisms of gene regulation, like DNA methylation executed by DNA methyltransferases (DNMTs), histone modifications and non-coding RNAs, call increasing attention in integrating “environmental information” in our genome and sculpting physiological processes in the brain relevant for human mental health. Several studies associate altered expression levels and function of the DNA methyltransferase 1 (DNMT1) in subsets of embryonic and adult cortical interneurons in patients diagnosed with schizophrenia. Although accumulating evidence supports the relevance of epigenetic signatures for instructing cell type-specific development, only very little is known about their functional implications in discrete developmental processes and in subtype-specific maturation of cortical interneurons. Similarly, little is known about the role of DNMT1 in regulating adult interneurons functionality. This talk will provide an overview about newly identified and roles DNMT1 has in orchestrating cortical interneuron development and adult function. Further, this talk will report about the implications of lncRNAs in mediating site-specific DNA methylation in response to discrete external stimuli.
How single neuron dynamics influence network activity and behaviour
To understand how the brain can perform complex tasks such as perception, we have to understand how information enters the brain, how it is transformed and how it is transferred. But, how do we measure information transfer in the brain? This presentation will start with a general introduction of what mutual information is and how to measure it in an experimental setup. Next, the talk will focus on how this can be used to develop brain models at different (spatial) levels, from the microscopic single neuron level to the macroscopic network and behavioural level. How can we incorporate the knowledge about single neurons, that already show complex dynamics, into network activity and link this to behaviour?
Optogenetic silencing of synaptic transmission with a mosquito rhodopsin
Long-range projections link distant circuits in the brain, allowing efficient transfer of information between regions and synchronization of distributed patterns of neural activity. Understanding the functional roles of defined neuronal projection pathways requires temporally precise manipulation of their activity, and optogenetic tools appear to be an obvious choice for such experiments. However, we and others have previously shown that commonly-used inhibitory optogenetic tools have low efficacy and off-target effects when applied to presynaptic terminals. In my talk, I will present a new solution to this problem: a targeting-enhanced mosquito homologue of the vertebrate encephalopsin (eOPN3), which upon activation can effectively suppress synaptic transmission through the Gi/o signaling pathway. Brief illumination of presynaptic terminals expressing eOPN3 triggers a lasting suppression of synaptic output that recovers spontaneously within minutes in vitro and in vivo. The efficacy of eOPN3 in suppressing presynaptic release opens new avenues for functional interrogation of long-range neuronal circuits in vivo.
Herbert Jasper Lecture
There is a long-standing tension between the notion that the hippocampal formation is essentially a spatial mapping system, and the notion that it plays an essential role in the establishment of episodic memory and the consolidation of such memory into structured knowledge about the world. One theory that resolves this tension is the notion that the hippocampus generates rather arbitrary 'index' codes that serve initially to link attributes of episodic memories that are stored in widely dispersed and only weakly connected neocortical modules. I will show how an essentially 'spatial' coding mechanism, with some tweaks, provides an ideal indexing system and discuss the neural coding strategies that the hippocampus apparently uses to overcome some biological constraints affecting the possibility of shipping the index code out widely to the neocortex. Finally, I will present new data suggesting that the hippocampal index code is indeed transferred to layer II-III of the neocortex.
Anterior Cingulate inputs to nucleus accumbens control the social transfer of pain and analgesia
Empathy plays a critical role in social interactions, and many species, including rodents, display evolutionarily conserved behavioral antecedents of empathy. In both humans and rodents, the anterior cingulate cortex (ACC) encodes information about the affective state of others. However, little is known about which downstream targets of the ACC contribute to empathy behaviors. We optimized a protocol for the social transfer of pain behavior in mice and compared the ACC-dependent neural circuitry responsible for this behavior with the neural circuitry required for the social transfer of two related states: analgesia and fear. We found that a 1-hour social interaction between a bystander mouse and a cagemate experiencing inflammatory pain led to congruent mechanical hyperalgesia in the bystander. This social transfer led to activation of neurons in the ACC and several downstream targets, including the nucleus accumbens (NAc), which was revealed by monosynaptic rabies virus tracing to be directly connected to the ACC. Bidirectional manipulation of activity in ACC-to-NAc inputs influenced the acquisition of socially transferred pain. Further, the social transfer of analgesia also depended upon ACC-NAc inputs. By contrast, the social transfer of fear instead required activity in ACC projections to the basolateral amygdala. This shows that mice rapidly adopt the sensory-affective state of a social partner, regardless of the valance of the information (pain, fear, or pain relief). We find that the ACC generates specific and appropriate empathic behavioral responses through distinct downstream targets. More sophisticated understanding of evolutionarily conserved brain mechanisms of empathy will also expedite the development of new therapies for the empathy-related deficits associated with a broad range of neuropsychiatric disorders.
Magnetic Resonance Measures of Brain Blood Vessels, Metabolic Activity, and Pathology in Multiple Sclerosis
The normally functioning blood-brain barrier (BBB) regulates the transfer of material between blood and brain. BBB dysfunction has long been recognized in multiple sclerosis (MS), and there is considerable interest in quantifying functional aspects of brain blood vessels and their role in disease progression. Parenchymal water content and its association with volume regulation is important for proper brain function, and is one of the key roles of the BBB. There is convincing evidence that the astrocyte is critical in establishing and maintaining a functional BBB and providing metabolic support to neurons. Increasing evidence suggests that functional interactions between endothelia, pericytes, astrocytes, and neurons, collectively known as the neurovascular unit, contribute to brain water regulation, capillary blood volume and flow, BBB permeability, and are responsive to metabolic demands. Increasing evidence suggests altered metabolism in MS brain which may contribute to reduced neuro-repair and increased neurodegeneration. Metabolically relevant biomarkers may provide sensitive readouts of brain tissue at risk of degeneration, and magnetic resonance offers substantial promise in this regard. Dynamic contrast enhanced MRI combined with appropriate pharmacokinetic modeling allows quantification of distinct features of BBB including permeabilities to contrast agent and water, with rate constants that differ by six orders of magnitude. Mapping of these rate constants provides unique biological aspects of brain vasculature relevant to MS.
STDP and the transfer of rhythmic signals in the brain
Rhythmic activity in the brain has been reported in relation to a wide range of cognitive processes. Changes in the rhythmic activity have been related to pathological states. These observations raise the question of the origin of these rhythms: can the mechanisms responsible for generation of these rhythms and that allow the propagation of the rhythmic signal be acquired via a process of learning? In my talk I will focus on spike timing dependent plasticity (STDP) and examine under what conditions this unsupervised learning rule can facilitate the propagation of rhythmic activity downstream in the central nervous system. Next, the I will apply the theory of STDP to the whisker system and demonstrate how STDP can shape the distribution of preferred phases of firing in a downstream population. Interestingly, in both these cases STDP dynamics does not relax to a fixed-point solution, rather the synaptic weights remain dynamic. Nevertheless, STDP allows for the system to retain its functionality in the face of continuous remodeling of the entire synaptic population.
Cross Domain Generalisation in Humans and Machines
Recent advances in deep learning have produced models that far outstrip human performance in a number of domains. However, where machine learning approaches still fall far short of human-level performance is in the capacity to transfer knowledge across domains. While a human learner will happily apply knowledge acquired in one domain (e.g., mathematics) to a different domain (e.g., cooking; a vinaigrette is really just a ratio between edible fat and acid), machine learning models still struggle profoundly at such tasks. I will present a case that human intelligence might be (at least partially) usefully characterised by our ability to transfer knowledge widely, and a framework that we have developed for learning representations that support such transfer. The model is compared to current machine learning approaches.
Cellular mechanisms behind stimulus evoked quenching of variability
A wealth of experimental studies show that the trial-to-trial variability of neuronal activity is quenched during stimulus evoked responses. This fact has helped ground a popular view that the variability of spiking activity can be decomposed into two components. The first is due to irregular spike timing conditioned on the firing rate of a neuron (i.e. a Poisson process), and the second is the trial-to-trial variability of the firing rate itself. Quenching of the variability of the overall response is assumed to be a reflection of a suppression of firing rate variability. Network models have explained this phenomenon through a variety of circuit mechanisms. However, in all cases, from the vantage of a neuron embedded within the network, quenching of its response variability is inherited from its synaptic input. We analyze in vivo whole cell recordings from principal cells in layer (L) 2/3 of mouse visual cortex. While the variability of the membrane potential is quenched upon stimulation, the variability of excitatory and inhibitory currents afferent to the neuron are amplified. This discord complicates the simple inheritance assumption that underpins network models of neuronal variability. We propose and validate an alternative (yet not mutually exclusive) mechanism for the quenching of neuronal variability. We show how an increase in synaptic conductance in the evoked state shunts the transfer of current to the membrane potential, formally decoupling changes in their trial-to-trial variability. The ubiquity of conductance based neuronal transfer combined with the simplicity of our model, provides an appealing framework. In particular, it shows how the dependence of cellular properties upon neuronal state is a critical, yet often ignored, factor. Further, our mechanism does not require a decomposition of variability into spiking and firing rate components, thereby challenging a long held view of neuronal activity.
The interaction of sensory and motor information to shape neuronal representations in mouse cortical networks
The neurons in our brain never function in isolation; they are organized into complex circuits which perform highly specialized information processing tasks and transfer information through large neuronal networks. The aim of Janelle Pakan's research group is to better understand how neural circuits function during the transformation of information from sensory perception to behavioural output. Importantly, they also aim to further understand the cell-type specific processes that interrupt the flow of information through neural circuits in neurodegenerative disorders with dementia. The Pakan group utilizes innovative neuroanatomical tracing techniques, advanced in vivo two-photon imaging, and genetically targeted manipulations of neuronal activity to investigate the cell-type specific microcircuitry of the cerebral cortex, the macrocircuitry of cortical output to subcortical structures, and the functional circuitry underlying processes of sensory perception and motor behaviour.
Infant Relational Learning - Interactions with Visual and Linguistic Factors
Humans are incredible learners, a talent supported by our ability to detect and transfer relational similarities between items and events. Spotting these common relations despite perceptual differences is challenging, yet there’s evidence that this ability begins early, with infants as young as 3 months discriminating same and different (Anderson et al., 2018; Ferry et al., 2015). How? To understand the underlying mechanisms, I examine how learning outcomes in the first year correspond with changes in input and in infant age. I discuss the commonalities in this process with that seen in older children and adults, as well as differences due to interactions with other maturing processes like language and visual attention.
Information transfer in (barrel) cortex: from single cell to network
Sustainability in Space and on Earth: Research Initiatives of the Space Enabled Research Group
The presentation will present the work of the Space Enabled Research Group at the MIT Media Lab. The mission of the Space Enabled Research Group is to advance justice in Earth’s complex systems using designs enabled by space. Our message is that six types of space technology are supporting societal needs, as defined by the United Nations Sustainable Development Goals. These six technologies include satellite earth observation, satellite communication, satellite positioning, microgravity research, technology transfer, and the infrastructure related to space research and education. While much good work has been done, barriers remain that limit the application of space technology as a tool for sustainable development. The Space Enabled Research Group works to increase the opportunities to apply space technology in support of the Sustainable Development Goals and to support space sustainability. Our research applies six methods, including design thinking, art, social science, complex systems, satellite engineering and data science. We pursue our work by collaborating with development leaders who represent multilateral organizations, national and local governments, non-profits and entrepreneurial firms to identify opportunities to apply space technology in their work. We strive to enable a more just future in which every community can easily and affordably apply space technology. The work toward our mission covers three themes: 1) Research to apply existing space technology to support the United Nations Sustainable Development Goals; 2) Research to design space systems that are accessible and sustainable; and 3) Research to study the relationship between technology design and justice. The presentation will give examples of research projects within each of these themes.
Molecular Biology of the Fragile X Syndrome
Silencing of FMR1 and loss of its gene product, FMRP, results in fragile X syndrome (FXS). FMRP binds brain mRNAs and inhibits polypeptide elongation. Using ribosome profiling of the hippocampus, we find that ribosome footprint levels in Fmr1-deficient tissue mostly reflect changes in RNA abundance. Profiling over a time course of ribosome runoff in wild-type tissue reveals a wide range of ribosome translocation rates; on many mRNAs, the ribosomes are stalled. Sucrose gradient ultracentrifugation of hippocampal slices after ribosome runoff reveals that FMRP co-sediments with stalled ribosomes, and its loss results in decline of ribosome stalling on specific mRNAs. One such mRNA encodes SETD2, a lysine methyltransferase that catalyzes H3K36me3. Chromatin immunoprecipitation sequencing (ChIP-seq) demonstrates that loss of FMRP alters the deployment of this histone mark. H3K36me3 is associated with alternative pre-RNA processing, which we find occurs in an FMRP-dependent manner on transcripts linked to neural function and autism spectrum disorders.
Evaluating different facets of category status for promoting spontaneous transfer
Existing accounts of analogical transfer highlight the importance of comparison-based schema abstraction in aiding retrieval of relevant prior knowledge from memory. In this talk, we discuss an alternative view, the category status hypothesis—which states that if knowledge of a target principle is represented as a relational category, it is easier to activate as a result of categorizing (as opposed to cue-based reminding)—and briefly review supporting evidence. We then further investigate this hypothesis by designing study tasks that promote different facets of category-level representations and assess their impact on spontaneous analogical transfer. A Baseline group compared two analogous cases; the remaining groups experienced comparison plus another task intended to impact the category status of the knowledge representation. The Intension group read an abstract statement of the principle with a supporting task of generating a new case. The Extension group read two more positive cases with the task of judging whether each exemplified the target principle. The Mapping group read a contrast case with the task of revising it into a positive example of the target principle (thereby providing practice moving in both directions between type and token, i.e., evaluating a given case relative to knowledge and using knowledge to generate a revised case). The results demonstrated that both Intension and Extension groups led to transfer improvements over Baseline (with the former demonstrating both improved accessibility of prior knowledge and ability to apply relational concepts). Implications for theories of analogical transfer are discussed.
Virus-like intercellular communication in the nervous system
The neuronal gene Arc is essential for long-lasting information storage in the mammalian brain and mediates various forms of synaptic plasticity. We recently discovered that Arc self-assembles into virus-like capsids that encapsulate RNA. Endogenous Arc protein is released from neurons in extracellular vesicles that mediate the transfer of Arc mRNA into new target cells. Evolutionary analysis indicates that Arc is derived from a vertebrate lineage of Ty3/gypsy retrotransposons, which are also ancestral to retroviruses such as HIV. These findings suggest that Gag retroelements have been repurposed during evolution to mediate intercellular communication in the nervous system that may underlie cognition and memory.
Dynamic computation in the retina by retuning of neurons and synapses
How does a circuit of neurons process sensory information? And how are transformations of neural signals altered by changes in synaptic strength? We investigate these questions in the context of the visual system and the lateral line of fish. A distinguishing feature of our approach is the imaging of activity across populations of synapses – the fundamental elements of signal transfer within all brain circuits. A guiding hypothesis is that the plasticity of neurotransmission plays a major part in controlling the input-output relation of sensory circuits, regulating the tuning and sensitivity of neurons to allow adaptation or sensitization to particular features of the input. Sensory systems continuously adjust their input-output relation according to the recent history of the stimulus. A common alteration is a decrease in the gain of the response to a constant feature of the input, termed adaptation. For instance, in the retina, many of the ganglion cells (RGCs) providing the output produce their strongest responses just after the temporal contrast of the stimulus increases, but the response declines if this input is maintained. The advantage of adaptation is that it prevents saturation of the response to strong stimuli and allows for continued signaling of future increases in stimulus strength. But adaptation comes at a cost: a reduced sensitivity to a future decrease in stimulus strength. The retina compensates for this loss of information through an intriguing strategy: while some RGCs adapt following a strong stimulus, a second population gradually becomes sensitized. We found that the underlying circuit mechanisms involve two opposing forms of synaptic plasticity in bipolar cells: synaptic depression causes adaptation and facilitation causes sensitization. Facilitation is in turn caused by depression in inhibitory synapses providing negative feedback. These opposing forms of plasticity can cause simultaneous increases and decreases in contrast-sensitivity of different RGCs, which suggests a general framework for understanding the function of sensory circuits: plasticity of both excitatory and inhibitory synapses control dynamic changes in tuning and gain.
Finding Needles in Genomic Haystacks
The ability to read the DNA sequences of different organisms has transformed biology in much the same way that the telescope transformed astronomy. And yet, much of the sequence found in these genomes is as enigmatic as the Rosetta Stone was to early Egyptologists. With the aim of making steps to crack the genomic Rosetta Stone, I will describe unexpected ways of using the physics of information transfer first developed at Bell Labs for thinking about telephone communications to try to decipher the meaning of the regulatory features of genomes. Specifically, I will show how we have been able to explore genes for which we know nothing about how they are regulated by using a combination of mutagenesis, deep sequencing and the physics of information, with the result that we now have falsifiable hypotheses about how those genes work. With those results in hand, I will show how simple tools from statistical physics can be used to predict the level of expression of different genes, followed by a description of precision measurements used to test those predictions. Bringing the two threads of the talk together, I will think about next steps in reading and writing genomes at will.
Synthesizing Machine Intelligence in Neuromorphic Computers with Differentiable Programming
The potential of machine learning and deep learning to advance artificial intelligence is driving a quest to build dedicated computers, such as neuromorphic hardware that emulate the biological processes of the brain. While the hardware technologies already exist, their application to real-world tasks is hindered by the lack of suitable programming methods. Advances at the interface of neural computation and machine learning showed that key aspects of deep learning models and tools can be transferred to biologically plausible neural circuits. Building on these advances, I will show that differentiable programming can address many challenges of programming spiking neural networks for solving real-world tasks, and help devise novel continual and local learning algorithms. In turn, these new algorithms pave the road towards systematically synthesizing machine intelligence in neuromorphic hardware without detailed knowledge of the hardware circuits.
On temporal coding in spiking neural networks with alpha synaptic function
The timing of individual neuronal spikes is essential for biological brains to make fast responses to sensory stimuli. However, conventional artificial neural networks lack the intrinsic temporal coding ability present in biological networks. We propose a spiking neural network model that encodes information in the relative timing of individual neuron spikes. In classification tasks, the output of the network is indicated by the first neuron to spike in the output layer. This temporal coding scheme allows the supervised training of the network with backpropagation, using locally exact derivatives of the postsynaptic spike times with respect to presynaptic spike times. The network operates using a biologically-plausible alpha synaptic transfer function. Additionally, we use trainable synchronisation pulses that provide bias, add flexibility during training and exploit the decay part of the alpha function. We show that such networks can be trained successfully on noisy Boolean logic tasks and on the MNIST dataset encoded in time. The results show that the spiking neural network outperforms comparable spiking models on MNIST and achieves similar quality to fully connected conventional networks with the same architecture. We also find that the spiking network spontaneously discovers two operating regimes, mirroring the accuracy-speed trade-off observed in human decision-making: a slow regime, where a decision is taken after all hidden neurons have spiked and the accuracy is very high, and a fast regime, where a decision is taken very fast but the accuracy is lower. These results demonstrate the computational power of spiking networks with biological characteristics that encode information in the timing of individual neurons. By studying temporal coding in spiking networks, we aim to create building blocks towards energy-efficient and more complex biologically-inspired neural architectures.
Mechanisms of pathogenesis in the tauopathies
The distribution of pathological tau in the brain of patients with AD is highly predicable, and as disease worsens, it spreads transynaptically from initial regions of vulnerability. The reason why only some neurons are vulnerable to the accumulation and propagation of pathological forms of tau, and the mechanisms by which tauopathy spreads through the brain are not well understood. Using a combination of immunohistochemistry and computational analysis we have examined pathway differences between vulnerable and resistant neurons. How tau spreads across a synapse has been examined in vitro using different model systems. Our data show that dysregulation of tau homeostasis determines the cellular and regional vulnerability of specific neurons to tau pathology (H. Fu et al. 2019. Nat. Neuro. 22 (1):47-56) and that deficits in tau homeostasis can exacerbate tau accumulation and propagation. Aging appears to impact similar neuronal populations. Mechanisms and consequences of abnormal tau accumulation within neurons, its transfer between cells, pathology propagation and therapeutic opportunities will be discussed.
Information transfer during dyadic interactions in perceptual decision-making.
Bernstein Conference 2024
Characterization of neuronal resonance and inter-areal transfer using optogenetics
COSYNE 2022
Neural basis of transferable representations for efficient learning
COSYNE 2022
Neural basis of transferable representations for efficient learning
COSYNE 2022
Alternating inference and learning: a thalamocortical model for continual and transfer learning
COSYNE 2023
Inter-animal variability in learning depends on transfer of pre-task experience via the hippocampus
COSYNE 2023
Learning beyond the synapse: activity-dependent myelination, neural correlations, and information transfer
COSYNE 2023
Coordinated Multi-frequency Oscillatory Bursts Enable Time-structured Dynamic Information Transfer
COSYNE 2025
A gated receptivity model of widely broadcast signals for the transfer of graded information
COSYNE 2025
Using transfer learning to identify a neural system's algorithm
COSYNE 2025
Artifact identification in transfer entropy connectivity inference of neuronal cultures
FENS Forum 2024
Cortical circuits for goal-directed cross-modal transfer learning
FENS Forum 2024
Effect of RNA m6A methyltransferase activation on anxiety- and depression-related behaviours, monoamine neurochemistry, and striatal gene expression in the rat
FENS Forum 2024
Endogenous alpha-Synuclein is essential for the transfer of pathology by exosome-enriched extracellular vesicles, following inoculation with preformed fibrils in vivo
FENS Forum 2024
Feasibility of the Pavlovian instrumental transfer task using a full transfer paradigm: A replication and meta-analysis
FENS Forum 2024
Fecal microbiota transfer reduces alcohol preference in stressed rats
FENS Forum 2024
Investigating information transfer at the single-cell level using ultra low-density neuronal networks
FENS Forum 2024
EMG monitoring of central neuroplastic changes after nerve transfer procedures in brachial plexus injury
FENS Forum 2024
Passive transfer of IgG from patients with long-COVID neurological symptoms induces tactile allodynia in mice
FENS Forum 2024
Placental transfer of NMDA receptor autoantibodies impairs correlated network activity by affecting GABAergic neurotransmission
FENS Forum 2024
Postnatal developmental dynamics of choline acetyltransferase (ChAT) and nerve growth factor (NGF) expression in rat oculomotor system
FENS Forum 2024
Transfer Learning from Real to Imagined Motor Actions in ECoG Data
Neuromatch 5