Topic: artificial neural networks

We have entered a golden age of artificial intelligence research, driven mainly by the advances in the artificial neural networks over the last several decades. Applications of these techniques—to machine vision, speech recognition, autonomous vehicles, natural language, and many other domains—are coming so quickly that many observers predict that the long-elusive goal of “Artificial General Intelligence” (AGI) is within our grasp. However, we still cannot build a machine capable of building a nest, stalking prey, or loading a dishwasher. I will describe how evolution may have shaped the algor

We build upon and expand the efficient coding and predictive information models of neurons, presenting a novel perspective that neurons not only predict but also actively influence their future inputs through their outputs. We introduce the concept of neurons as feedback controllers of their environments, a role traditionally considered computationally demanding, particularly when the dynamical system characterizing the environment is unknown. By harnessing a novel data-driven control framework, we illustrate the feasibility of biological neurons functioning as effective feedback controllers.

Across the primate neocortex, neurons dedicated to similar functions are likely to be found physically nearby. In the high-level visual cortex, this principle gives rise to cortical patches with distinct category selectivity that were previously observed across many species including macaque monkeys and humans. Models of visual cortex based on artificial neural networks (ANN) have been shown to contain internal representations that are remarkably similar to those observed in the visual cortex. However, unit selectivity in these models emerges without any particular spatial order (i.e. no topog

In the first part of this talk I will present work showing that recurrent neural networks can replicate broad behavioral patterns associated with dynamic visual object recognition in humans. An analysis of these networks shows that different types of recurrence use different strategies to solve the object recognition problem. The similarities between artificial neural networks and the brain presents another opportunity, beyond using them just as models of biological processing. In the second part of this talk, I will discuss—and solicit feedback on—a proposed research plan for testing a wide r

Artificial neural networks simplify complex biological circuits into tractable models for computational exploration and experimentation. However, the simplification of artificial models also undermines their applicability to real brain dynamics. Typical efforts to address this mismatch add complexity to increasingly unwieldy models. Here, we take a different approach; by reducing the complexity of a biological cortical culture, we aim to distil the essential factors of neuronal dynamics and plasticity. We leverage recent advances in growing neurons from human induced pluripotent stem cells (hi

Artificial neural networks can be useful for studying brain functions. In cognitive neuroscience, recurrent neural networks are often used to model cognitive functions. I will first offer my opinion on what is missing in the classical use of recurrent neural networks. Then I will discuss two lines of ongoing efforts in our group to move beyond the classical recurrent neural networks by studying multi-system neural networks (the talk will focus on two-system networks). These are networks that combine modules for several neural systems, such as vision, audition, prefrontal, hippocampal systems.

Hopfield networks are artificial neural networks which store memory patterns on the states of their neurons by choosing recurrent connection weights and update rules such that the energy landscape of the network forms attractors around the memories. How many stable, sufficiently-attracting memory patterns can we store in such a network using N neurons? The answer depends on the choice of weights and update rule. Inspired by setwise connectivity in biology, we extend Hopfield networks by adding setwise connections and embedding these connections in a simplicial complex. Simplicial complexes are

One of the important subfields of computational neuroscience is network neuroscience, which widely uses machine learning techniques, these include artificial neural networks. The latter ones owe a lot to neuroscience in the history of their development. Artificial neural networks (ANN) are often represented in the form of a graph – a visual representation of connections between the elements of the system, which form a network. However, this does not mean that an ANN was designed based on graph theory or with consideration of the special properties of complex networks. These representations ar

Neural representations of time are central to our understanding of the world around us. I review cognitive, neurophysiological and theoretical work that converges on three simple ideas. First, the time of past events is remembered via populations of neurons with a continuum of functional time constants. Second, these time constants evenly tile the log time axis. This results in a neural Weber-Fechner scale for time which can support behavioral Weber-Fechner laws and characteristic behavioral effects in memory experiments. Third, these populations appear as dual pairs---one type of population c

Gradient descent is a central tool in modern machine learning, largely because it scales well to train large networks on computationally challenging tasks. As such, computational neuroscientists are exploring potential means by which the brain could leverage gradient descent. However, it is unlikely that the brain could implement perfect gradient descent, but instead could rely on using approximations of the true gradient signal. Therefore, it is likely that the gradient estimates used by the brain would have both bias and variance. Here, we use model agnostic mathematical analysis and simulat

The last twenty years have witnessed extraordinarily rapid progress in basic neuroscience, including breakthrough technologies such as optogenetics, and the collection of unprecedented amounts of neuroimaging, genetic and other data relevant to neuroscience and mental health. However, the translation of this progress into improved understanding of brain function and dysfunction has been comparatively slow. As a result, the development of therapeutics for mental health has stagnated too. One central challenge has been to extract meaning from these large, complex, multivariate datasets, which re

Knowledge discovery with computers requires a huge amount of search. Analogical reasoning is effective for efficient knowledge discovery. Therefore, we proposed analogical reasoning systems based on first-order predicate logic using Neuro-Symbolic AI. Neuro-Symbolic AI is a combination of Symbolic AI and artificial neural networks and has features that are easy for human interpretation and robust against data ambiguity and errors. We have implemented analogical reasoning systems by Neuro-symbolic AI models with word embedding which can represent similarity between words. Using the proposed sys

Object recognition relies on the complex visual representations in cortical areas at the top of the ventral stream hierarchy. While these are thought to be derived from low-level stages of visual processing, this has not been shown, yet. Here, I describe the results of two projects exploring the contributions of primary visual cortex (V1) processing to object recognition using artificial neural networks (ANNs). First, we developed hundreds of ANN-based V1 models and evaluated how their single neurons approximate those in the macaque V1. We found that, for some models, single neurons in interme

Tali's work emphasized the tradeoff between compression and information preservation. In this talk I will explore this theme in the context of deep learning. Artificial neural networks have recently revolutionized the field of machine learning. However, we still do not have sufficient theoretical understanding of how such models can be successfully learned. Two specific questions in this context are: how can neural nets be learned despite the non-convexity of the learning problem, and how can they generalize well despite often having more parameters than training data. I will describe our rec

In recent years, artificial neural networks have demonstrated human-like or super-human performance in many tasks including image or speech recognition, natural language processing (NLP), playing Go, chess, poker and video-games. One remarkable feature of the resulting models is that they can develop very intuitive latent representations of their inputs. In these latent spaces, simple linear operations tend to give meaningful results, as in the well-known analogy QUEEN-WOMAN+MAN=KING. We postulate that human brain representations share essential properties with these deep learning latent space

Activity patterns of neural populations in natural and artificial neural networks constitute representations of data. The nature of these representations and how they are learned are key questions in neuroscience and deep learning. In his talk, I will describe my group's efforts in building a theory of representations as feature maps leading to sample efficient function approximation. Kernel methods are at the heart of these developments. I will present applications to deep learning and neuronal data.

Many regions of the human brain conduct highly specific functions, such as recognizing faces, understanding language, and thinking about other people’s thoughts. Why might this domain specific organization be a good design strategy for brains, and what is the origin of domain specificity in the first place? In this talk, I will present recent work testing whether the segregation of face and object perception in human brains emerges naturally from an optimization for both tasks. We trained artificial neural networks on face and object recognition, and found that networks were able to perform bo

Artificial neural networks (ANNs) form the basis of most successful Deep Learning (DL) algorithms$^1$, which are capable of solving complex problems such as image recognition and natural language processing$^{2,3}$. However, unlike biological brains, which efficiently solve similar problems, DL algorithms require a large number of adjustable parameters, making them energy-intensive and susceptible to overfitting. In this study, a new ANN architecture is introduced that incorporates the structured connectivity and restricted sampling properties of biological dendrites, aiming to overcome these

It has long been believed that the brain is highly modular both in terms of structure and function, although recent evidence has led some to question the extent of both types of modularity. We used artificial neural networks to test the hypothesis that structural modularity is sufficient to guarantee functional specialization, and find that in general, this doesn't necessarily hold. We then systematically tested which features of the environment and network do lead to the emergence of specialization. We used a simple toy environment, task and network, allowing us precise control, and show that

Artificial neural networks (ANNs) achieve remarkable results on various tasks, but understanding the computational mechanisms underlying their performance remains difficult. Furthermore, traditionally employed artificial networks have little in common with real biological networks. Overcoming this difference, machine-learning-based training methods for spiking neuronal networks (SNNs) have been developed to create functional networks, enabling the investigation of these neural networks with neuroscientific analysis methods. Here we analyze one such SNN trained with backpropagation based on a

Recent progress in artificial intelligence (AI), particularly through the development of artificial neural networks (ANNs), has significantly benefited from insights gained from neuroscience. This progress has enhanced the replication of complex cognitive tasks such as vision and natural language processing (1,2). However, ANNs still struggle with continual learning, adaptable knowledge transfer, robustness, and resource efficiency - capabilities that biological systems handle seamlessly (3–8). Neuromodulators, the chemical messengers that target neurons and synapses in specific brain regions,

Continual learning involves a network adapting to new data while retaining previously learned information. In Artificial Neural Networks , this process faces the challenge of catastrophic forgetting, where updating parameters with new data causes an exponential decline in accuracy over time for previously learned tasks. Several methods have been proposed to address this problem and enable continual learning on future neuromorphic hardware. One such neuroscience-inspired method is metaplasticity, which refers to a network's ability to change the learning capacity of its synapses based on its cu

The success of Artificial Neural Networks (ANNs) trained with error-backpropagation algorithms relies on the combination of offline training, vast amounts of data, and energy-intensive simulations on dedicated hardware accelerators. In contrast, biological neurons can learn quickly in a self-supervised manner by utilizing (a) recurrent connections within their local network and from other areas, (b) segregated dendrites receiving inputs from feedforward and feedback pathways, and (c) learning rules that combine locally available information from pre- and -post-synaptic sources. Inspired by loc

Despite being inspired by biological neuronal networks (BNNs), artificial neural networks (ANNs) have fundamental differences from their biological counterparts. While ANNs excel at solving complex problems across various domains, BNNs outperform them in terms of energy efficiency and processing highly complex parallel inputs. However, the mechanisms behind these advantages remain unclear, making it challenging to integrate them into artificial computation. It has been shown that biological neurons exhibit a characteristic sigmoid-like firing rate response [1], termed bio-sigmoid, as a functi