Gradient descent

…worse than solutions reached by gradient descent and motivate the problem of sampling them.

Yousef Saad examines relaxed gradient descent for large-scale optimization. Relaxing the optimal step length in Cauchy's steepest descent avoids its characteristic zigzag behavior and can bring the search direction close to an eigenvector of the Hessian. Once that alignment is sufficiently accurate, properties of the Lanczos method can accelerate convergence. The talk analyzes several such strategies and illustrates them in global minimization of strictly convex functions, retaining the simplicity and low memory requirements that make gradient methods attractive for machine learning.

…when RNNs learn long timescales via gradient descent. Here we build a mathematical theory of the learning dynamics of linear RNNs trained to integrate white noise. We show that when the initial recurrent weights are small, the dynamics of lea

Fast and adaptable learning is often characterized by sudden jumps in performance rather than slow adaptation typical of updating synaptic weights. Cognitive science theories posit that rapid insights are due to compositional symbolic representations that allow animals to quickly recombine past knowledge to adapt to new settings. Traditionally, it has been difficult to model this type of compositional learning with neural systems. Here we leverage recent work on compositional computations in multitasking Recurrent Neural Networks (RNNs) to construct a neural model of compositional learning. Pa

Temporal sequences are an important feature of neural information processing in biology. Neurons can fire a spike with millisecond precision, and, at the network level, repetitions of spatiotemporal spike patterns are observed in neurobiological data. However, methods for detecting precise temporal patterns in neural activity suffer from high computational complexity and poor robustness to noise, and quantitative detection of these repetitive patterns remains an open problem. Here, we propose a new method to extract spike patterns embedded in raster plots using a 1D convolutional autoencoder w

Motor control is a complex, high-dimensional task. It has been suggested that the brain may reduce the dimensionality of this control problem by using a set of motor primitives (or muscle synergies) that encode frequently used patterns of muscle co-activations in the time and/or weight-domain (Bizzi et al., 1991; Kutch and Valero-Cuevas, 2012; Bizzi and Cheung, 2013; Alessandro et al., 2013). In this project we investigate how motor primitives may affect the learning of movement in motor cortex. We model motor cortex as a recurrent neural network (RNN) and compare two potential output scenario

By causally mapping neural activity to behavior [1], Brain-Computer Interfaces (BCI) offer a means to study the dynamics of sensorimotor learning. Here, we combine computational modeling and data analysis to study the neural learning algorithm [2] monkeys use to adapt to a changed output mapping in a center-out reaching task. We exploit that the mapping from neural space (ca. 100 dimensions) to the 2D cursor position is a credit assignment problem [3] that is underconstrained, because changes along a large number of output-null dimensions do not influence the behavioral output. We hypothesized

…non-spiking neural networks is gradient descent. In spiking neural networks, however, gradient descent learning is complicated by the all-or-none character of spikes (Fig. A). It can lead to unexpected (dis-)appearances of spikes during training, resulting in disruptive, discontinuous changes in the network dynamics [1]. Further, it seemingly prohibits exact gradients to systematically generate or remove spikes

Recently, there has been an increase in interest in exploring the hypothesis that neural activity conveys information through precise spiking motifs. To investigate this phenomenon, several algorithms have been proposed to detect such motifs in Single Unit Activity recorded from populations of neurons. Based on the inversion of a generative model of raster plot synthesis, we present a novel detection model. This model derives an optimal detection procedure in the form of logistic regression combined with temporal convolution. Its differentiability allows for a supervised learning approach usin

…of learning rules which include exact gradient descent (GD), Feedback-Alignment (FA), Direct-Feedback-Alignment (DFA), Hebbian learning (Hebb) and gat

Recent experimental results have found that a cortical neuron typically has multiple synaptic contacts with a given postsynaptic neuron. Here, we refer to such synaptic connections with the same presynaptic axon and the same postsynaptic neuron as parallel synapses. Whether they provide a functional benefit is currently under investigation. To avoid functional redundancy, these parallel synapses must have different computational properties. To model this, we assume that the amount of neurotransmitter released by each synapse can be described by a sigmoidal transmission function of its presynap

A hallmark of intelligence is the ability to infer how stimuli in the world are interrelated. Transitive inference (TI) tasks test if subjects can generalize transitively (A">"B and B">"C implies A">"C), an important case of such relational inference. Despite longstanding study, it remains an open question how the brain, or even statistical learning systems more generally, implements TI. Here we show that a wide range of statistical and biological learning models generalizes transitively and replicates key behavioral patterns observed in humans and animals performing TI. Further, we c

…and Hebb rules, as well as implausible gradient descent (GD). In the sm

…update algorithm called exponentiated gradient descent (EG). In comparison to gradient descent (GD), this corresponds to changing geometries and measuring parameter differences with the un-normalised relative entropy function instead of Euclidean distance. In line with biology, the multiplicative updates p

Brains often infer low-dimensional features from high-dimensional input. For example, during visual processing a picture captured by the retina is processed hierarchically through the ventral stream, which gradually infers higher-level features encoded by the lower number of neurons. However, it is still unclear how neural networks learn to perform such tasks. Hebbian Learning (HL), a local form of plasticity that uses pre- and postsynaptic activity to change synaptic efficacy, has been used extensively in receptive field models, sparse coding, independent and principal component analysis, an

Learning in cortical multi-layer networks is thought to occur through spatially and temporally local synaptic plasticity aided by ubiquitous feedback connections from upper layers. However, the known biologically plausible learning rules for such networks are yet to reach the high performance of the machine learning algorithms, such as backpropagation [1], on difficult tasks that the brain is capable of solving. It has been shown that one of the obstacles to backpropagation-like algorithm implementation in the brain, the ’weight transport’ problem, can be overcome with initially random feedbac

Gradient descent is a central tool in modern machine learning, largely because it scales well to train large networks on computationally challenging tasks. As such, computational neuroscientists are exploring potential means by which the brain could leverage gradient descent. However, it is unlikely that the brain could implement perfect gradient descent, but instead could rely on using approximations of the true gradient signal. Therefore, it is likely that the gradient estimates used by the brain would have both bias and variance. Here, we use model agnostic mathematical analysis and simulat

…when network training is governed by gradient descent. The potential complexity of learning dynamics is particularly apparent in so called insight moments, when useful task representations are discovered suddenly following a delay during which no learning is noticeable. Such insights are commonly observed in animals and humans, where they are often taken to refl

There has been a recent surge of interest in the question of whether and how deep learning algorithms might be capable of abstract reasoning, much of which has centered around datasets based on Raven’s Progressive Matrices (RPM), a visual analogy problem set commonly employed to assess fluid intelligence. This has led to the development of algorithms that are capable of solving RPM-like problems directly from pixel-level inputs. However, these algorithms require extensive direct training on analogy problems, and typically generalize poorly to novel problem types. This is in stark contrast to h

…learning, usually rely on gradient-based learning methods such as backpropagation through time (BPTT) as a core learning algorithm. However, BPTT is not argued to be biologically plausible, since it requires to a propagate gradients backwards in time and across neurons. Here, we propose e-prop, a novel gradient-based learning method with local and online weight update rules for recurrent neural networks, and in particular recurrent spiking neural networks (RSNNs). As a result, e-prop has the potential to provide a substantial