Topic: gradient descent

Yousef Saad examines relaxed gradient descent for large-scale optimization. Relaxing the optimal step length in Cauchy's steepest descent avoids its characteristic zigzag behavior and can bring the search direction close to an eigenvector of the Hessian. Once that alignment is sufficiently accurate, properties of the Lanczos method can accelerate convergence. The talk analyzes several such strategies and illustrates them in global minimization of strictly convex functions, retaining the simplicity and low memory requirements that make gradient methods attractive for machine learning.

Brains often infer low-dimensional features from high-dimensional input. For example, during visual processing a picture captured by the retina is processed hierarchically through the ventral stream, which gradually infers higher-level features encoded by the lower number of neurons. However, it is still unclear how neural networks learn to perform such tasks. Hebbian Learning (HL), a local form of plasticity that uses pre- and postsynaptic activity to change synaptic efficacy, has been used extensively in receptive field models, sparse coding, independent and principal component analysis, an

Gradient descent is a central tool in modern machine learning, largely because it scales well to train large networks on computationally challenging tasks. As such, computational neuroscientists are exploring potential means by which the brain could leverage gradient descent. However, it is unlikely that the brain could implement perfect gradient descent, but instead could rely on using approximations of the true gradient signal. Therefore, it is likely that the gradient estimates used by the brain would have both bias and variance. Here, we use model agnostic mathematical analysis and simulat

The success of neural networks derives from the fact that they can learn useful representations of observed inputs. However, the dynamics of when and how fast a network will discover good representations are not well understood, even when network training is governed by gradient descent. The potential complexity of learning dynamics is particularly apparent in so called insight moments, when useful task representations are discovered suddenly following a delay during which no learning is noticeable. Such insights are commonly observed in animals and humans, where they are often taken to refl

There has been a recent surge of interest in the question of whether and how deep learning algorithms might be capable of abstract reasoning, much of which has centered around datasets based on Raven’s Progressive Matrices (RPM), a visual analogy problem set commonly employed to assess fluid intelligence. This has led to the development of algorithms that are capable of solving RPM-like problems directly from pixel-level inputs. However, these algorithms require extensive direct training on analogy problems, and typically generalize poorly to novel problem types. This is in stark contrast to h

Transformative advances in deep learning, such as deep reinforcement learning, usually rely on gradient-based learning methods such as backpropagation through time (BPTT) as a core learning algorithm. However, BPTT is not argued to be biologically plausible, since it requires to a propagate gradients backwards in time and across neurons. Here, we propose e-prop, a novel gradient-based learning method with local and online weight update rules for recurrent neural networks, and in particular recurrent spiking neural networks (RSNNs). As a result, e-prop has the potential to provide a substantial

Motor control is a complex, high-dimensional task. It has been suggested that the brain may reduce the dimensionality of this control problem by using a set of motor primitives (or muscle synergies) that encode frequently used patterns of muscle co-activations in the time and/or weight-domain (Bizzi et al., 1991; Kutch and Valero-Cuevas, 2012; Bizzi and Cheung, 2013; Alessandro et al., 2013). In this project we investigate how motor primitives may affect the learning of movement in motor cortex. We model motor cortex as a recurrent neural network (RNN) and compare two potential output scenario

By causally mapping neural activity to behavior [1], Brain-Computer Interfaces (BCI) offer a means to study the dynamics of sensorimotor learning. Here, we combine computational modeling and data analysis to study the neural learning algorithm [2] monkeys use to adapt to a changed output mapping in a center-out reaching task. We exploit that the mapping from neural space (ca. 100 dimensions) to the 2D cursor position is a credit assignment problem [3] that is underconstrained, because changes along a large number of output-null dimensions do not influence the behavioral output. We hypothesized

The ability to train spiking neural network models is essential for the modeling of biological neural networks as well as for neuromorphic computing. The standard approach when training non-spiking neural networks is gradient descent. In spiking neural networks, however, gradient descent learning is complicated by the all-or-none character of spikes (Fig. A). It can lead to unexpected (dis-)appearances of spikes during training, resulting in disruptive, discontinuous changes in the network dynamics [1]. Further, it seemingly prohibits exact gradients to systematically generate or remove spikes