Topic: gradient

Metaplasticity is the change in plasticity of a synapse, and an important mechanism for resolving the stability-plasticity dilemma. However, the exact synaptic or neuron properties that modulate metaplasticity still remains unclear. In this work we use meta-continual learning to discover the importance of factors that contribute to metaplasticity. We use a linear model to assign the relative contribution of four commonly used synaptic properties to metaplasticity: the Hessian, gradient, magnitude of the weights, and activity of the post-synaptic neuron. The coefficients of the linear model ar

Gradient-based optimization powered by the backpropagation algorithm proved to be the pivotal method in the training of non-spiking artificial neural networks. At the same time, spiking neural networks hold the promise for efficient processing of real-world sensory data by communicating using discrete events in continuous time. We derive the backpropagation algorithm for a recurrent network of spiking (leaky integrate-and-fire) neurons with hard thresholds and show that the backward dynamics amount to an event-based backpropagation of errors through time. Our derivation uses the jump condition