Topic: storage capacity

The mammalian brain forms intricate connectivity patterns, yet its connectivity is ubiquitously sparse, enabling efficient information processing. Hopfield networks have been proposed as a schematic model of auto-associative memory retrieval in the brain; they learn to store memories by modifying their connections’ weights, proxies for biological synaptic strengths. But Hopfield networks are fully connected, which is at odds with the brain’s sparse connectivity. In this work, we ask how a Hopfield network’s memory retrieval accuracy changes when some of its connections are pruned. We hypothesi

It is widely believed that storing and maintaining memories on long-time scales depends on modifying synapses in the brain in an activity-dependent way. Classical studies of learning and memory in neural networks model synaptic efficacy as a continuous or discrete scalar value [1–3]. Theoretical work has shown such models have a reasonably large capacity, especially in the biologically relevant sparse coding limit [4]. However, multiple recent results suggest an intermediate scenario in which synaptic efficacy can be described by a continuous variable, but whose distribution is peaked around a

Neurons in the brain communicate through action potentials (spikes) that are transmitted through chemical synapses. Throughout the last decades, the question how networks of spiking neurons represent and process information has remained an important challenge. Some progress has resulted from a recent family of supervised learning rules (tempotrons) for models of spiking neurons. However, these studies have viewed synaptic transmission as static and characterized synaptic efficacies as scalar quantities that change only on slow time scales of learning across trials but remain fixed on the fast