Topic: Hopfield networks

Dense Associative Memories (Dense AMs) are energy-based neural networks that share many desirable features of celebrated Hopfield Networks but have superior information storage capabilities. In contrast to conventional Hopfield Networks, which were popular in the 1980s, DenseAMs have a very large memory storage capacity - possibly exponential in the size of the network. This aspect makes them appealing tools for many problems in AI and neurobiology. In this talk I will describe two theories of how DenseAMs might be built in biological “hardware”. According to the first theory, DenseAMs arise a

Wulfram Gerstner discusses the history, operation and continuing influence of Hopfield networks as models of associative memory.

The mammalian brain forms intricate connectivity patterns, yet its connectivity is ubiquitously sparse, enabling efficient information processing. Hopfield networks have been proposed as a schematic model of auto-associative memory retrieval in the brain; they learn to store memories by modifying their connections’ weights, proxies for biological synaptic strengths. But Hopfield networks are fully connected, which is at odds with the brain’s sparse connectivity. In this work, we ask how a Hopfield network’s memory retrieval accuracy changes when some of its connections are pruned. We hypothesi

Hopfield networks are artificial neural networks which store memory patterns on the states of their neurons by choosing recurrent connection weights and update rules such that the energy landscape of the network forms attractors around the memories. How many stable, sufficiently-attracting memory patterns can we store in such a network using N neurons? The answer depends on the choice of weights and update rule. Inspired by setwise connectivity in biology, we extend Hopfield networks by adding setwise connections and embedding these connections in a simplicial complex. Simplicial complexes are

In neuroscience, classical Hopfield networks are the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, memory-augmented neural networks in machine learning commonly use a key-value mechanism to store and read out memories in a single step. Such augmented networks achieve impressive feats of memory compared to traditional variants, yet it remains unclear whether they can be implemented by biological systems. In our work, we bridge this gap by proposing a set of of biologically plausible three-fact