Topic: Hopfield Networks

Dense Associative Memories (Dense AMs) are energy-based neural networks that share many desirable features of celebrated Hopfield Networks but have superior information storage capabilities. In contrast to conventional Hopfield Networks, which were popular in the 1980s, DenseAMs have a very large memory storage capacity - possibly exponential in the size of the network. This aspect makes them appealing tools for many problems in AI and neurobiology. In this talk I will describe two theories of how DenseAMs might be built in biological “hardware”. According to the first theory, DenseAMs arise a

John Hopfield and Geoffrey Hinton explain how ideas from physics helped establish computational models that store patterns and learn from data. Hopfield develops a physical perspective on collective computation, including associative memories in which network dynamics recover stored patterns from incomplete or distorted inputs. Hinton examines Boltzmann machines, connecting probabilistic neural networks and learning to concepts from statistical physics. Together, the lectures link energy landscapes, interacting units and stochastic behaviour with mechanisms for representation and computation.