ePoster, COSYNE 2023
Robust Sequence Recall in Asymmetric Modern Hopfield Networks
Hamza Chaudhry, Dmitry Krotov and Cengiz Pehlevan
Harvard University; School of Engineering and Applied Sciences; IBM Research; Harvard University
- Montreal, Canada
- Computational Neuroscience, Machine Learning and
Abstract
Sequential memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions. Biologically plausible models of sequential memory have been proposed where recurrent neural networks are trained with temporal asymmetric Hebbian (TAH) rules. However, these networks have limited capacity, defined by the maximum length of sequence they can reliably recall, due to overlaps between stored memories reducing the signal to noise ratio. Inspired by recent work in Modern Hopfield Networks (MHN), we expand the capacity of this model by introducing a polynomial nonlinearity to the interaction term. This enhances the separation of patterns. We derive theoretically the improved scaling law for the maximal length of the stored sequence of patterns and verify this result with numerical simulation. Furthermore, we show that our extended model can be implemented as a biologically plausible network with similarities to the cortico-basal ganglia-thalamo-cortical loop, responsible for selecting and generating complex motor sequences.
Topics
- Sequential memory, search World Wide
- artificial intelligence, search World Wide
- recurrent neural networks, search World Wide
- Modern Hopfield Networks, search World Wide
- polynomial nonlinearity, search World Wide
- scaling law, search World Wide
- cortico-basal ganglia-thalamo-cortical loop, search World Wide
- complex motor sequences, search World Wide
Details
- Session
- Poster Session II