Brain-computer interfaces (BCIs) have the potential to enable seamless interaction between humans and technology, with particular power for helping patients with paralysis, motor disabilities, or other neurodegenerative diseases move or communicate with the world. BCIs use ``decoders'‘ to relate neural activity to some desired action. Early approaches used highly specific and targeted statistical models, which required careful adaptation to different sessions or days for the same user, and lacked the ability to generalize across users or tasks. Recently, the availability of a large amount of neural data and advances in sequence modelling using deep learning have led to the development of transformer-based decoders that generalize to different users, days and tasks. However, the transformer backbone of these approaches is not amenable to real-time deployment, and may require neurons across days and datasets to be aligned. Furthermore, given the high computational complexity of self-attention used in transformers, they do not scale well to longer sequences. In this work, we propose an architecture that combines powerful attention-based tokenization of spiking activity with recurrent neural network models to facilitate real-time neural decoding. This approach leverages the best of both worlds, demonstrating scalable generalization performance on par with state-of-the-art transformer models combined with the efficiency of simpler recurrent architectures. Our approach achieves an average $R^2$ score of 0.89 in decoding behavior within a session and is able to generalize to unseen sessions with an average $R^2$ score of 0.88 -- even when using fewer than half the number of parameters as a state-of-art transformer-based model. Overall, our results pave the way for scalable and generalizable deep learning models for real-time neural decoding.