Language emergence in reinforcement learning agents performing navigational tasks

Biological neural networks have evolved to produce complex social behavior such as communication and cooperation. Social communication only works if cues provided by one individual influence the neural activity and neural representations of the environment in another - particularly internal abstractions in planning and learning new tasks. How this translation works at the level of neural networks is still unclear. We hypothesize that synthesizing and learning to use a state representation that is common across several neural networks is critical to understanding the brain’s function underlying animals' generalization capabilities.
In this work, we use artificial neural networks (ANNs) - used in computational neuroscience as model systems to reveal the mechanisms at work in neural networks in vivo - to understand how communicated information about a navigational task learned by one network can enhance the learning process of another.
Work establishing communication protocols between two artificial intelligence (AI) agents grounded in natural language has already been performed, albeit with finite pre-provided vocabularies. We lift this restriction by allowing language to develop freely, gaining a deeper understanding of how biological neural networks synthesize shared and individual state-space representations.
Using a gridworld model, we establish a communication protocol for navigation between ANN teacher and student agents. A message is passed by the teacher (trained on task) to an untrained student to improve goal-finding. However, as language is a lower-dimensional representation of a high-dimensional object, the maze solution is compressed to a message in a sparsity-promoting manner before it is passed. We find that decoding the message and using it helps the student achieve a significantly higher goal-finding rate and the developed “language” generalizes the goal’s location across different task worlds. Intriguingly, a two-dimensional state space representation develops in the message space, similar to how place cells represent two-dimensional spaces in animal brains.