Topic: sensorimotor representations

One of the most essential language skills that humans possess is the ability to correctly execute actions based on linguistic instructions. Here we use the latest advances in Natural Language Processing (NLP) to create a neural model of this cognitive ability. Models are trained on a set of 50 psychophysical tasks which require cognitive competencies well studied in the literature. Information about tasks demands is embedded using a transformer architecture pre-trained on an NLP task. Our best performing model, clipNet, uses 33M parameters pre-trained on embedding images and their correspondin

Complex behavioral tasks like grasping require precise and accurate motor control. This demands an intricate interplay between proprioceptive processing and motor command generation within the sensorimotor system. While the task-driven modeling approach has proven successful in capturing sensory representations, adapting it to model the sensorimotor system has remained a challenge. Here, we investigate the potential of deep reinforcement learning to capture sensorimotor representations. By training artificial agents to imitate natural grasp movements using imitation learning, we develop stimul