Semantic representations

It is beyond doubt that cognitive phenomena such as language and reasoning are the direct product of the activity of neurons and synapses. Making progress on this open question, often called the bridging problem, was identified by Richard Axel as the most important challenge in neuroscience today. The research community must identify, from among the ever growing volume of neuroscientific knowledge a conceptual core of basic elements and principles which may suffice for bringing about the brain’s function. Here we describe such a minimalistic model of the brain, and we demonstrate its power by

…speech can be predicted by representations extracted from state-of-the-art text- and speech-based LLMs. In particular, text-based LLMs tend to align better with higher-level language regions, capturing more semantic aspects, while speech

We can draw on both episodic memories and semantic knowledge to guide behavior in novel situations. Despite growing interest, key questions remain on the interplay between these systems. How do semantic representations guide the retrieval of episodic memories? How are low-level semantic relationships abstracted from everyday observations (episodes)? How are these low-level relationships transformed to higher-order knowledge structures? By bridging insights from memory-neuroscience with findings in memory-based decision-making, we propose that the sequential sampling of related episodic memorie

Semantic symmetry (e.g., “intelligent” vs. “stupid”) is a key structural feature of semantic information that enhances the flexibility of human semantic cognition and communication. Although previous neuroscience research has extensively investigated the neural representation of semantic information using various approaches, it is still unclear how symmetric semantic information is represented in the human brain. To uncover the representational patterns of symmetric semantic information in the human cerebral cortex using a data-driven approach, we conducted voxelwise modeling of movie-evoked c

The ability to build, store, and manipulate semantic representations lies at the core of all our (inter)actions. Combining evidence from cognitive neuroimaging and experimental neuropsychology, I study the neurocognitive correlates of semantic knowledge in relation to other cognitive functions, chiefly language. In this talk, I will start by reviewing neuroimaging findings supporting the idea that semantic representations are encoded in distributed yet specialized cortical areas (1), and rapidly recovered (2) according to the requirement of the task at hand (3). I will then focus on studies co

One of the most essential language skills that humans possess is the ability to correctly execute actions based on linguistic instructions. Here we use the latest advances in Natural Language Processing (NLP) to create a neural model of this cognitive ability. Models are trained on a set of 50 psychophysical tasks which require cognitive competencies well studied in the literature. Information about tasks demands is embedded using a transformer architecture pre-trained on an NLP task. Our best performing model, clipNet, uses 33M parameters pre-trained on embedding images and their correspondin

Learning a word, involves mapping low-level perceptual representations of world-referents to high-level semantic representations. It is widely held that conscious access to the visual world-referents is needed to learn the meaning of words. The aim of this study was to use fMRI-based decoded neurofeedback (DecNef) to incept a perceptual meaning into brain representations of meaningless symbols, and do so without participant’s awareness, hence by-passing the explicit mapping between words and perceptual referents in brain regions involved in semantic representation. Accordingly, we developed

…with respect to face recognition and semantic representations. Moreover, there is an increasi