Cognition seminars

What is perception? Our sensory modalities transmit information about the external world into electrochemical signals that somehow give rise to our conscious experience of our environment. Normally there is too much information to be processed in any given moment, and the mechanisms of attention focus the limited resources of the mind to some information at the expense of others. My research has advanced from first examining visual perception and attention to now examine how multisensory processing contributes to perception and cognition. There are fundamental constraints on how much information can be processed by the different senses on their own and in combination. Here I will explore information processing from the perspective of sensory substitution and augmentation, and how "seeing" with the ears and tongue can advance fundamental and translational research.

Human cognitive abilities seem qualitatively different from the cognitive abilities of other primates, a difference Penn, Holyoak, and Povinelli (2008) attribute to role-based relational reasoning—inferences and generalizations based on the relational roles to which objects (and other relations) are bound, rather than just the features of the objects themselves. Role-based relational reasoning depends on the ability to dynamically bind arguments to relational roles. But dynamic binding cannot be sufficient for relational thinking: Some non-human animals solve the dynamic binding problem, at least in some domains; and many non-human species generalize affordances to completely novel objects and scenes, a kind of universal generalization that likely depends on dynamic binding. If they can solve the dynamic binding problem, then why can they not reason about relations? What are they missing? I will present simulations with the LISA model of analogical reasoning (Hummel & Holyoak, 1997, 2003) suggesting that the missing pieces are multi-role integration (the capacity to combine multiple role bindings into complete relations) and structure mapping (the capacity to map different systems of role bindings onto one another). When LISA is deprived of either of these capacities, it can still generalize affordances universally, but it cannot reason symbolically; granted both abilities, LISA enjoys the full power of relational (symbolic) thought. I speculate that one reason it may have taken relational reasoning so long to evolve is that it required evolution to solve both problems simultaneously, since neither multi-role integration nor structure mapping appears to confer any adaptive advantage over simple role binding on its own.

Voluntary control of behavior requires the ability to dynamically integrate internal states and external evidence to achieve one’s goals. However, neuroscientific studies of intentional action and critical philosophical commentary of that research have taken a rather narrow turn in recent years, focussing on the neural precursors of spontaneous simple actions as potential realizers of intentions. In this session, we show how the debate can benefit from incorporating other types of experimental approaches, focussing on agency in dynamic contexts.

Holding information in working memory is essential for cognition, but removing unwanted thoughts is equally important. There is great flexibility in how we can manipulate information in working memory, but the processes and consequences of these operations are poorly understood. In this talk I will discuss our recent findings using multivariate pattern analyses of fMRI brain data to demonstrate the successful removal of information from working memory using three different strategies: suppressing a specific thought, replacing a thought with a different one, and clearing the mind of all thought. These strategies are supported by distinct brain regions and have differential consequences on the encoding of new information. I will discuss implications of these results on theories of memory and I will highlight some new directions involving the use of real-time neurofeedback to investigate causal links between brain and behavior.

Analogical reasoning remains foundational to the human ability to forge meaningful patterns within the sea of information that continually inundates the senses. Yet, meaningful patterns rely not only on the recognition of attributional similarities but also dissimilarities. Just as the perception of images rests on the juxtaposition of lightness and darkness, reasoning relationally requires systematic attention to both similarities and dissimilarities. With that awareness, my colleagues and I have expanded the study of relational reasoning beyond analogous reasoning and attributional similarities to highlight forms based on the nature of core dissimilarities: anomalous, antinomous, and antithetical reasoning. In this presentation, I will delineate the character of these relational reasoning forms; summarize procedures and measures used to assess them; overview key research findings; and describe how the forms of relational reasoning work together in the performance of complex problem solving. Finally, I will share critical next steps for research which has implications for instructional practice.

Predictive learning hypotheses posit that the neocortex learns a hierarchical model of the structure of features in the environment. Under these hypotheses, expected or predictable features are differentiated from unexpected ones by comparing bottom-up and top-down streams of data, with unexpected features then driving changes in the representation of incoming stimuli. This is supported by numerous studies in early sensory cortices showing that pyramidal neurons respond particularly strongly to unexpected stimulus events. However, it remains unknown how their responses govern subsequent changes in stimulus representations, and thus, govern learning. Here, I present results from our study of layer 2/3 and layer 5 pyramidal neurons imaged in primary visual cortex of awake, behaving mice using two-photon calcium microscopy at both the somatic and distal apical planes. Our data reveals that individual neurons and distal apical dendrites show distinct, but predictable changes in unexpected event responses when tracked over several days. Considering existing evidence that bottom-up information is primarily targeted to somata, with distal apical dendrites receiving the bulk of top-down inputs, our findings corroborate hypothesized complementary roles for these two neuronal compartments in hierarchical computing. Altogether, our work provides novel evidence that the neocortex indeed instantiates a predictive hierarchical model in which unexpected events drive learning.

Sensory stimuli are often predictable consequences of one’s actions, and behavior exerts a correspondingly strong influence over sensory responses in the brain. Closed-loop experiments with the ability to control the sensory outcomes of specific animal behaviors have revealed that neural responses to self-generated sounds are suppressed in the auditory cortex, suggesting a role for prediction in local sensory processing. However, it is unclear whether this phenomenon derives from a precise movement-based prediction or how it affects the neural representation of incoming stimuli. We address these questions by designing a behavioral paradigm where mice learn to expect the predictable acoustic consequences of a simple forelimb movement. Neuronal recordings from auditory cortex revealed suppression of neural responses that was strongest for the expected tone and specific to the time of the sound-associated movement. Predictive suppression in the auditory cortex was layer-specific, preceded by the arrival of movement information, and unaffected by behavioral relevance or reward association. These findings illustrate that expectation, learned through motor-sensory experience, drives layer-specific predictive processing in the mouse auditory cortex.

Did we eat spaghetti for lunch because we saw our colleague eat spaghetti? What drives a risk decision? How can our breakfast impact our decisions throughout the day? Research from different disciplines such as economics, psychology and neuroscience have attempted to investigate the motives and modulators of human decision making. Human decisions can be flexibly modulated by the different experiences we have in our daily lives, at the same time, bodily processes, such as metabolism can also impact economic behavior. These modulations can occur through our social networks, through the impact of our own behavior on the social environment, but also simply by the food we have eaten. Here, I will present a series of recent studies from my lab in which we shed light on the psychological, neural and metabolic motives and modulators of human decision making.

Given the complex and dynamic nature of our social relationships, the human brain needs to quickly learn and adapt to new social situations. The breakdown of any of these computations could lead to social deficits, as observed in many psychiatric disorders. In this talk, I will present our recent neurocomputational and intracranial work that attempts to model both 1) how humans dynamically adapt beliefs about other people and 2) how individuals can exert influence over social others through model-based forward thinking. Lastly, I will present our findings of how impaired social computations might manifest in different disorders such as addiction, delusion, and autism. Taken together, these findings reveal the dynamic and proactive nature of human interactions as well as the clinical significance of these high-order social processes.

Everything we see has a location. We know where things are before we know what they are. But how do we know where things are? Receptive fields in the visual system specify location but neural delays lead to serious errors whenever targets or eyes are moving. Motion may be the problem here but motion can also be the solution, correcting for the effects of delays and eye movements. To demonstrate this, I will present results from three motion illusions where perceived location differs radically from physical location. These help understand how and where position is coded. We first look at the effects of a target’s simple forward motion on its perceived location. Second, we look at perceived location of a target that has internal motion as well as forward motion. The two directions combine to produce an illusory path. This “double-drift” illusion strongly affects perceived position but, surprisingly, not eye movements or attention. Even more surprising, fMRI shows that the shifted percept does not emerge in the visual cortex but is seen instead in the frontal lobes. Finally, we report that a moving frame also shifts the perceived positions of dots flashed within it. Participants report the dot positions relative to the frame, as if the frame were not moving. These frame-induced position effects suggest a link to visual stability where we see a steady world despite massive displacements during saccades. These motion-based effects on perceived location lead to new insights concerning how and where position is coded in the brain.

In an ever-changing environment, uncertainty is omnipresent. To deal with this, organisms have evolved mechanisms that allow them to take advantage of environmental regularities in order to make decisions robustly and adjust their behavior efficiently, thus maximizing their chances of survival. In this talk, I will present behavioral evidence that animals perform model-based state inference to predict environmental state changes and adjust their behavior rapidly, rather than slowly updating choice values. This model-based inference process can be described using Bayesian change-point models. Furthermore, I will show that neural populations in the prefrontal cortex accurately predict behavioral switches, and that the activity of these populations is associated with Bayesian estimates. In addition, we will see that learning leads to the emergence of a high-dimensional representational subspace that can be reused when the animals re-learn a previously learned set of action-value associations. Altogether, these findings highlight the role of the PFC in representing a belief about the current state of the world.