Cognitive Psychology seminars

Several lines of research have demonstrated that internally generated sensory experience - such as during memory, dreaming and mental imagery - activates similar neural representations as externally triggered perception. This overlap raises a fundamental challenge: how is the brain able to keep apart signals reflecting imagination and reality? In a series of online psychophysics experiments combined with computational modelling, we investigated to what extent imagination and perception are confused when the same content is simultaneously imagined and perceived. We found that simultaneous congruent mental imagery consistently led to an increase in perceptual presence responses, and that congruent perceptual presence responses were in turn associated with a more vivid imagery experience. Our findings can be best explained by a simple signal detection model in which imagined and perceived signals are added together. Perceptual reality monitoring can then easily be implemented by evaluating whether this intermixed signal is strong or vivid enough to pass a ‘reality threshold’. Our model suggests that, in contrast to self-generated sensory changes during movement, our brain does not discount self-generated sensory signals during mental imagery. This has profound implications for our understanding of reality monitoring and perception in general.

Selection is the hallmark of attention: processing improves for attended items but is relatively impaired for unattended items. It is well known that visual spatial attention changes sensory signals and perception in this selective fashion. In the work I will present, we asked whether and how attentional selection happens across time. First, our experiments revealed that voluntary temporal attention (attention to specific points in time) is selective, resulting in perceptual tradeoffs across time. Second, we measured small eye movements called microsaccades and found that directing voluntary temporal attention increases the stability of the eyes in anticipation of an attended stimulus. Third, we developed a computational model of dynamic attention, which proposes specific mechanisms underlying temporal attention and its selectivity. Lastly, I will mention how we are testing predictions of the model with MEG. Altogether, this research shows how precisely timed voluntary attention helps manage inherent limits in visual processing across short time intervals, advancing our understanding of attention as a dynamic process.

When we remember a stimulus we rarely maintain a full fidelity representation of the observed item. Our working memory instead maintains a mixture of the observed feature values and categorical/gist information. I will discuss evidence from computational models supporting a mix of categorical and fine-detail information in working memory. Having established the need for two memory formats in working memory, I will discuss whether categorical and fine-detailed information for a stimulus are represented separately or as a single unified representation. Computational models of these two potential cognitive structures make differing predictions about the pattern of responses in visual working memory recall tests. The present study required participants to remember the orientation of stimuli for later reproduction. The pattern of responses are used to test the competing representational structures and to quantify the relative amount of fine-detailed and categorical information maintained. The effects of set size, encoding time, serial order, and response order on memory precision, categorical information, and guessing rates are also explored. (This is a 60 min talk).

In decision-making under risk (DMR) participants' choices are based on probability values systematically different from those that are objectively correct. Similar systematic distortions are found in tasks involving relative frequency judgments (JRF). These distortions limit performance in a wide variety of tasks and an evident question is, why do we systematically fail in our use of probability and relative frequency information? We propose a Bounded Log-Odds Model (BLO) of probability and relative frequency distortion based on three assumptions: (1) log-odds: probability and relative frequency are mapped to an internal log-odds scale, (2) boundedness: the range of representations of probability and relative frequency are bounded and the bounds change dynamically with task, and (3) variance compensation: the mapping compensates in part for uncertainty in probability and relative frequency values. We compared human performance in both DMR and JRF tasks to the predictions of the BLO model as well as eleven alternative models each missing one or more of the underlying BLO assumptions (factorial model comparison). The BLO model and its assumptions proved to be superior to any of the alternatives. In a separate analysis, we found that BLO accounts for individual participants’ data better than any previous model in the DMR literature. We also found that, subject to the boundedness limitation, participants’ choice of distortion approximately maximized the mutual information between objective task-relevant values and internal values, a form of bounded rationality.

The talk will give a tour of Guided Search 6.0 (GS6), the latest evolution of the Guided Search model of visual search. Part 1 describes The Mechanics of Search. Because we cannot recognize more than a few items at a time, selective attention is used to prioritize items for processing. Selective attention to an item allows its features to be bound together into a representation that can be matched to a target template in memory or rejected as a distractor. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 msec/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid serial and parallel model. If a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Part 2 elaborates on the five sources of Guidance that are combined into a spatial “priority map” to guide the deployment of attention (hence “guided search”). These are (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g. priming), (4) reward, and (5) scene syntax and semantics. Finally, in Part 3, we will consider the internal representation of what we are searching for; what is often called “the search template”. That search template is really two templates: a guiding template (probably in working memory) and a target template (in long term memory). Put these pieces together and you have GS6.

Not all individuals are equally competent at recognizing the faces they interact with. Revealing how the brains of different individuals support variations in this ability is a crucial step to develop an understanding of real-world human visual behaviour. In this talk, I will present findings from a large high-density EEG dataset (>100k trials of participants processing various stimulus categories) and computational approaches which aimed to characterise the brain representations behind real-world proficiency of “super-recognizers”—individuals at the top of face recognition ability spectrum. Using decoding analysis of time-resolved EEG patterns, we predicted with high precision the trial-by-trial activity of super-recognizers participants, and showed that evidence for face recognition ability variations is disseminated along early, intermediate and late brain processing steps. Computational modeling of the underlying brain activity uncovered two representational signatures supporting higher face recognition ability—i) mid-level visual & ii) semantic computations. Both components were dissociable in brain processing-time (the first around the N170, the last around the P600) and levels of computations (the first emerging from mid-level layers of visual Convolutional Neural Networks, the last from a semantic model characterising sentence descriptions of images). I will conclude by presenting ongoing analyses from a well-known case of acquired prosopagnosia (PS) using similar computational modeling of high-density EEG activity.

Our research investigates how processes of attention, visual working memory (VWM), and decision-making combine to translate perception into action. Within this framework, the role of VWM is to form stable representations of transient stimulus events that allow them to be identified by a decision process, which we model as a diffusion process. In psychophysical tasks, we find the capacity of VWM is well defined by a sample-size model, which attributes changes in VWM precision with set-size to differences in the number evidence samples recruited to represent stimuli. In the first part of the talk, I review evidence for the sample-size model and highlight the model's strengths: It provides a parameter-free characterization of the set-size effect; it has plausible neural and cognitive interpretations; an attention-weighted version of the model accounts for the power-law of VWM, and it accounts for the selective updating of VWM in multiple-look experiments. In the second part of the talk, I provide a characterization of the theoretical relationship between two-choice and continuous-outcome decision tasks using the circular diffusion model, highlighting their common features. I describe recent work characterizing the joint distributions of decision outcomes and response times in continuous-outcome tasks using the circular diffusion model and show that the model can clearly distinguish variable-precision and simple mixture models of the evidence entering the decision process. The ability to distinguish these kinds of processes is central to current VWM studies.

I will begin by describing recent research taking a new, model-based approach to perceptual development. This approach uncovers fundamental changes in information processing underlying the protracted development of perception, action, and decision-making in childhood. For example, integration of multiple sensory estimates via reliability-weighted averaging – widely used by adults to improve perception – is often not seen until surprisingly late into childhood, as assessed by both behaviour and neural representations. This approach forms the basis for a newer question: the scope for the nervous system to deploy useful computations (e.g. reliability-weighted averaging) to optimise perception and action using newly-learned sensory signals provided by technology. Our initial model system is augmenting visual depth perception with devices translating distance into auditory or vibro-tactile signals. This problem has immediate applications to people with partial vision loss, but the broader question concerns our scope to use technology to tune in to any signal not available to our native biological receptors. I will describe initial progress on this problem, and our approach to operationalising what it might mean to adopt a new signal comparably to a native sense. This will include testing for its integration (weighted averaging) alongside the native senses, assessing the level at which this integration happens in the brain, and measuring the degree of ‘automaticity’ with which new signals are used, compared with native perception.

It seems we see the world in full detail. However, the eye is not a camera nor is the brain a computer. Incredible metabolic constraints render us unable to encode more than a fraction of information available in each glance. Instead, our illusion of stable and complete perception is accomplished by parsimonious representation relying on natural order inherent in the surrounding environment. I will begin by discussing previous behavioral work from our lab demonstrating one such strategy by which the visual system represents average properties of Gestalt-grouped sets of individual objects, warping individual object representations toward the Gestalt-defined mean. I will then discuss on-going work using a behavioral index of averaging Gestalt-grouped information established in our previous work in conjunction with an ERP-index of VSTM capacity (the CDA) to measure whether the Gestalt-grouping and perceptual averaging strategy acts to boost memory capacity above the classic “four-item” limit. Finally, I will outline our pre-registered study to determine whether this perceptual strategy is indeed engaged in a “smart” manner under normal circumstances, or compromises fidelity for capacity by perceptually-averaging in trials with only four items that could otherwise be individually represented.

Behavioural measurements are often overlooked by computational neuroscientists, who prefer to focus on electrophysiological recordings or neuroimaging data. This attitude is largely due to perceived lack of depth/richness in relation to behavioural datasets. I will show how contemporary psychophysics can deliver extremely rich and highly constraining datasets that naturally interface with computational modelling. More specifically, I will demonstrate how psychophysics can be used to guide/constrain/refine computational models, and how models can be exploited to design/motivate/interpret psychophysical experiments. Examples will span a wide range of topics (from feature detection to natural scene understanding) and methodologies (from cascade models to deep learning architectures).

Humans have a remarkable ability to figure out what happened and why. In this talk, I will shed light on this ability from multiple angles. I will present a computational framework for modeling causal explanations in terms of counterfactual simulations, and several lines of experiments testing this framework in the domain of intuitive physics. The model predicts people's causal judgments about a variety of physical scenes, including dynamic collision events, complex situations that involve multiple causes, omissions as causes, and causal responsibility for a system's stability. It also captures the cognitive processes underlying these judgments as revealed by spontaneous eye-movements. More recently, we have applied our computational framework to explain multisensory integration. I will show how people's inferences about what happened are well-accounted for by a model that integrates visual and auditory evidence through approximate physical simulations.

The idea that behavior results from a dynamic interplay between automatic and controlled processing underlies much of decision science, but has also generated considerable controversy. In this talk, I will highlight behavioral and neural data showing how recently-developed computational models of decision making can be used to shed new light on whether, when, and how decisions result from distinct processes operating at different timescales. Across diverse domains ranging from altruism to risky choice biases and self-regulation, our work suggests that a model of prioritized attentional sampling and evidence accumulation may provide an alternative explanation for many phenomena previously interpreted as supporting dual process models of choice. However, I also show how some features of the model might be taken as support for specific aspects of dual-process models, providing a way to reconcile conflicting accounts and generating new predictions and insights along the way.

Constraints on control-dependent processing have become a fundamental concept in general theories of cognition that explain human behavior in terms of rational adaptations to these constraints. However, theories miss a rationale for why such constraints would exist in the first place. Recent work suggests that constraints on the allocation of control facilitate flexible task switching at the expense of the stability needed to support goal-directed behavior in face of distraction. We formulate this problem in a dynamical system, in which control signals are represented as attractors and in which constraints on control allocation limit the depth of these attractors. We derive formal expressions of the stability-flexibility tradeoff, showing that constraints on control allocation improve cognitive flexibility but impair cognitive stability. We provide evidence that human participants adapt higher constraints on the allocation of control as the demand for flexibility increases but that participants deviate from optimal constraints. In continuing work, we are investigating how collaborative performance of a group of individuals can benefit from individual differences defined in terms of balance between cognitive stability and flexibility.

The eyes send a continuous stream of about two million nerve fibers to the brain, but only a fraction of this information is stored as visual memories. This talk will detail three neurocomputational models that attempt an understanding how the visual system makes on-the-fly decisions about how to encode that information. First, the STST family of models (Bowman & Wyble 2007; Wyble, Potter, Bowman & Nieuwenstein 2011) proposes mechanisms for temporal segmentation of continuous input. The conclusion of this work is that the visual system has mechanisms for rapidly creating brief episodes of attention that highlight important moments in time, and also separates each episode from temporally adjacent neighbors to benefit learning. Next, the RAGNAROC model (Wyble et al. 2019) describes a decision process for determining the spatial focus (or foci) of attention in a spatiotopic field and the neural mechanisms that provide enhancement of targets and suppression of highly distracting information. This work highlights the importance of integrating behavioral and electrophysiological data to provide empirical constraints on a neurally plausible model of spatial attention. The model also highlights how a neural circuit can make decisions in a continuous space, rather than among discrete alternatives. Finally, the binding pool (Swan & Wyble 2014; Hedayati, O’Donnell, Wyble in Prep) provides a mechanism for selectively encoding specific attributes (i.e. color, shape, category) of a visual object to be stored in a consolidated memory representation. The binding pool is akin to a holographic memory system that layers representations of select latent representations corresponding to different attributes of a given object. Moreover, it can bind features into distinct objects by linking them to token placeholders. Future work looks toward combining these models into a coherent framework for understanding the full measure of on-the-fly attentional mechanisms and how they improve learning.

Many psychological theories assume that qualitatively different cognitive processes can result in identical responses. Multinomial processing tree (MPT) models allow researchers to disentangle latent cognitive processes based on observed response frequencies. Recently, MPT models have been extended to explicitly account for participant and item heterogeneity. These hierarchical Bayesian MPT models provide the opportunity to connect two traditionally isolated disciplines. Whereas cognitive psychology has often focused on the experimental validation of MPT model parameters on the group level, psychometrics provides the necessary concepts and tools for measuring differences in MPT parameters on the item or person level. Moreover, MPT parameters can be regressed on covariates to model latent processes as a function of personality traits or other person characteristics.

Interval timing is a fundamental component action, and is susceptible to motor-related temporal distortions. Previous studies have shown that movement biases temporal estimates, but have primarily considered self-modulated movement only. However, real-world encounters often include situations in which movement is restricted or perturbed by environmental factors. In the following experiments, we introduced viscous movement environments to externally modulate movement and investigated the resulting effects on temporal perception. In two separate tasks, participants timed auditory intervals while moving a robotic arm that randomly applied four levels of viscosity. Results demonstrated that higher viscosity led to shorter perceived durations. Using a drift-diffusion model and a Bayesian observer model, we confirmed these biasing effects arose from perceptual mechanisms, instead of biases in decision making. These findings suggest that environmental perturbations are an important factor in movement-related temporal distortions, and enhance the current understanding of the interactions of motor activity and cognitive processes. https://www.biorxiv.org/content/10.1101/2020.10.26.355396v1

How do people learn in real-world environments where the space of possible actions can be vast or even infinite? The study of human learning has made rapid progress in past decades, from discovering the neural substrate of reward prediction errors, to building AI capable of mastering the game of Go. Yet this line of research has primarily focused on learning through repeated interactions with the same stimuli. How are humans able to rapidly adapt to novel situations and learn from such sparse examples? I propose a theory of how generalization guides human learning, by making predictions about which unobserved options are most promising to explore. Inspired by Roger Shepard’s law of generalization, I show how a Bayesian function learning model provides a mechanism for generalizing limited experiences to a wide set of novel possibilities, based on the simple principle that similar actions produce similar outcomes. This model of generalization generates predictions about the expected reward and underlying uncertainty of unexplored options, where both are vital components in how people actively explore the world. This model allows us to explain developmental differences in the explorative behavior of children, and suggests a general principle of learning across spatial, conceptual, and structured domains.

Reliable and validated measures of emotion in animals are of great import; they are crucial to better understanding and developing treatments for human mood disorders, and they are necessary for ensuring good animal welfare. We have developed a novel measure of emotion in animals that is grounded in theory and psychological research – decision-making under ambiguity. Specifically, we consider that more ‘optimistic’ decisions about ambiguous stimuli reflect more positive emotional states, while the opposite is true for more ‘pessimistic’ decisions. In this talk, we will outline the background behind and implementation of this measure, meta-analyses that have been conducted to validate the measure, and discuss how computational modelling has been used to further understand the cognitive processes underlying ‘optimistic’ and ‘pessimistic’ decision-making as an indicator of animal emotion and welfare.

Framing effects in individual decision-making have puzzled economists for decades because they are hard, if at all, to explain with rational choice theories. Why should mere changes in the description of a choice problem affect decision-making? Here, we examine the hypothesis that changes in framing cause changes in the allocation of attention to the different options – measured via eye-tracking – and give rise to changes in decision-making. We document that the framing of a sure alternative as a gain – as opposed to a loss – in a risk-taking task increases the attentional advantage of the sure option and induces a higher choice frequency of that option – a finding that is predicted by the attentional drift-diffusion model (aDDM). The model also correctly predicts other key findings such as that the increased attentional advantage of the sure option in the gain frame should also lead quicker decisions in this frame. In addition, the data reveal that increasing risk aversion at higher stake sizes may also be driven by attentional processes because the sure option receives significantly more attention – regardless of frame – at higher stakes. We also corroborate the causal impact of framing-induced changes of attention on choice with an additional experiment that manipulates attention exogenously. Finally, to study the precise mechanisms underlying the framing effect we structurally estimate an aDDM that allows for frame and option-dependent parameters. The estimation results indicate that – in addition to the direct effects of framing-induced changes in attention on choice – the gain frame also causes (i) an increase in the attentional discount of the gamble and (ii) an increased concavity of utility. Our findings suggest that the traditional explanation of framing effects in risky choice in terms of a more concave value function in the gain domain is seriously incomplete and that attentional mechanisms as hypothesized in the aDDM play a key role.

Cognitive architectures are attempts to build larger-scale models of minds. This talk will explore how structure-mapping models of analogical matching, retrieval, and generalization are used in the Companion cognitive architecture. Examples will include modeling conceptual change, learning by reading, and analogical Q/A training.