Research in the brain and cognitive sciences attempts to uncover the neural mechanisms underlying intelligent behavior in domains such as vision. Due to the complexities of brain processing, studies necessarily had to start with a narrow scope of experimental investigation and computational modeling. I argue that it is time for our field to take the next step: build system models that capture a range of visual intelligence behaviors along with the underlying neural mechanisms. To make progress on system models, we propose integrative benchmarking – integrating experimental results from many laboratories into suites of benchmarks that guide and constrain those models at multiple stages and scales. We show-case this approach by developing Brain-Score benchmark suites for neural (spike rates) and behavioral experiments in the primate visual ventral stream. By systematically evaluating a wide variety of model candidates, we not only identify models beginning to match a range of brain data (~50% explained variance), but also discover that models’ brain scores are predicted by their object categorization performance (up to 70% ImageNet accuracy). Using the integrative benchmarks, we develop improved state-of-the-art system models that more closely match shallow recurrent neuroanatomy and early visual processing to predict primate temporal processing and become more robust, and require fewer supervised synaptic updates. Taken together, these integrative benchmarks and system models are first steps to modeling the complexities of brain processing in an entire domain of intelligence.

We all have a sense of time. Yet it is a particularly intangible sensation. So how is our “sense” of time represented in the brain? Functional neuroimaging studies have consistently identified a network of regions, including Supplementary Motor Area and basal ganglia, that are activated when participants make judgements about the duration of currently unfolding events. In parallel, left parietal cortex and cerebellum are activated when participants predict when future events are likely to occur. These structures are activated by temporal processing even when task goals are purely perceptual. So why should the perception of time be represented in regions of the brain that have more traditionally been implicated in motor function? One possibility is that we learn about time through action. In other words, action could provide the functional scaffolding for learning about time in childhood, explaining why it has come to be represented in motor circuits of the adult brain.

Playing action video games involves both explicit (conscious) and implicit (non-conscious) expectations of timed events, such as the appearance of foes. While studies revealed that explicit attention skills are improved in action video game players (VGPs), their implicit skills remained untested. To this end, we investigated explicit and implicit temporal processing in VGPs and non-VGPs (control participants). In our variable foreperiod task, participants were immersed in a virtual reality and instructed to respond to a visual target appearing at variable delays after a cue. I will present behavioral, oculomotor and EEG data and discuss possible markers of the implicit passage of time and explicit temporal attention processing. All evidence indicates that VGPs have enhanced implicit skills to track the passage of time, which does not require conscious attention. Thus, action video game play may improve a temporal processing found altered in psychopathologies, such as schizophrenia. Could digital (game-based) interventions help remediate temporal processing deficits in psychiatric populations?

Estimating intervals is essential for adaptive behavior and decision-making. Although several theoretical models have been proposed to explain how the brain keeps track of time, there is still no evidence toward a single one. It is often hard to compare different models due to their overlap in behavioral predictions. For this reason, several studies have looked for neural signatures of temporal processing using methods such as electrophysiological recordings (EEG). However, for this strategy to work, it is essential to have consistent EEG markers of temporal processing. In this talk, I'll present results from several studies investigating how temporal information is encoded in the EEG signal. Specifically, across different experiments, we have investigated whether different neural signatures of temporal processing (such as the CNV, the LPC, and early ERPs): 1. Depend on the task to be executed (whether or not it is a temporal task or different types of temporal tasks); 2. Are encoding the physical duration of an interval or how much longer/shorter an interval is relative to a reference. Lastly, I will discuss how these results are consistent with recent proposals that approximate temporal processing with decisional models.

The theory of predictive processing posits that the brain computes expectations to process information predictively. Empirical evidence in support of this theory, however, is scarce and largely limited to sensory areas. Here, we report a precise and adaptive mechanism in the frontal cortex of non-human primates consistent with predictive processing of temporal events. We found that the speed of neural dynamics is precisely adjusted according to the average time of an expected stimulus. This speed adjustment, in turn, enables neurons to encode stimuli in terms of deviations from expectation. This lawful relationship was evident across multiple experiments and held true during learning: when temporal statistics underwent covert changes, neural responses underwent predictable changes that reflected the new mean. Together, these results highlight a precise mathematical relationship between temporal statistics in the environment and neural activity in the frontal cortex that may serve as a mechanism for predictive temporal processing.

Medium-spiny neurons (MSNs) in the striatum are required for interval timing, or the estimation of the time over several seconds via a motor response. We and others have shown that striatal MSNs can encode the duration of temporal intervals via time-dependent ramping activity, progressive monotonic changes in firing rate preceding behaviorally salient points in time. Here, we investigated how timing-related activity within striatal ensembles changes with experience. We leveraged a rodent-optimized interval timing task in which mice ‘switch’ response ports after an amount of time has passed without reward. We report three main results. First, we found that the proportion of MSNs exhibiting time-dependent modulations of firing rate increased after 10 days of task overtraining. Second, temporal decoding by MSN ensembles increased with experience and was largely driven by time-related ramping activity. Finally, we found that time-related ramping activity generalized across both correct and error trials. These results enhance our understanding of striatal temporal processing by demonstrating that time-dependent activity within MSN ensembles evolves with experience and is dissociable from motor- and reward-related processes.

Olfactory navigation provides a tractable model for studying the circuit basis of sensori-motor transformations and goal-directed behaviour. Macroscopic organisms typically navigate in odor plumes that provide a noisy and uncertain signal about the location of an odor source. Work in many species has suggested that animals accomplish this task by combining temporal processing of dynamic odor information with an estimate of wind direction. Our lab has been using adult walking Drosophila to understand both the computational algorithms and the neural circuits that support navigation in a plume of attractive food odor. We developed a high-throughput paradigm to study behavioural responses to temporally-controlled odor and wind stimuli. Using this paradigm we found that flies respond to a food odor (apple cider vinegar) with two behaviours: during the odor they run upwind, while after odor loss they perform a local search. A simple computational model based one these two responses is sufficient to replicate many aspects of fly behaviour in a natural turbulent plume. In on-going work, we are seeking to identify the neural circuits and biophysical mechanisms that perform the computations delineated by our model. Using electrophysiology, we have identified mechanosensory neurons that compute wind direction from movements of the two antennae and central mechanosensory neurons that encode wind direction are are involved in generating a stable downwind orientation. Using optogenetic activation, we have traced olfactory circuits capable of evoking upwind orientation and offset search from the periphery, through the mushroom body and lateral horn, to the central complex. Finally, we have used optogenetic activation, in combination with molecular manipulation of specific synapses, to localize temporal computations performed on the odor signal to olfactory transduction and transmission at specific synapses. Our work illustrates how the tools available in fruit fly can be applied to dissect the mechanisms underlying a complex goal-directed behaviour.