Natural language understanding comprises a wide range of diverse tasks such as textual entailment, question answering, semantic similarity assessment, and document classification. Although large unlabeled text corpora are abundant, labeled data for learning these specific tasks is scarce, making it challenging for discriminatively trained models to perform adequately. We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task. In contrast to previous approaches, we make use of task-aware input transformations during fine-tuning to achieve effective transfer while requiring minimal changes to the model architecture. We demonstrate the effectiveness of our approach on a wide range of benchmarks for natural language understanding. Our general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, significantly improving upon the state of the art in 9 out of the 12 tasks studied. For instance, we achieve absolute improvements of 8.9% on commonsense reasoning (Stories Cloze Test), 5.7% on question answering (RACE), and 1.5% on textual entailment (MultiNLI).

Coping with threatening situations requires both identifying stimuli predicting danger and selecting adaptive behavioral responses in order to survive. The dorso medial prefrontal cortex (dmPFC) is a critical structure involved in the regulation of threat-related behaviour, yet it is still largely unclear how threat-predicting stimuli and defensive behaviours are associated within prefrontal networks in order to successfully drive adaptive responses. Over the past years, we used a combination we used a combination of extracellular recordings, neuronal decoding approaches, and state of the art optogenetic manipulations to identify key neuronal elements and mechanisms controlling defensive fear responses. I will present an overview of our recent work ranging from analyses of dedicated neuronal types and oscillatory and synchronization mechanisms to artificial intelligence approaches used to decode the activity or large population of neurons. Ultimately these analyses allowed the identification of high dimensional representations of defensive behavior unfolding within prefrontal networks.

Every thought is associated with a unique pattern of brain activity. Thus, in principle, it should be possible to use these activity patterns as "brain fingerprints" for different thoughts and to read out what a person is thinking based on their brain activity alone. Indeed, using machine learning considerable progress has been made in such "brainreading" in recent years. It is now possible to decode which image a person is viewing, which film sequence they are watching, which emotional state they are in or which intentions they hold in mind. This talk will provide an overview of the current state of the art in brain reading. It will also highlight the main challenges and limitations of this research field. For example, mathematical models are needed to cope with the high dimensionality of potential mental states. Furthermore, the ethical concerns raised by (often premature) commercial applications of brain reading will also be discussed.

Two-photon (2P) in vivo functional imaging of genetically encoded fluorescent Ca2+indicators (GECIs) for neuronal activity has become a broadly applied standard tool in modern neuroscience, because it allows simultaneous imaging of the activity of many neurons at high spatial resolution within living animals. Unfortunately, the most commonly used light-sources – tunable femtosecond pulsed ti:sapphire lasers – can be prohibitively expensive for many labs and fall short of delivering sufficient powers for some new ultra-fast 2P microscopy modalities. Inexpensive homebuilt or industrial light sources such as Ytterbium fiber lasers (YbFLs) show great promise to overcome these limitations as they are becoming widely available at costs orders of magnitude lower and power outputs of up to many times higher than conventional ti:sapphire lasers. However, these lasers are typically bound to emitting a single wavelength (i.e., not tunable) centered around 1020-1060 nm, which fails to efficiently excite state of the art green GECIs such as jGCaMP7 or 8. To this end, we designed and characterized spectral variants (yellow CaMP = YCaMP) of the ultrasensitive genetically encoded calcium indicator jGCaMP7, that allows for efficient 2P-excitation at wavelengths above 1010nm. In this talk I will give a brief overview over some of the reasons why using a fiber laser for 2P excitation might be right for you. I will talk about the development of jYCaMP and some exciting new experimental avenues that it has opened while touching on the prospect that shifting biosensors yellow could have for the 2P imaging community. Please join me for an interesting and fun discussion on whether “yellow is the new green” after the talk!

In this talk, I aim to show that the dynamical properties of emerging nanodevices can accelerate the development of smart, and environmentally friendly chips that inherently learn through their physics. The goal of neuromorphic computing is to draw inspiration from the architecture of the brain to build low-power circuits for artificial intelligence. I will first give a brief overview of the state of the art of neuromorphic computing, highlighting the opportunities offered by emerging nanodevices in this field, and the associated challenges. I will then show that the intrinsic dynamical properties of these nanodevices can be exploited at the device and algorithmic level to assemble systems that infer and learn though their physics. I will illustrate these possibilities with examples from our work on spintronic neural networks that communicate and compute through their microwave oscillations, and on an algorithm called Equilibrium Propagation that minimizes both the error and energy of a dynamical system.

Designing artificially intelligent systems and interfaces with socio-emotional skills is a challenging task. Progress in industry and developments in academia provide us a positive outlook, however, the artificial social and emotional intelligence of the current technology is still limited. My lab’s research has been pushing the state of the art in a wide spectrum of research topics in this area, including the design and creation of new datasets; novel feature representations and learning algorithms for sensing and understanding human nonverbal behaviours in solo, dyadic and group settings; designing longitudinal human-robot interaction studies for wellbeing; and investigating how to mitigate the bias that creeps into these systems. In this talk, I will present some of my research team’s explorations in these areas including social appropriateness of robot actions, virtual reality based cognitive training with affective adaptation, and bias and fairness in data-driven emotionally intelligent systems.

The European University for Brain and Technology, NeurotechEU, is opening its doors on the 16th of December. From health & healthcare to learning & education, Neuroscience has a key role in addressing some of the most pressing challenges that we face in Europe today. Whether the challenge is the translation of fundamental research to advance the state of the art in prevention, diagnosis or treatment of brain disorders or explaining the complex interactions between the brain, individuals and their environments to design novel practices in cities, schools, hospitals, or companies, brain research is already providing solutions for society at large. There has never been a branch of study that is as inter- and multi-disciplinary as Neuroscience. From the humanities, social sciences and law to natural sciences, engineering and mathematics all traditional disciplines in modern universities have an interest in brain and behaviour as a subject matter. Neuroscience has a great promise to become an applied science, to provide brain-centred or brain-inspired solutions that could benefit the society and kindle a new economy in Europe. The European University of Brain and Technology (NeurotechEU) aims to be the backbone of this new vision by bringing together eight leading universities, 250+ partner research institutions, companies, societal stakeholders, cities, and non-governmental organizations to shape education and training for all segments of society and in all regions of Europe. We will educate students across all levels (bachelor’s, master’s, doctoral as well as life-long learners) and train the next generation multidisciplinary scientists, scholars and graduates, provide them direct access to cutting-edge infrastructure for fundamental, translational and applied research to help Europe address this unmet challenge.

50 years ago it was found that sleep somehow made memories better and more permanent, but neither sleep nor memory researchers knew enough about sleep and memory to devise robust, effective tests. Today the fields of sleep and memory have grown and what is now understood is astounding. Still, great mysteries remain. What is the functional difference between the subtly different slow oscillation vs the slow wave of sleep and do they really have opposite memory consolidation effects? How do short spindles (e.g. <0.5 s as in schizophrenia) differ in function from longer ones and are longer spindles key to integrating new memories with old? Is the nesting of slow oscillations together with sleep spindles and hippocampal ripples necessary? What happens if all else is fine but the neurochemical environment is altered? Does sleep become maladaptive and “cement” memories into the hippocampal warehouse where they are assembled, together with all of their emotional baggage? Does maladaptive sleep underlie post-traumatic stress disorder and other stress-related disorders? How do we optimize sleep characteristics for top emotional and cognitive function? State of the art findings and current hypotheses will be presented.