Pong
pong
Learning Relational Rules from Rewards
Humans perceive the world in terms of objects and relations between them. In fact, for any given pair of objects, there is a myriad of relations that apply to them. How does the cognitive system learn which relations are useful to characterize the task at hand? And how can it use these representations to build a relational policy to interact effectively with the environment? In this paper we propose that this problem can be understood through the lens of a sub-field of symbolic machine learning called relational reinforcement learning (RRL). To demonstrate the potential of our approach, we build a simple model of relational policy learning based on a function approximator developed in RRL. We trained and tested our model in three Atari games that required to consider an increasingly number of potential relations: Breakout, Pong and Demon Attack. In each game, our model was able to select adequate relational representations and build a relational policy incrementally. We discuss the relationship between our model with models of relational and analogical reasoning, as well as its limitations and future directions of research.
Anatomical and functional characterization of the neuronal circuits underlying ejaculation
During sexual behavior, copulation related sensory information and modulatory signals from the brain must be integrated and converted into the motor and secretory outputs that characterize ejaculation (Lenschow and Lima, Current Opinion in Neurobiology, 2020). Studies in humans and rats suggest the existence of interneurons in the lumbar spinal cord that mediates that step: the spinal ejaculation generator (SEG). My work aimed at gaining mechanistic insights about the neuronal circuits controlling ejaculation thereby applying cutting-edge techniques. More specifically, we mapped anatomically and functionally the spinal circuit for ejaculation starting from the main muscle being involved in sperm expulsion: the bulbospongiosus muscle (BSM). Combining viral tracing strategies with electrophysiology, we specifically show that the BSM motoneurons receive direct synaptic input from a group of interneurons located in between lumbar segment 2 and 3 and expressing the peptide galanin. Electrically and optogenetically activating the galanin positive cells (the SEG) lead to the activation of the motoneurons innervating the BSM and the muscle itself. Finally, inhibition of SEG cells using DREADDs (Designer Receptors Exclusively Activated by Designer Drugs) in sexual behaving animals is currently conducted to reveal whether ejaculation can be prevented.
Advances and setbacks in prion biology
Transmissible spongiform encephalopathies (TSEs) are neurodegenerative diseases of humans and many animal species caused by prions. The main constituent of prions is PrPSc, an aggregated moiety of the host-derived membrane glycolipoprotein PrPC. Prions were found to encipher many phenotypic, genetically stable TSE variants. The latter is very surprising, since PrPC is encoded by the host genome and all prion strains share the same amino acid sequence. Here I will review what is known about the infectivity, the neurotoxicity, and the neuroinvasiveness of prions. Also, I will explain why I regard the prion strain question as a fascinating challenge – with implications that go well beyond prion science. Finally, I will report some recent results obtained in my laboratory, which is attempting to address the strain question and some other basic issues of prion biology with a “systems” approach that utilizes organic chemistry, photophysics, proteomics, and mouse transgenesis.
Preschoolers' Comprehension of Functional Metaphors
Previous work suggests that children’s ability to understand metaphors emerges late in development. Researchers argue that children’s initial failure to understand metaphors is due to an inability to reason about shared relational structures between concepts. However, recent work demonstrates that preschoolers, toddlers, and even infants are already capable of relational reasoning. Might preschoolers also be capable of understanding metaphors, given more sensitive experimental paradigms? I explore whether preschoolers (N = 200, ages 4-5) understand functional metaphors, namely metaphors based on functional similarities. In Experiment 1a, preschoolers rated functional metaphors (e.g. “Roofs are hats”; “Clouds are sponges”) as “smarter” than nonsense statements. In Experiment 1b, adults (N = 48) also rated functional metaphors as “smarter” than nonsense statements (e.g. “Dogs are scissors”; “Boats are skirts”). In Experiment 2, preschoolers preferred functional explanations (e.g. “Both hold water”) over perceptual explanations (e.g. “Both are fluffy”) when interpreting a functional metaphor (e.g. “Clouds are sponges”). In Experiment 3, preschoolers preferred functional metaphors over nonsense statements in a dichotomous-choice task. Overall, this work demonstrates preschoolers’ early-emerging ability to understand functional metaphors.
Learning Neurobiology with electric fish
Electric Gymnotiform fish live in muddy, shallow waters near the shore – hiding in the dense filamentous roots of floating plants such as Eichornia crassipes (“camalote”). They explore their surroundings by using a series of electric pulses that serve as self emitted carrier of electrosensory signals. This propagates at the speed of light through this spongiform habitat and is barely sensed by the lateral line of predators and prey. The emitted field polarizes the surroundings according to the difference in impedance with water which in turn modifies the profile of transcutaneous currents considered as an electrosensory image. Using this system, pulse Gymnotiformes create an electrosensory bubble where an object’s location, impedance, size and other characteristics are discriminated and probably recognized. Although consciousness is still not well-proven, cognitive functions as volition, attention, and path integration have been shown. Here I will summarize different aspects of the electromotor electrosensory loop of pulse Gymnotiforms. First, I will address how objects are polarized with a stereotyped but temporospatially complex electric field, consisting of brief pulses emitted at regular intervals. This relies on complex electric organs quasi periodically activated through an electromotor coordination system by a pacemaker in the medulla. Second, I will deal with the imaging mechanisms of pulse gymnotiform fish and the presence of two regions in the electrosensory field, a rostral region where the field time course is coherent and field vector direction is constant all along the electric organ discharge and a lateral region where the field time course is site specific and field vector direction describes a stereotyped 3D trajectory. Third, I will describe the electrosensory mosaic and their characteristics. Receptor and primary afferents correspond one to one showing subtypes optimally responding to the time course of the self generated pulse with a characteristic train of spikes. While polarized objects at the rostral region project their electric images on the perioral region where electrosensory receptor density, subtypes and central projection are maximal, the image of objects on the side recruit a single type of scattered receptors. Therefore, the rostral mosaic has been likened to an electrosensory fovea and its receptive field referred to as foveal field. The rest of the mosaic and field are referred to as peripheral. Finally, I will describe ongoing work on early processing structures. I will try to generate an integrated view, including anatomical and functional data obtained in vitro, acute experiments, and unitary recordings in freely moving fish. We have recently shown have shown that these fish tract allo-generated fields and the virtual fields generated by nearby objects in the presence of self-generated fields to explore the nearby environment. These data together with the presence of a multimodal receptor mosaic at the cutaneous surface particularly surrounding the mouth and an important role of proprioception in early sensory processing suggests the hypothesis that the active electrosensory system is part of a multimodal haptic sense.
Characterization of transgenic mouse lines overexpressing the ovine prion protein using well-defined scrapie and bovine spongiform encephalopathy strains
FENS Forum 2024