vision

CAREER: Frontiers of Knowledge in Foundation Models

Foundation models are large artificial intelligence (AI) systems trained on vast amounts of data to perform a wide range of tasks, including answering questions, generating content, and assisting decision-making. These models are increasingly used in areas that affect everyday life, such as healthcare, education, and environmental planning. However, despite their impressive capabilities, they often rely on patterns and correlations in data rather than true causal relationships. This limitation can lead to unreliable or misleading outputs, especially in high-stakes situations. For example, a model may incorrectly assume that one factor causes another simply because they frequently appear together in data. This project addresses this critical challenge by enabling foundation models to better understand and use causal knowledge, which describes how one factor directly influences another in the real world. By improving the ability of these models to reason about cause and effect, the project aims to make them more reliable, transparent, and aligned with human reasoning. The results will support safer and more effective use of AI in important societal domains, strengthen decision-making in complex environments, and contribute to education and workforce development by training students in emerging areas of trustworthy AI. This project develops a systematic framework for understanding, leveraging, editing, and applying causal knowledge in foundation models, organized into four complementary thrusts. The first thrust introduces methods to interpret causal relationships embedded within large-scale models by analyzing internal components that encode causal knowledge across language, vision, and multimodal systems. The second thrust designs approaches to incorporate external causal knowledge into model reasoning, improving performance in tasks such as question answering, causal reasoning, and video understanding. The third thrust establishes techniques for editing causal knowledge within models, enabling targeted updates to specific relationships while preserving overall model performance and consistency. The fourth thrust focuses on empirical evaluation and application of the proposed methods across diverse application domains, including healthcare, materials science, and environmental systems. The project integrates research with education through curriculum development, student mentoring, and outreach activities, and produces open-source tools and resources to support broader adoption. Together, these efforts advance the development of interpretable, controllable, and generalizable foundation models grounded in a causal perspective. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

CAREER: Structure-Aware Learning from Weak Supervision for Knowledge Acquisition

Knowledge acquisition—the ability of artificial intelligence (AI) systems to extract actionable insights from vast amounts of unstructured text—is critical for advancements in healthcare, education, and scientific discovery. While Large Language Models (LLMs) have shown impressive capabilities, their reliability depends heavily on massive, perfectly curated datasets, which are expensive and often unavailable in specialized domains. This CAREER project addresses this bottleneck by developing a new paradigm called “structure-aware weak supervision.” Instead of relying on perfect human annotations, the project enables AI systems to learn autonomously from incomplete, noisy, and ambiguous data by discovering and utilizing underlying semantic structures, such as concept hierarchies and retrieval pathways. By reducing the dependency on expensive labeled data, this research democratizes the development of highly accurate, domain-specific AI tools for resource-constrained environments, such as public health agencies and community organizations. The project also integrates these research outcomes into new undergraduate and graduate curricula, open-source educational toolkits, and targeted K-12 outreach programs designed to broaden participation in computing and teach the next generation how to build reliable, human-centered AI systems. This project proposes a unified framework for learning under weak supervision by bridging unstructured language data with structured, interpretable knowledge representations. The research is organized into three synergistic thrusts. Thrust 1 tackles incomplete supervision by inducing latent ontologies from unlabeled corpora via a novel Spherical Hierarchical Expectation-Maximization (SHEM) algorithm, enabling scalable information extraction and classification without predefined schemas. Thrust 2 addresses noisy supervision by designing a Denoising Retrieval-Augmented Generation (DeRAG) framework. It integrates symbolic reasoning over the induced ontologies with Structure-Aware Contrastive Retrieval (SACRet) to actively filter distractors and reliably ground language model outputs. Thrust 3 tackles ambiguous supervision by modeling complex, multi-faceted human preferences. It introduces a Tree of Reward Models (TreeRM) and Hierarchical Dirichlet Thompson Sampling (HDTS) to capture both shared foundational values (e.g., safety, factuality) and personalized user preferences (e.g., tone), ensuring robust AI alignment. Together, these contributions advance the theoretical foundations and practical methodologies of knowledge-centric AI, creating systems that autonomously construct knowledge, dynamically adapt to supervision gaps, and reliably align with hierarchical human values. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Baby Toolbox Training and Certification Program

PROJECT SUMMARY Our objective is to improve early childhood outcomes and support the expansion of the NIH Infant and Toddler Toolbox (Baby Toolbox) by providing comprehensive training support to those interested in using it. The Baby Toolbox is a brand new, nationally-normed assessment for infants 1-42 months, commissioned by NICHD and released for public use in 2025. The Baby Toolbox is administered entirely on an iPad and includes 35 measures across six domains using novel technology (e.g., gaze tracking, automatic scoring, computerized adaptive testing). It has the potential to bring harmonization to the developmental fields, but in order for it to become a common currency for developmental research as envisioned, researchers need to know how to administer it and how to train others to administer it. We propose an education program that will include a week-long training workshop, certification activities, and post-workshop support to create expert cohorts of Baby Toolbox test administrators. Individuals who attend the workshops can become certified test trainers, capable of training others at their home institutions to administer the assessment thus creating a self-sufficient training model. Through the proposed educational program, we will provide funding to cover lodging, meals, and incidentals during the workshop, in addition to subsidizing transportation to/from the workshop and provide a one-year subscription to the Baby Toolbox. A portion of slots will also be set aside for those without current grant funding. Our team is highly qualified to complete these tasks because we have led the effort to develop the Baby Toolbox assessment and have already completed multiple training workshops for contract deliverables. This grant would continue the efforts started by the NICHD in funding the Baby Toolbox by helping support its rollout, implementation, and growth. To meet these goals, we have the following aims: Aim 1: Create cohorts of trained Baby Toolbox examiners who can catapult the Baby Toolbox into widespread use by hosting a comprehensive week-long education program (training workshop) yearly for individuals to learn how to administer and train others to administer the Baby Toolbox, Aim 2: Expand the use of the Baby Toolbox by recruiting and financially supporting individuals who will bring the Baby Toolbox into a variety of research and clinical settings. Aim 3: Build a virtual training resource of videos and materials to support ongoing fidelity checks with certified trainers, and future training efforts.

Mentoring investigators in patient-oriented research on HIV and public health

PROJECT SUMMARY/ABSTRACT Despite marked progress in treatment and prevention, HIV remains a significant public health threat in the US and globally. Innovative strategies are needed to effectively deploy interventions and reduce HIV incidence, which requires a sustained and committed workforce. Dr. Dennis is an infectious disease physician and researcher at the University of North Carolina (UNC) at Chapel Hill, Division of Infectious Diseases. She seeks the protected time of the K24 award to ensure adequate time and effort to provide mentorship in patient- oriented HIV research focused on applied public health strategies. Dr. Dennis has a track record of performing high-quality patient-oriented research supported by independent funding. Her research bridges basic, clinical, and epidemiologic science by using HIV-1 molecular epidemiology and phylogenetics to understand HIV transmission at the population level and to use this information to direct prevention. She has expanded this work to optimize strategies to detect and respond to HIV networks using mixed-methods approaches. The overall goal of this work is to uncover the links between these sub-epidemics - which are overlapping sub- epidemics defined by risk groups, geography, social interaction - to facilitate the design of timely, effective interventions. The research specific aims are 1) Investigate HIV transmission networks using molecular epidemiology and phylodynamics (R01AI135970), 2) Evaluate uptake of HIV treatment and prevention services in public health with social network approaches (supported by R01AI169602), and 3) Pilot a network-based characterization of early syphilis infections to inform strategies to increase the uptake of injectable antiretrovirals for HIV treatment and prevention (supported by K24). With the support of the K24, she will leverage resources at UNC to support mentorship and professional development to strengthen new directions (implementation science, community-engaged research). Dr. Dennis is deeply committed to expanding her mentorship and dedicated to fostering diverse mentees with lived experiences that are critical for sustaining the HIV workforce. Dr. Dennis is Co-Director of the UNC Center for AIDS Research (CFAR) Scientific Working Group which focuses on Ending the HIV Epidemic efforts in North and South Carolina. She has strong institutional support and a multidisciplinary team of advisors, including the UNC CFAR, and is an advisor on the UNC T32 HIV/STI institutional training program. She has collaborated for the past 10 years with NC Division of Public Health and with multiple investigators and trainees at the UNC Gillings School of Public Health. She is active in the UNC Infectious Diseases Fellowship program, providing clinical and research mentorship to numerous ID fellows. Her clinical activity provides practical grounding and relevance in patient-oriented research. The K24 will provide 50% of Dr. Dennis’ salary and additional funds to support mentees’ research. The proposed research is timely and aligned with the National HIV/AIDS Strategy and will support the protected time needed to mentor the next-generation of investigators in HIV patient-oriented research.

Factory-treated, long-lasting permethrin baby wraps for the prevention of malaria: A phase III randomized controlled trial

PROJECT SUMMARY/ABSTRACT Progress against malaria has stalled. Novel interventions – particularly those targeting outdoor and daytime biting – are needed. In a randomized, placebo-controlled trial of permethrin- vs. sham-treated baby wraps in Uganda, we found a significant reduction in clinical malaria incidence among children carried in permethrin- as compared to sham-treated wraps (Boyce et al, NEJM, 2025). Despite these promising results, our trial incorporated a monthly re-treatment strategy that would be difficult to operationalize at scale. Furthermore, we only followed participants for 6 months, which is shorter than the expected period of use. Therefore, implementation studies - and specifically trials of long-lasting, factory-treated textiles - are now needed. Factory-treated materials would not only eliminate the need for retreatment for up to 12 months, but because the chemicals are more tightly bound, result in less absorption across the skin. Therefore, we now propose to conduct a randomized, double-blind trial of factory-treated, long-lasting (FTLL) wraps. AIM 1: Determine the effectiveness of FTLL permethrin wraps in combination with existing interventions for the prevention of malaria in children. We will enroll 750 mother-infant pairs from routine immunization visits (~3 months of age) at 3 sites of varying transmission intensity across Uganda. All participants will receive new dual active ingredient (AI) bed nets and be randomized (1:1) to either FTLL or untreated wraps. The primary outcome will be clinical malaria incidence during the period of wrap use, defined as fever a positive malaria rapid diagnostic test (RDT) between the FTLL and untreated arms. AIM 2: Confirm the safety of extended exposure to FTLL permethrin wraps for use in young children. Although a review of factory-treated clothing by the US Environmental Protection Agency, including clothing for children and toddlers, did not identify scenarios of concern, the frequency of use envisioned here may be beyond that modeled. To accomplish this, we will perform semi-annual assessments of growth (e.g., height-for-weight) and neurodevelopment (ND) during the period of use and 12-months after discontinuation. AIM 3: Assess the effect of FTLL permethrin wraps on Anopheles mosquito indices and blood-meal seeking behaviors. We will conduct longitudinal entomological surveillance, including CDC-light trap and aspirator collections, supplemented by human landing catches at sentinel households (~10-15%) from both the FTLL and untreated arms. This work tests a novel intervention, which leverages technology developed by the US military, to reduce the burden of malaria in endemic countries. Addressing malaria in these countries minimizes the risk of importation into the US. If successful, the project will provide additional evidence for treated textiles, which may be used to protect American travelers and deployed military servicemembers. The project will be conducted in Uganda, where malaria is highly endemic and it will be possible to enroll at-risk women-infant pairs.

Factors Driving Wear and Implant Failure in Total Shoulder Arthroplasty

Polyethylene (PE) wear and implant-related failure remain leading causes of revision in total shoulder arthroplasty (TSA), a procedure which now surpasses the growth rate of hip and knee arthroplasty. Both anatomic (aTSA) and reverse (rTSA) TSA outcomes are heavily influenced by complex interactions between rotator cuff function, scapular motion, implant design, and patient-specific loading—factors not adequately captured in current preclinical implant testing standards. Emerging evidence suggests that PE wear progression in TSA is highly dependent on shoulder kinematics, joint loading, implant positioning, and individual patient factors. Nonetheless, data on in vivo motion and load profiles remain sparse, and few tools exist to link these profiles to clinically relevant wear patterns or associated periprosthetic inflammatory tissue responses. Accordingly, the primary objective of this project is to develop validated, patient-specific models that predict PE wear in TSA and identify modifiable surgical, design, and rehabilitation targets to improve implant longevity and restore patient mobility. Additionally, we will establish histopathological hallmarks that indicate TSA failure caused by PE wear debris. Our central hypothesis is that specific shoulder kinematics and joint loading drive distinct PE wear patterns in TSA associated with mechanical failure or inflammatory-mediated osteolysis, depending on implant design and positioning. To achieve the overall objective of this work, shoulder motions and muscle excitations across 25 activities of daily living will be collected at pre-op and post-op (>6 months) in both aTSA and rTSA patients, with long-term follow-up of patient-reported outcomes via validated surveys (5 years). Unsupervised machine learning will categorize patients into movement-based phenotypes, which will then inform a multi-scale modeling framework to estimate in vivo shoulder joint loads and implant wear across the varying movement strategies. Predicted wear patterns will be validated using state-of-the-art preclinical wear simulators. Simultaneously, we will quantify how patient, surgical, and implant factors contribute to wear in retrieved TSA components (>400 samples), correlating imaging-based wear patterns with clinical outcomes, patient-reported function, inflammatory tissue responses, and radiographic indications of loosening. For that purpose, we will establish benchmarks of TSA wear rates and introduce a new histopathological approach augmented by infrared spectroscopic imaging. This work is innovative because we are linking patient-specific movement patterns following TSA with multi-scale computational models to predict PE wear, breaking the current approaches of using generic motions and loads in existing testing standards. This work will produce the first integrated, publicly available database of TSA kinematics, joint loading, and PE wear patterns and rates, along with validated computational tools to inform implant design, surgical planning, rehabilitation strategies, and personalized risk assessment. Ultimately, these advances will improve functional outcomes and long-term success for TSA patients and enable better preclinical testing methods and standards.

Cytoskeletal connectors: Deciphering the fundamental mechanisms of cytoskeletal dynamics and transport

PROJECT SUMMARY The cytoskeleton is a dynamic network of filamentous structures, including microtubules and actin, that regulate essential cellular processes such as cell shape, growth, and signaling. Cytoskeleton also serves as tracks for molecular motors, which transport a variety of cellular cargoes, including organelles, macromolecules, and vesicles. These cargoes are linked to motors by specialized connector proteins. Disruptions in connector proteins are implicated in a range of neurodevelopmental and neurodegenerative diseases, as well as cancers. Despite their importance, these proteins continue to be understudied, primarily due to their perceived role as passive linkers and the technical challenges in working with them. However, recent discoveries suggest that connector proteins may play more active roles, in some cases even have enzymatic functions. This proposal aims to uncover mechanisms of connector protein functions through a detailed investigation of actin-microtubule and motor-cargo interactions. Actin and microtubules are linked by the spectraplakin family of large and evolutionarily conserved proteins, critical for neuronal development and differentiation. Recent discoveries of ATPase domains within these proteins suggest they may haves beyond simply linking cytoskeletal components. One goal of this proposal is to investigate the role of spectraplakin’s ATPase domains via structural, biochemical, and cell biology approaches. Another goal is to explore how dynamic changes in motor-cargo connectors facilitate the transport of diverse cargoes along microtubule tracks. The focus will be on the cytoplasmic dynein-1 (dynein) and the connectors (adaptors) that activate and link dynein to cargo. Dynein is a microtubule minus-end directed motor that plays essential roles in cell division, and transports hundreds of different cellular cargoes. While several motor-cargo connectors have been identified, the regulatory mechanisms enabling cargo transport are not fully understood. We are investigating whether connector proteins work together to activate dynein movement and/or facilitate cargo handoff between different dynein complexes. Using innovative approaches, including time- resolved cryo-EM, complex in-vitro reconstitutions, and live-cell imaging in induced neurons, we are uncovering critical mechanisms that govern cytoskeletal connector proteins, furthering our understanding of how the cytoskeleton regulates essential cellular processes.

RET Site: Research Experiences for Teachers in Cybersecurity and Artificial Intelligence

Computing and cybersecurity skills are increasingly essential across nearly every sector of the modern economy, yet many students in Mississippi and similar rural states have limited exposure to these fields, particularly in schools with fewer resources. Teachers play a pivotal role in shaping student interest in science and technology careers, but many lack the research experience needed to bring authentic computing content into their classrooms. This project addresses that gap by immersing high school and community college educators in hands-on cybersecurity research at Mississippi State University, enabling them to return to their classrooms with deeper knowledge, stronger confidence, and ready-to-use instructional materials. By strengthening teacher expertise in computing and cybersecurity, the project generates a multiplier effect that reaches hundreds of students annually, expanding access to computing pathways in communities. The project also supports workforce development by preparing a pipeline of students who are better equipped for careers in computing, national security, and related fields that are critical to the nation's economic strength and security. This award supports a Research Experiences for Teachers (RET) site in which thirty educators participate over three years in a six-week summer research program at Mississippi State University. Participants are embedded in faculty-led research groups focused on areas such as machine learning for radio frequency interference detection, algorithmic resilience in wireless networks, software reverse engineering, adversarial attacks on robot vision systems, and cybersecurity of electric vehicle charging infrastructure. The first week provides foundational training in computing concepts, research ethics, and laboratory practices, after which participants engage directly in ongoing research projects under faculty mentorship. Beginning in the fourth week, teachers develop standards-aligned curricular modules that connect their research experiences to mathematics and computing courses at the secondary and community college level. Academic-year follow-up activities include faculty classroom visits, virtual mentoring, peer collaboration through a shared digital platform, and presentations at state and regional teacher conferences. Program effectiveness is assessed through a mixed-methods evaluation framework examining teacher self-efficacy, curricular integration, and long-term changes in classroom practice. The project is led by faculty in the Department of Computer Science and Engineering at Mississippi State University, in partnership with the Starkville Oktibbeha Consolidated School District, Hinds Community College, and others. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

REU Site: Hands-On Research in Federated Learning Security through Red Team vs. Blue Team Exercises

This Research Experiences for Undergraduates Site at the University of Nevada, Las Vegas supports 10 students each year in a 10-week summer research program on the security of federated learning, a way for many devices or organizations to train a shared artificial intelligence model without exchanging their raw data. This approach can help protect privacy, but it also creates new security risks because attackers may try to corrupt the training process, steal information from the model, or reduce system reliability. The project’s novelties are the integration of hands-on attack-and-defense research across the full federated learning process and the use of Red Team versus Blue Team exercises to study these problems in realistic settings. The project's broader significance and importance are that it advances safer privacy-preserving artificial intelligence, expands access to advanced undergraduate research opportunities, and helps prepare the future artificial intelligence and cybersecurity workforce. The project contributes to a stronger national capacity for building trustworthy data-driven systems. The research project focuses on threats and defenses in the data collection, training, and inference stages of federated learning. Students and mentors investigate representative attacks including botnet-style disruption, poisoning, backdoor insertion, privacy leakage, membership inference, and data reconstruction, and they evaluate defenses such as robust aggregation, anomaly detection, and differential privacy. The work uses a dedicated federated learning cybersecurity range, realistic datasets from computer vision, language, and network traffic applications, and distributed computing resources for controlled experimentation. Through iterative Red Team and Blue Team studies, the project produces software, tutorials, datasets, and empirical results that improve understanding of secure and privacy-preserving distributed learning. The anticipated outcome is stronger technical foundations for trustworthy artificial intelligence and a broader pipeline of students prepared for research and professional practice in cybersecurity and artificial intelligence. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

REU Site: Undergraduate Research Experience in Edge Intelligence

This project establishes a Research Experiences for Undergraduates (REU) site at the University of Nevada, Reno, to provide undergraduate students nationwide with immersive research experiences in the field of edge intelligence. Edge intelligence, where data collection, processing, and analysis are performed close to the data source rather than on centralized cloud-based systems, is a rapidly advancing paradigm critical for applications such as autonomous vehicles, smart cities, industrial automation, and healthcare systems. However, deploying artificial intelligence (AI) on resource-limited edge devices presents substantial challenges in accuracy, efficiency, and robustness, with significant implications for public health, national security, and economic competitiveness. By engaging students in hands-on research and equipping them with the skills to address these challenges, this project aims to develop a highly skilled STEM workforce capable of advancing practical AI technologies. The program will foster sustained interest in research and encourage participants to pursue graduate studies and careers in STEM. Through the development of accurate, efficient, and trustworthy AI systems, the project will contribute to technological innovation, strengthen public trust in AI, and support societal well-being and national security. This REU site develops undergraduate research capacity in edge intelligence by leveraging the interdisciplinary expertise of University of Nevada, Reno faculty in AI, robotics, and cyber-physical systems. The research is structured around three core thrusts in edge-based machine learning (ML) systems: (1) algorithms and architectures for resource-constrained devices, (2) scalable and robust learning in heterogeneous edge environments, and (3) resource management for energy-efficient and sustainable operation. Representative projects include accelerating federated learning in heterogeneous wireless networks, developing memory-efficient training of large language models for edge devices, designing decentralized federated learning with workload balancing, federated fine-tuning of vision models for wildfire detection, and collaborative multi-robot deep reinforcement learning. Guided by experienced faculty and graduate student mentors, students will engage in cutting-edge research and contribute to the development of ML systems for real-world applications such as smart Internet-of-Things devices, autonomous vehicles, and robotics. The program is complemented by training in research methodology, scientific communication, and career development, preparing participants for graduate study and careers in STEM fields. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

REU Site: Frontier Technologies for Biometrics and Authentication

Biometrics, the science of recognizing individuals through physical and behavioral traits such as fingerprints, face, iris, voice, and gait, is increasingly important for securing access to devices, facilities, and services. Applications range from smartphone authentication and border security to national identification systems and financial account protection. As biometrics replaces traditional passwords and becomes the preferred authentication approach in everyday scenarios, the demand for scientists and engineers with biometric computing skills continues to grow rapidly. Despite this demand, undergraduate students have limited opportunities to gain hands-on research experience in this critical area. This Research Experiences for Undergraduates (REU) Site, hosted by the Department of Computer Science and Engineering at the University at Buffalo (UB), will address that gap by providing intensive summer research training in biometrics and authentication to undergraduate students recruited from across the nation. Through mentored research projects, hands-on workshops, seminars, field trips, professional development activities, and project demonstrations, participants will build practical research skills while exploring technologies with direct applications in national security, healthcare, and consumer electronics. The program will recruit students nationwide through broad outreach open to all eligible participants, with emphasis on reaching students from institutions with limited research infrastructure, economically disadvantaged backgrounds, and first-generation college students. The program will contribute to building a skilled cybersecurity and AI workforce prepared to meet pressing national needs. This REU Site on Frontier Technologies in Authentication and Biometrics will support ten undergraduate researchers per year for three years. Despite considerable advances, there remain unresolved challenges regarding the effectiveness and security of biometric recognition systems, including the introduction of new biometric modalities, anti-spoofing measures, system cancelability, continuous authentication, and societal acceptance. Under the long-term research vision of biometrics and authentication, this project will focus on two fundamental directions: biometric modality and biometric security. Research projects will be built on six foundational areas, including sensors and hardware, pattern recognition, machine learning, computer and network security, human-computer interaction, and usability. Specific projects will address topics such as micro-expression biometric algorithms using video and physiological data, three-dimensional finger vein imaging via near-infrared sensing, fingerprint security practice using phantom finger models, continuous biometric authentication via electrocardiography, and cancelable biometric systems based on brainwave signals. Each project will be supervised by faculty mentors and industry advisors with expertise spanning the breadth of biometric computing. Participants will receive introductory lectures, biometrics workshops led by graduate students, seminars on research methodology and professional development, and tours of local industry and border security facilities. The site leverages UB's established partnerships with industry affiliates in the biometrics field. Program effectiveness will be assessed through internal and external evaluations, and research outcomes will be disseminated through publications, open-source tools, and public demonstrations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Collaborative Research: CER: From Classroom to Career Readiness: Enhancing Undergraduate Computing Education Through Collaborative Research Experience in AI Security and Privacy

Artificial Intelligence (AI) technologies are transforming daily life, but their rapid adoption has also introduced serious security and privacy challenges. Addressing these risks requires a workforce that can both advance AI innovation and safeguard its deployment. This project will help meet that need by strengthening undergraduate computing education through a curriculum-based research experience program that connects classroom learning with real world research experiences. The effort will integrate the latest AI security and privacy topics into existing computing courses while helping students build professional skills such as communication, teamwork, and leadership. By creating flexible learning modules that can be used across a range of undergraduate computing courses and institutions, the project will support workforce development and contribute to the secure, reliable, and responsible use of AI in society. The project will establish a curriculum-based undergraduate research experience program focused on AI security and privacy across partner institutions. The research team will design, implement, and evaluate flexible educational modules including labs, tutorials, assignments, and research activities in computer vision, speech and audio, and network systems. These modules will address vulnerabilities across the AI lifecycle and will be designed for seamless integration into undergraduate computing courses. The instructional materials will also be aligned with the NICE (National Initiative for Cybersecurity Education) Cybersecurity Workforce Framework to strengthen career-relevant competencies. In parallel, the research team will study educational approaches that embed research into coursework, including project-based and competition-based learning, and evaluate their effects on student engagement, success, technical growth, and professional skill development. The project will generate transferable resources and evidence-based practices that can be adopted more broadly in computing education and shared with academic and community audiences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Collaborative Research: Frameworks: Cloud Conversations: AI-Augmented Interfaces to Research Infrastructure

Cloud computing is essential to a growing number of science use cases, but configuring scientific environments for deployment in the cloud can be challenging given that it often requires specialized knowledge in system administration, networking, security, and involves numerous configuration settings. Furthermore, many scientific workloads depend on tightly coupled virtual clusters, specialized hardware, fast interconnects, accelerators, and custom drivers. Managing this complexity consumes valuable time and attention that researchers could otherwise devote to science. This project develops an AI-based conversational assistant for configuring scientific computing environments that lets researchers describe what they need in everyday language and then creates the environment and verifies its integrity on shared research cloud infrastructure. The benefits of this approach range from increasing scientific productivity and lowering the cost of using cloud computing to enabling practical reproducibility of computational experimentation. The project designs and deploys an AI-based agent framework that can plan, provision, and validate scientific computing environments on open research computing infrastructure, such as Chameleon and Jetstream2. The framework combines large language models running on open, high-performance academic hardware with a set of software tools exposed through standard interfaces that include cloud-based services for resource provisioning, hardware and software environment templates, correctness checks, as well as validation benchmark suite. Key components include planning modules with built-in checks on resource limits, timing, and hardware compatibility; state and error handling modules that track multi-step workflows and summarize system events; and search pipelines that organize information from a wide variety of sources, including documentation, logs, help desk tickets, and environment artifacts into a searchable knowledge base. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Collaborative Research: CER: From Classroom to Career Readiness: Enhancing Undergraduate Computing Education Through Collaborative Research Experience in AI Security and Privacy

Artificial Intelligence (AI) technologies are transforming daily life, but their rapid adoption has also introduced serious security and privacy challenges. Addressing these risks requires a workforce that can both advance AI innovation and safeguard its deployment. This project will help meet that need by strengthening undergraduate computing education through a curriculum-based research experience program that connects classroom learning with real world research experiences. The effort will integrate the latest AI security and privacy topics into existing computing courses while helping students build professional skills such as communication, teamwork, and leadership. By creating flexible learning modules that can be used across a range of undergraduate computing courses and institutions, the project will support workforce development and contribute to the secure, reliable, and responsible use of AI in society. The project will establish a curriculum-based undergraduate research experience program focused on AI security and privacy across partner institutions. The research team will design, implement, and evaluate flexible educational modules including labs, tutorials, assignments, and research activities in computer vision, speech and audio, and network systems. These modules will address vulnerabilities across the AI lifecycle and will be designed for seamless integration into undergraduate computing courses. The instructional materials will also be aligned with the NICE (National Initiative for Cybersecurity Education) Cybersecurity Workforce Framework to strengthen career-relevant competencies. In parallel, the research team will study educational approaches that embed research into coursework, including project-based and competition-based learning, and evaluate their effects on student engagement, success, technical growth, and professional skill development. The project will generate transferable resources and evidence-based practices that can be adopted more broadly in computing education and shared with academic and community audiences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

REU Site: Engineers for Exploration

Engineers for Exploration is an experiential program that enables undergraduates to conduct computer systems research that addresses technology gaps in scientific exploration. These undergraduates are provided with an impactful research project, supported by a multi-tiered mentor network and structured collaboration with researchers, scientists, and explorers. Research problems they get to work on are framed by real-world, high-impact applications, providing participants with an inspiring, multidisciplinary research experience. The Engineers for Exploration Research Experience for Undergraduates Site offers research projects rooted in real-world applications and guided by mentorship from scientists and explorers. These use-inspired artificial intelligence research projects include the creation of machine learning and digital signal processing algorithms to automatically classify bird calls from acoustic wildlife sensors, the building of novel sensing strategies on the Smartfin embedded computing system to allow surfers to collect coastal oceanographic data, and the development of FishSense underwater 3D camera systems that automate the collection of fish measurements to turn recreational divers into citizen scientists aiding in fisheries management. Engineers for Exploration involves participants in multidisciplinary research projects spanning embedded systems, remote sensing, digital signal processing, computer vision, machine learning, robotics, and data analytics. The projects develop innovative computer systems that support exploration, and are deployed worldwide by scientists and explorers. For participants, the site fosters mentorship, leadership, and teamwork skills, providing a positive, memorable experience that encourages them to pursue further opportunities, such as research training or graduate education. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Collaborative Research: Frameworks: Cloud Conversations: AI-Augmented Interfaces to Research Infrastructure

Cloud computing is essential to a growing number of science use cases, but configuring scientific environments for deployment in the cloud can be challenging given that it often requires specialized knowledge in system administration, networking, security, and involves numerous configuration settings. Furthermore, many scientific workloads depend on tightly coupled virtual clusters, specialized hardware, fast interconnects, accelerators, and custom drivers. Managing this complexity consumes valuable time and attention that researchers could otherwise devote to science. This project develops an AI-based conversational assistant for configuring scientific computing environments that lets researchers describe what they need in everyday language and then creates the environment and verifies its integrity on shared research cloud infrastructure. The benefits of this approach range from increasing scientific productivity and lowering the cost of using cloud computing to enabling practical reproducibility of computational experimentation. The project designs and deploys an AI-based agent framework that can plan, provision, and validate scientific computing environments on open research computing infrastructure, such as Chameleon and Jetstream2. The framework combines large language models running on open, high-performance academic hardware with a set of software tools exposed through standard interfaces that include cloud-based services for resource provisioning, hardware and software environment templates, correctness checks, as well as validation benchmark suite. Key components include planning modules with built-in checks on resource limits, timing, and hardware compatibility; state and error handling modules that track multi-step workflows and summarize system events; and search pipelines that organize information from a wide variety of sources, including documentation, logs, help desk tickets, and environment artifacts into a searchable knowledge base. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Collaborative Research: CER: From Classroom to Career Readiness: Enhancing Undergraduate Computing Education Through Collaborative Research Experience in AI Security and Privacy

Artificial Intelligence (AI) technologies are transforming daily life, but their rapid adoption has also introduced serious security and privacy challenges. Addressing these risks requires a workforce that can both advance AI innovation and safeguard its deployment. This project will help meet that need by strengthening undergraduate computing education through a curriculum-based research experience program that connects classroom learning with real world research experiences. The effort will integrate the latest AI security and privacy topics into existing computing courses while helping students build professional skills such as communication, teamwork, and leadership. By creating flexible learning modules that can be used across a range of undergraduate computing courses and institutions, the project will support workforce development and contribute to the secure, reliable, and responsible use of AI in society. The project will establish a curriculum-based undergraduate research experience program focused on AI security and privacy across partner institutions. The research team will design, implement, and evaluate flexible educational modules including labs, tutorials, assignments, and research activities in computer vision, speech and audio, and network systems. These modules will address vulnerabilities across the AI lifecycle and will be designed for seamless integration into undergraduate computing courses. The instructional materials will also be aligned with the NICE (National Initiative for Cybersecurity Education) Cybersecurity Workforce Framework to strengthen career-relevant competencies. In parallel, the research team will study educational approaches that embed research into coursework, including project-based and competition-based learning, and evaluate their effects on student engagement, success, technical growth, and professional skill development. The project will generate transferable resources and evidence-based practices that can be adopted more broadly in computing education and shared with academic and community audiences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Collaborative Research: Frameworks: Cloud Conversations: AI-Augmented Interfaces to Research Infrastructure

Cloud computing is essential to a growing number of science use cases, but configuring scientific environments for deployment in the cloud can be challenging given that it often requires specialized knowledge in system administration, networking, security, and involves numerous configuration settings. Furthermore, many scientific workloads depend on tightly coupled virtual clusters, specialized hardware, fast interconnects, accelerators, and custom drivers. Managing this complexity consumes valuable time and attention that researchers could otherwise devote to science. This project develops an AI-based conversational assistant for configuring scientific computing environments that lets researchers describe what they need in everyday language and then creates the environment and verifies its integrity on shared research cloud infrastructure. The benefits of this approach range from increasing scientific productivity and lowering the cost of using cloud computing to enabling practical reproducibility of computational experimentation. The project designs and deploys an AI-based agent framework that can plan, provision, and validate scientific computing environments on open research computing infrastructure, such as Chameleon and Jetstream2. The framework combines large language models running on open, high-performance academic hardware with a set of software tools exposed through standard interfaces that include cloud-based services for resource provisioning, hardware and software environment templates, correctness checks, as well as validation benchmark suite. Key components include planning modules with built-in checks on resource limits, timing, and hardware compatibility; state and error handling modules that track multi-step workflows and summarize system events; and search pipelines that organize information from a wide variety of sources, including documentation, logs, help desk tickets, and environment artifacts into a searchable knowledge base. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

REU Site: Research Experience for Undergraduates Intersecting Computing and Evolution

This Research Experiences for Undergraduates (REU) Site project is focused on the training of students at the intersection of Computer Science and Digital Evolution. The students in this program conduct interdisciplinary research under the supervision of faculty members from the GVSU College of Computing, the GVSU Annis Water Research Institute, and external experts from the Van Andel Institute, an independent biomedical research institute. The site supports 10 undergraduate students for 10 weeks during the summers of 2027-2029, where participants work on projects spanning digital evolution, evolutionary computation, search-based software engineering, cell and molecular biology, and ecology. The project’s novelties are in the advancements of evolutionary theory where students are developing new techniques for solving real-world problems. Students work on multi-disciplinary research teams, guided by mentors, to advance and exploit evolutionary theory in solving complex, computational problems. The project's broader significance and importance are in preparing the next generation of interdisciplinary scientists to bridge computational problem solving with real-world challenges. The REU site intersecting computing and evolution offers a rewarding research experience for students interested in complex problems that may not have clear solutions. The students work at the frontiers of computer science, digital evolution, software engineering, biology, and mathematics, and gain critical experience for the future workforce through interdisciplinary presentations, workshops, student-driven discussions, and hands-on fieldwork under mentor supervision. Students participate in professional development activities via training on empirical research methods, ethics, and methods of communicating their findings to both the general public and scientific experts. They acquire new skills that are in demand in a broad range of technical careers. To support these activities, students meet regularly as a cohort and with mentors and GVSU graduate students to ensure continuing progress and growth on their respective research projects and training activities. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Development of an at-home weight-shifting balance game with musical biofeedback for older adults

Reducing fall risk is a dire societal need that requires interventions that over-prepare individuals to perform maneuvers important to daily mobility. Falling is often caused by improper weight shifting, and interventions that focus on developing weight-shifting abilities have shown improvements in clinical balance outcomes, including reduced fall incidence. Interventions that combine challenges to the cognitive and motor systems may be necessary to reduce fall-risk. Our central hypothesis is that leveraging gamification and “musical biofeedback” will improve balance abilities through practicing weight-shifting skills with increased cognitive and physical demands. Musical biofeedback conveys biological sensor data from the participant through specific musical sound parameters in real-time. Of particular interest in the proposal is the applicability to use musical biofeedback to train weight-shifting skills in a musical game. The goal is to develop a wearable sensor system that can be used at-home to practice and develop balance skills, while supporting cognitive engagement and motivation to adhere to exercise goals. To start, we are focusing on older adult end-users who typically have home exercise programs focused on weight-shifting. However, in the future, many other populations can benefit from this technology. In this Trailblazer award, the PI is leveraging her background in studying complex human maneuvers, developing musical biofeedback for older adults, and in algorithm development for mHealth sensors. The transdisciplinary team includes expertise in engineering, gamified rehabilitation technologies, home exercise programs, psychology of aging, and music. In the proposed research, our goals are to evaluate responses to the musical biofeedback game (Aim 1), validate the mHealth sensor system (Aim 2), and phenotype the gameplay behavior of fallers vs. non-fallers (Aim 3), relative to their baseline characteristics (Sub-Aim 3). Our long-term goal is for a variety of people to improve their balance control patterns while supporting and building their self-efficacy. We envision users, including older adults, training with musical biofeedback to safely (and enjoyably) prepare themselves to ambulate in their community – improving and preserving their mobility. The proposed research will pioneer using an emerging clinical technology – musical biofeedback – to train balance during weight-shifting tasks. The proposed research innovates how musical biofeedback, gamification, and focusing on weight-shifting and turns in balance training can be leveraged to challenge cognitive and physical body systems in fall-risk populations. By developing new therapy options and better understanding responses relative to baseline characteristics, this research improves clinical practices to reduce fall risk and deepens our understanding of dynamic balance control. Finally, the results of the proposed research will have translational impacts to help other fall-risk groups.

Untitled Seminar

Untitled Seminar

Predictive Coding Light

Current machine learning systems consume vastly more energy than biological brains. Neuromorphic systems aim to overcome this difference by mimicking the brain’s information coding via discrete voltage spikes. However, it remains unclear how both artificial and natural networks of spiking neurons can learn energy-efficient information processing strategies. Here we propose Predictive Coding Light (PCL), a recurrent hierarchical spiking neural network for unsupervised representation learning. In contrast to previous predictive coding approaches, PCL does not transmit prediction errors to higher processing stages. Instead, it suppresses the most predictable spikes and transmits a compressed representation of the input. Using only biologically plausible spike-timing based learning rules, PCL reproduces a wealth of findings on information processing in visual cortex and permits strong performance in downstream classification tasks. Overall, PCL offers a new approach to predictive coding and its implementation in natural and artificial spiking neural networks

sensorimotor control, mouvement, touch, EEG

Traditionally, touch is associated with exteroception and is rarely considered a relevant sensory cue for controlling movements in space, unlike vision. We developed a technique to isolate and measure tactile involvement in controlling sliding finger movements over a surface. Young adults traced a 2D shape with their index finger under direct or mirror-reversed visual feedback to create a conflict between visual and somatosensory inputs. In this context, increased reliance on somatosensory input compromises movement accuracy. Based on the hypothesis that tactile cues contribute to guiding hand movements when in contact with a surface, we predicted poorer performance when the participants traced with their bare finger compared to when their tactile sensation was dampened by a smooth, rigid finger splint. The results supported this prediction. EEG source analyses revealed smaller current in the source-localized somatosensory cortex during sensory conflict when the finger directly touched the surface. This finding supports the hypothesis that, in response to mirror-reversed visual feedback, the central nervous system selectively gated task-irrelevant somatosensory inputs, thereby mitigating, though not entirely resolving, the visuo-somatosensory conflict. Together, our results emphasize touch’s involvement in movement control over a surface, challenging the notion that vision predominantly governs goal-directed hand or finger movements.

Computational Mechanisms of Predictive Processing in Brains and Machines

Predictive processing offers a unifying view of neural computation, proposing that brains continuously anticipate sensory input and update internal models based on prediction errors. In this talk, I will present converging evidence for the computational mechanisms underlying this framework across human neuroscience and deep neural networks. I will begin with recent work showing that large-scale distributed prediction-error encoding in the human brain directly predicts how sensory representations reorganize through predictive learning. I will then turn to PredNet, a popular predictive coding inspired deep network that has been widely used to model real-world biological vision systems. Using dynamic stimuli generated with our Spatiotemporal Style Transfer algorithm, we demonstrate that PredNet relies primarily on low-level spatiotemporal structure and remains insensitive to high-level content, revealing limits in its generalization capacity. Finally, I will discuss new recurrent vision models that integrate top-down feedback connections with intrinsic neural variability, uncovering a dual mechanism for robust sensory coding in which neural variability decorrelates unit responses, while top-down feedback stabilizes network dynamics. Together, these results outline how prediction error signaling and top-down feedback pathways shape adaptive sensory processing in biological and artificial systems.

Go with the visual flow: circuit mechanisms for gaze control during locomotion

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

The Systems Vision Science Summer School & Symposium, August 11 – 22, 2025, Tuebingen, Germany

Applications are invited for our third edition of Systems Vision Science (SVS) summer school since 2023, designed for everyone interested in gaining a systems level understanding of biological vision. We plan a coherent, graduate-level, syllabus on the integration of experimental data with theory and models, featuring lectures, guided exercises and discussion sessions. The summer school will end with a Systems Vision Science symposium on frontier topics on August 20-22, with additional invited and contributed presentations and posters. Call for contributions and participations to the symposium will be sent out spring of 2025. All summer school participants are invited to attend, and welcome to submit contributions to the symposium.

Continuity and segmentation - two ends of a spectrum or independent processes?

Representational drift in human visual cortex

“Brain theory, what is it or what should it be?”

n the neurosciences the need for some 'overarching' theory is sometimes expressed, but it is not always obvious what is meant by this. One can perhaps agree that in modern science observation and experimentation is normally complemented by 'theory', i.e. the development of theoretical concepts that help guiding and evaluating experiments and measurements. A deeper discussion of 'brain theory' will require the clarification of some further distictions, in particular: theory vs. model and brain research (and its theory) vs. neuroscience. Other questions are: Does a theory require mathematics? Or even differential equations? Today it is often taken for granted that the whole universe including everything in it, for example humans, animals, and plants, can be adequately treated by physics and therefore theoretical physics is the overarching theory. Even if this is the case, it has turned out that in some particular parts of physics (the historical example is thermodynamics) it may be useful to simplify the theory by introducing additional theoretical concepts that can in principle be 'reduced' to more complex descriptions on the 'microscopic' level of basic physical particals and forces. In this sense, brain theory may be regarded as part of theoretical neuroscience, which is inside biophysics and therefore inside physics, or theoretical physics. Still, in neuroscience and brain research, additional concepts are typically used to describe results and help guiding experimentation that are 'outside' physics, beginning with neurons and synapses, names of brain parts and areas, up to concepts like 'learning', 'motivation', 'attention'. Certainly, we do not yet have one theory that includes all these concepts. So 'brain theory' is still in a 'pre-newtonian' state. However, it may still be useful to understand in general the relations between a larger theory and its 'parts', or between microscopic and macroscopic theories, or between theories at different 'levels' of description. This is what I plan to do.

Seeing a changing world through the eyes of coral fishes

Open SPM: A Modular Framework for Scanning Probe Microscopy

OpenSPM aims to democratize innovation in the field of scanning probe microscopy (SPM), which is currently dominated by a few proprietary, closed systems that limit user-driven development. Our platform includes a high-speed OpenAFM head and base optimized for small cantilevers, an OpenAFM controller, a high-voltage amplifier, and interfaces compatible with several commercial AFM systems such as the Bruker Multimode, Nanosurf DriveAFM, Witec Alpha SNOM, Zeiss FIB-SEM XB550, and Nenovision Litescope. We have created a fully documented and community-driven OpenSPM platform, with training resources and sourcing information, which has already enabled the construction of more than 15 systems outside our lab. The controller is integrated with open-source tools like Gwyddion, HDF5, and Pycroscopy. We have also engaged external companies, two of which are integrating our controller into their products or interfaces. We see growing interest in applying parts of the OpenSPM platform to related techniques such as correlated microscopy, nanoindentation, and scanning electron/confocal microscopy. To support this, we are developing more generic and modular software, alongside a structured development workflow. A key feature of the OpenSPM system is its Python-based API, which makes the platform fully scriptable and ideal for AI and machine learning applications. This enables, for instance, automatic control and optimization of PID parameters, setpoints, and experiment workflows. With a growing contributor base and industry involvement, OpenSPM is well positioned to become a global, open platform for next-generation SPM innovation.

From Spiking Predictive Coding to Learning Abstract Object Representation

In a first part of the talk, I will present Predictive Coding Light (PCL), a novel unsupervised learning architecture for spiking neural networks. In contrast to conventional predictive coding approaches, which only transmit prediction errors to higher processing stages, PCL learns inhibitory lateral and top-down connectivity to suppress the most predictable spikes and passes a compressed representation of the input to higher processing stages. We show that PCL reproduces a range of biological findings and exhibits a favorable tradeoff between energy consumption and downstream classification performance on challenging benchmarks. A second part of the talk will feature our lab’s efforts to explain how infants and toddlers might learn abstract object representations without supervision. I will present deep learning models that exploit the temporal and multimodal structure of their sensory inputs to learn representations of individual objects, object categories, or abstract super-categories such as „kitchen object“ in a fully unsupervised fashion. These models offer a parsimonious account of how abstract semantic knowledge may be rooted in children's embodied first-person experiences.

“Development and application of gaze control models for active perception”

Gaze shifts in humans serve to direct high-resolution vision provided by the fovea towards areas in the environment. Gaze can be considered a proxy for attention or indicator of the relative importance of different parts of the environment. In this talk, we discuss the development of generative models of human gaze in response to visual input. We discuss how such models can be learned, both using supervised learning and using implicit feedback as an agent interacts with the environment, the latter being more plausible in biological agents. We also discuss two ways such models can be used. First, they can be used to improve the performance of artificial autonomous systems, in applications such as autonomous navigation. Second, because these models are contingent on the human’s task, goals, and/or state in the context of the environment, observations of gaze can be used to infer information about user intent. This information can be used to improve human-machine and human robot interaction, by making interfaces more anticipative. We discuss example applications in gaze-typing, robotic tele-operation and human-robot interaction.

The Unconscious Eye: What Involuntary Eye Movements Reveal About Brain Processing

Neuro-Optometric Rehabilitation - an introduction to the diagnosis and treatment of vision disorders secondary to neurological impairment

Restoring Sight to the Blind: Effects of Structural and Functional Plasticity

Visual restoration after decades of blindness is now becoming possible by means of retinal and cortical prostheses, as well as emerging stem cell and gene therapeutic approaches. After restoring visual perception, however, a key question remains. Are there optimal means and methods for retraining the visual cortex to process visual inputs, and for learning or relearning to “see”? Up to this point, it has been largely assumed that if the sensory loss is visual, then the rehabilitation focus should also be primarily visual. However, the other senses play a key role in visual rehabilitation due to the plastic repurposing of visual cortex during blindness by audition and somatosensation, and also to the reintegration of restored vision with the other senses. I will present multisensory neuroimaging results, cortical thickness changes, as well as behavioral outcomes for patients with Retinitis Pigmentosa (RP), which causes blindness by destroying photoreceptors in the retina. These patients have had their vision partially restored by the implantation of a retinal prosthesis, which electrically stimulates still viable retinal ganglion cells in the eye. Our multisensory and structural neuroimaging and behavioral results suggest a new, holistic concept of visual rehabilitation that leverages rather than neglects audition, somatosensation, and other sensory modalities.

Cognitive maps, navigational strategies, and the human brain

The hippocampus, visual perception and visual memory

Reading Scenes

Plasticity of the adult visual system

Computational modelling of ocular pharmacokinetics

Pharmacokinetics in the eye is an important factor for the success of ocular drug delivery and treatment. Pharmacokinetic features determine the feasible routes of drug administration, dosing levels and intervals, and it has impact on eventual drug responses. Several physical, biochemical, and flow-related barriers limit drug exposure of anterior and posterior ocular target tissues during treatment during local (topical, subconjunctival, intravitreal) and systemic administration (intravenous, per oral). Mathematical models integrate joint impact of various barriers on ocular pharmacokinetics (PKs) thereby helping drug development. The models are useful in describing (top-down) and predicting (bottom-up) pharmacokinetics of ocular drugs. This is useful also in the design and development of new drug molecules and drug delivery systems. Furthermore, the models can be used for interspecies translation and probing of disease effects on pharmacokinetics. In this lecture, ocular pharmacokinetics and current modelling methods (noncompartmental analyses, compartmental, physiologically based, and finite element models) are introduced. Future challenges are also highlighted (e.g. intra-tissue distribution, prediction of drug responses, active transport).

Deepfake emotional expressions trigger the uncanny valley brain response, even when they are not recognised as fake

Facial expressions are inherently dynamic, and our visual system is sensitive to subtle changes in their temporal sequence. However, researchers often use dynamic morphs of photographs—simplified, linear representations of motion—to study the neural correlates of dynamic face perception. To explore the brain's sensitivity to natural facial motion, we constructed a novel dynamic face database using generative neural networks, trained on a verified set of video-recorded emotional expressions. The resulting deepfakes, consciously indistinguishable from videos, enabled us to separate biological motion from photorealistic form. Results showed that conventional dynamic morphs elicit distinct responses in the brain compared to videos and photos, suggesting they violate expectations (n400) and have reduced social salience (late positive potential). This suggests that dynamic morphs misrepresent facial dynamism, resulting in misleading insights about the neural and behavioural correlates of face perception. Deepfakes and videos elicited largely similar neural responses, suggesting they could be used as a proxy for real faces in vision research, where video recordings cannot be experimentally manipulated. And yet, despite being consciously undetectable as fake, deepfakes elicited an expectation violation response in the brain. This points to a neural sensitivity to naturalistic facial motion, beyond conscious awareness. Despite some differences in neural responses, the realism and manipulability of deepfakes make them a valuable asset for research where videos are unfeasible. Using these stimuli, we proposed a novel marker for the conscious perception of naturalistic facial motion – Frontal delta activity – which was elevated for videos and deepfakes, but not for photos or dynamic morphs.

Retinal input integration in excitatory and inhibitory neurons in the mouse superior colliculus in vivo

An inconvenient truth: pathophysiological remodeling of the inner retina in photoreceptor degeneration

Photoreceptor loss is the primary cause behind vision impairment and blindness in diseases such as retinitis pigmentosa and age-related macular degeneration. However, the death of rods and cones allows retinoids to permeate the inner retina, causing retinal ganglion cells to become spontaneously hyperactive, severely reducing the signal-to-noise ratio, and creating interference in the communication between the surviving retina and the brain. Treatments aimed at blocking or reducing hyperactivity improve vision initiated from surviving photoreceptors and could enhance the signal fidelity generated by vision restoration methodologies.

The speed of prioritizing information for consciousness: A robust and mysterious human trait

Altered grid-like coding in early blind people and the role of vision in conceptual navigation

Vision for perception versus vision for action: dissociable contributions of visual sensory drives from primary visual cortex and superior colliculus neurons to orienting behaviors

The primary visual cortex (V1) directly projects to the superior colliculus (SC) and is believed to provide sensory drive for eye movements. Consistent with this, a majority of saccade-related SC neurons also exhibit short-latency, stimulus-driven visual responses, which are additionally feature-tuned. However, direct neurophysiological comparisons of the visual response properties of the two anatomically-connected brain areas are surprisingly lacking, especially with respect to active looking behaviors. I will describe a series of experiments characterizing visual response properties in primate V1 and SC neurons, exploring feature dimensions like visual field location, spatial frequency, orientation, contrast, and luminance polarity. The results suggest a substantial, qualitative reformatting of SC visual responses when compared to V1. For example, SC visual response latencies are actively delayed, independent of individual neuron tuning preferences, as a function of increasing spatial frequency, and this phenomenon is directly correlated with saccadic reaction times. Such “coarse-to-fine” rank ordering of SC visual response latencies as a function of spatial frequency is much weaker in V1, suggesting a dissociation of V1 responses from saccade timing. Consistent with this, when we next explored trial-by-trial correlations of individual neurons’ visual response strengths and visual response latencies with saccadic reaction times, we found that most SC neurons exhibited, on a trial-by-trial basis, stronger and earlier visual responses for faster saccadic reaction times. Moreover, these correlations were substantially higher for visual-motor neurons in the intermediate and deep layers than for more superficial visual-only neurons. No such correlations existed systematically in V1. Thus, visual responses in SC and V1 serve fundamentally different roles in active vision: V1 jumpstarts sensing and image analysis, but SC jumpstarts moving. I will finish by demonstrating, using V1 reversible inactivation, that, despite reformatting of signals from V1 to the brainstem, V1 is still a necessary gateway for visually-driven oculomotor responses to occur, even for the most reflexive of eye movement phenomena. This is a fundamental difference from rodent studies demonstrating clear V1-independent processing in afferent visual pathways bypassing the geniculostriate one, and it demonstrates the importance of multi-species comparisons in the study of oculomotor control.

Contentopic mapping and object dimensionality - a novel understanding on the organization of object knowledge

Our ability to recognize an object amongst many others is one of the most important features of the human mind. However, object recognition requires tremendous computational effort, as we need to solve a complex and recursive environment with ease and proficiency. This challenging feat is dependent on the implementation of an effective organization of knowledge in the brain. Here I put forth a novel understanding of how object knowledge is organized in the brain, by proposing that the organization of object knowledge follows key object-related dimensions, analogously to how sensory information is organized in the brain. Moreover, I will also put forth that this knowledge is topographically laid out in the cortical surface according to these object-related dimensions that code for different types of representational content – I call this contentopic mapping. I will show a combination of fMRI and behavioral data to support these hypotheses and present a principled way to explore the multidimensionality of object processing.

Guiding Visual Attention in Dynamic Scenes

Rethinking Attention: Dynamic Prioritization

Decades of research on understanding the mechanisms of attentional selection have focused on identifying the units (representations) on which attention operates in order to guide prioritized sensory processing. These attentional units fit neatly to accommodate our understanding of how attention is allocated in a top-down, bottom-up, or historical fashion. In this talk, I will focus on attentional phenomena that are not easily accommodated within current theories of attentional selection – the “attentional platypuses,” as they allude to an observation that within biological taxonomies the platypus does not fit into either mammal or bird categories. Similarly, attentional phenomena that do not fit neatly within current attentional models suggest that current models need to be revised. I list a few instances of the ‘attentional platypuses” and then offer a new approach, the Dynamically Weighted Prioritization, stipulating that multiple factors impinge onto the attentional priority map, each with a corresponding weight. The interaction between factors and their corresponding weights determines the current state of the priority map which subsequently constrains/guides attention allocation. I propose that this new approach should be considered as a supplement to existing models of attention, especially those that emphasize categorical organizations.

Traumatic brain injury and the visual sequela

Mind Perception and Behaviour: A Study of Quantitative and Qualitative Effects

Perceptual illusions we understand well, and illusions which aren’t really illusions

Imagining and seeing: two faces of prosopagnosia

Why age-related macular degeneration is a mathematically tractable disease

Among all prevalent diseases with a central neurodegeneration, AMD can be considered the most promising in terms of prevention and early intervention, due to several factors surrounding the neural geometry of the foveal singularity. • Steep gradients of cell density, deployed in a radially symmetric fashion, can be modeled with a difference of Gaussian curves. • These steep gradients give rise to huge, spatially aligned biologic effects, summarized as the Center of Cone Resilience, Surround of Rod Vulnerability. • Widely used clinical imaging technology provides cellular and subcellular level information. • Data are now available at all timelines: clinical, lifespan, evolutionary • Snapshots are available from tissues (histology, analytic chemistry, gene expression) • A viable biogenesis model exists for drusen, the largest population-level intraocular risk factor for progression. • The biogenesis model shares molecular commonality with atherosclerotic cardiovascular disease, for which there has been decades of public health success. • Animal and cell model systems are emerging to test these ideas.

Reactivation in the human brain connects the past with the present

Error Consistency between Humans and Machines as a function of presentation duration

Within the last decade, Deep Artificial Neural Networks (DNNs) have emerged as powerful computer vision systems that match or exceed human performance on many benchmark tasks such as image classification. But whether current DNNs are suitable computational models of the human visual system remains an open question: While DNNs have proven to be capable of predicting neural activations in primate visual cortex, psychophysical experiments have shown behavioral differences between DNNs and human subjects, as quantified by error consistency. Error consistency is typically measured by briefly presenting natural or corrupted images to human subjects and asking them to perform an n-way classification task under time pressure. But for how long should stimuli ideally be presented to guarantee a fair comparison with DNNs? Here we investigate the influence of presentation time on error consistency, to test the hypothesis that higher-level processing drives behavioral differences. We systematically vary presentation times of backward-masked stimuli from 8.3ms to 266ms and measure human performance and reaction times on natural, lowpass-filtered and noisy images. Our experiment constitutes a fine-grained analysis of human image classification under both image corruptions and time pressure, showing that even drastically time-constrained humans who are exposed to the stimuli for only two frames, i.e. 16.6ms, can still solve our 8-way classification task with success rates way above chance. We also find that human-to-human error consistency is already stable at 16.6ms.

Attending to moments in time

Visuomotor learning of location, action, and prediction

Trends in NeuroAI - Brain-like topography in transformers (Topoformer)

Dr. Nicholas Blauch will present on his work "Topoformer: Brain-like topographic organization in transformer language models through spatial querying and reweighting". Dr. Blauch is a postdoctoral fellow in the Harvard Vision Lab advised by Talia Konkle and George Alvarez. Paper link: https://openreview.net/pdf?id=3pLMzgoZSA Trends in NeuroAI is a reading group hosted by the MedARC Neuroimaging & AI lab (https://medarc.ai/fmri | https://groups.google.com/g/medarc-fmri).

Retinal Photoreceptor Diversity Across Mammals

Generative models for video games (rescheduled)

Developing agents capable of modeling complex environments and human behaviors within them is a key goal of artificial intelligence research. Progress towards this goal has exciting potential for applications in video games, from new tools that empower game developers to realize new creative visions, to enabling new kinds of immersive player experiences. This talk focuses on recent advances of my team at Microsoft Research towards scalable machine learning architectures that effectively capture human gameplay data. In the first part of my talk, I will focus on diffusion models as generative models of human behavior. Previously shown to have impressive image generation capabilities, I present insights that unlock applications to imitation learning for sequential decision making. In the second part of my talk, I discuss a recent project taking ideas from language modeling to build a generative sequence model of an Xbox game.

Applied cognitive neuroscience to improve learning and therapeutics

Advancements in cognitive neuroscience have provided profound insights into the workings of the human brain and the methods used offer opportunities to enhance performance, cognition, and mental health. Drawing upon interdisciplinary collaborations in the University of California San Diego, Human Performance Optimization Lab, this talk explores the application of cognitive neuroscience principles in three domains to improve human performance and alleviate mental health challenges. The first section will discuss studies addressing the role of vision and oculomotor function in athletic performance and the potential to train these foundational abilities to improve performance and sports outcomes. The second domain considers the use of electrophysiological measurements of the brain and heart to detect, and possibly predict, errors in manual performance, as shown in a series of studies with surgeons as they perform robot-assisted surgery. Lastly, findings from clinical trials testing personalized interventional treatments for mood disorders will be discussed in which the temporal and spatial parameters of transcranial magnetic stimulation (TMS) are individualized to test if personalization improves treatment response and can be used as predictive biomarkers to guide treatment selection. Together, these translational studies use the measurement tools and constructs of cognitive neuroscience to improve human performance and well-being.

Characterizing the causal role of large-scale network interactions in supporting complex cognition

Neuroimaging has greatly extended our capacity to study the workings of the human brain. Despite the wealth of knowledge this tool has generated however, there are still critical gaps in our understanding. While tremendous progress has been made in mapping areas of the brain that are specialized for particular stimuli, or cognitive processes, we still know very little about how large-scale interactions between different cortical networks facilitate the integration of information and the execution of complex tasks. Yet even the simplest behavioral tasks are complex, requiring integration over multiple cognitive domains. Our knowledge falls short not only in understanding how this integration takes place, but also in what drives the profound variation in behavior that can be observed on almost every task, even within the typically developing (TD) population. The search for the neural underpinnings of individual differences is important not only philosophically, but also in the service of precision medicine. We approach these questions using a three-pronged approach. First, we create a battery of behavioral tasks from which we can calculate objective measures for different aspects of the behaviors of interest, with sufficient variance across the TD population. Second, using these individual differences in behavior, we identify the neural variance which explains the behavioral variance at the network level. Finally, using covert neurofeedback, we perturb the networks hypothesized to correspond to each of these components, thus directly testing their casual contribution. I will discuss our overall approach, as well as a few of the new directions we are currently pursuing.

Vision Unveiled: Understanding Face Perception in Children Treated for Congenital Blindness

Generative models for video games

Developing agents capable of modeling complex environments and human behaviors within them is a key goal of artificial intelligence research. Progress towards this goal has exciting potential for applications in video games, from new tools that empower game developers to realize new creative visions, to enabling new kinds of immersive player experiences. This talk focuses on recent advances of my team at Microsoft Research towards scalable machine learning architectures that effectively capture human gameplay data. In the first part of my talk, I will focus on diffusion models as generative models of human behavior. Previously shown to have impressive image generation capabilities, I present insights that unlock applications to imitation learning for sequential decision making. In the second part of my talk, I discuss a recent project taking ideas from language modeling to build a generative sequence model of an Xbox game.

Inhibition in the retina

Perception in Autism: Testing Recent Bayesian Inference Accounts

Stability of visual processing in passive and active vision

The visual system faces a dual challenge. On the one hand, features of the natural visual environment should be stably processed - irrespective of ongoing wiring changes, representational drift, and behavior. On the other hand, eye, head, and body motion require a robust integration of pose and gaze shifts in visual computations for a stable perception of the world. We address these dimensions of stable visual processing by studying the circuit mechanism of long-term representational stability, focusing on the role of plasticity, network structure, experience, and behavioral state while recording large-scale neuronal activity with miniature two-photon microscopy.

Modeling idiosyncratic evaluation of faces

Scalable microelectrode arrays: moving beyond time division multiplexing

Bernstein Conference 2024

Timing and transmission: the role of axonal action potential propagation speed in the synchronization of foveal vision

Bernstein Conference 2024

Evaluating Noise Tolerance in Drosophila Vision

COSYNE 2022

An insect vision-based flight control model with a plastic efference copy

COSYNE 2022

An insect vision-based flight control model with a plastic efference copy

COSYNE 2022

Organization of local directionally selective neurons informs global motion vision encoding

COSYNE 2022

Organization of local directionally selective neurons informs global motion vision encoding

COSYNE 2022

A two-way luminance gain control in the fly brain ensures luminance invariance in dynamic vision

COSYNE 2022

A two-way luminance gain control in the fly brain ensures luminance invariance in dynamic vision

COSYNE 2022

Inferring the order of stable and context dependent perceptual biases in human vision

COSYNE 2023

Leveraging computational and animal models of vision to probe atypical emotion recognition in autism

COSYNE 2023

Biologically Realistic Computational Primitives of Neocortex Implemented on Neuromorphic Hardware Improve Vision Transformer Performance

COSYNE 2025

Enhancing Vision Robustness to Adversarial Attacks through Foveal-Peripheral and Saccadic Mechanisms

COSYNE 2025

Recurrent connectivity supports motion detection in connectome-constrained models of fly vision

COSYNE 2025

TweetyBERT, a self-supervised vision transformer to automate birdsong annotation

COSYNE 2025

The contribution of thalamic subdivisions to learning is associated with interindividual variability in memory performance

Deep Learning of Brain Spacetime to Predict Outcome of Vision Restoration Therapy using Non-invasive Brain Stimulation

Retinal waves align the concentric orientation map in mouse superior colliculus to the center of vision

Restoring vision by conjugated polymer nanoparticles in a model of Retinitis Pigmentosa

Role of dopamine neurons in inter-individual variability during social labor division task in mice

A role for neurons of the medial division of the central amygdala in appetitive behaviours

Structural plasticity during vision-dependent learning in mouse visual cortex

Toward a new dimensional approach to addiction: Linking addiction markers to the connectivity profiles of striatum subdivisions

A two-way luminance gain control in the fly brain ensures luminance invariance in dynamic vision

Vision dependent and independent processes shape the organization of cortico-cortical feedback in the mouse visual cortex

Visual encoding by retinal ganglion cells in optogenetic models for vision restoration

In vivo biocompatibility and functionality of porous-graphene-based subretinal implants for vision restoration

Analyzing animal behavior with domain-adapted vision-language models

FENS Forum 2024

Compromised binocular vision and reduced binocularity in the visual cortex of postsynaptic density 95 (PSD-95) knock-out mice

FENS Forum 2024

Computer vision and image processing applications on astrocyte-glioma interactions in 3D cell culture

FENS Forum 2024

Connectional subdivisions reflect neuronal features of the various sectors of the macaque ventrolateral prefrontal cortex

FENS Forum 2024

Exploring laryngeal effects of dorsolateral periaqueductal grey stimulation in anesthetized rats: Implications for c-Fos and FOXP2 expression in the nucleus ambiguus subdivisions

FENS Forum 2024

Exploring pupil dynamics in freely moving rats during active integration of vision and posture

FENS Forum 2024

Impact of barrel cortex lesions and sensory deprivation on perceptual decision-making: Insights from computer vision and time series clustering of freely moving behavioral strategies

FENS Forum 2024

The impact of the retinotopic subdivisions of area V1 on shaping the macaque connectome

FENS Forum 2024

Oculomotor vergence system through fMRI in persons with binocularly normal vision and persistent post-concussive symptoms with convergence insufficiency

FENS Forum 2024

Optogenetic stimulation in the visual thalamus for future brain vision prostheses

FENS Forum 2024

Projections from the ventral nucleus of the trapezoid body to all subdivisions of the rat cochlear nucleus

FENS Forum 2024

REST as a target for vision restoration

FENS Forum 2024

A retinotopic-and-orientation-based stimulation strategy induces neural activity patterns mimicking natural vision

FENS Forum 2024