Seminars

Deep networks are often thought of as black boxes. Their ability to encompass vast swathes of knowledge indeed makes them hard to explain. Yet many of their behaviours — generalization under overparametrization, grokking, OOD failures, neural scaling laws — recur across architectures and scales, and each, however surprising, can be reproduced and explained in controlled settings. I will review these efforts to identify and explain the universal phenomena of deep learning, and suggest that an LLM may amount to a sum of such tractable sub-phenomena, interpolative in nature. Finally, leaving scientific rigor aside, I'll argue that what separates this prosaic picture from the apparent magic of LLMs may well be the industrial scale of compute and human labour behind it, and that AGI in its deeper extrapolative sense may be much further away than claimed.

Jie Yang presents BRIDGE, a multilingual evaluation benchmark built from real clinical tasks and more than one million samples derived from electronic health records. The benchmark addresses the gap between simplified examination-style tests and the complexity of clinical data, while enabling comparison of rapidly changing language models. Evaluation of 95 models involved over 24,000 experiments and 39 million predictions. Results vary substantially by model family, task and language, with some open-source systems matching proprietary models. The study also finds that chain-of-thought prompting frequently reduces accuracy on these tasks and examines stigmatizing language produced during model reasoning at large scale.

Matthias Vigl examines whether the computational scaling strategies behind modern machine learning can benefit particle physics. Empirical scaling laws relate model performance to computing resources and help balance model size against training-data volume. For a fixed dataset, increasing model capacity beyond the interpolation threshold can improve generalization through double descent, but gains eventually encounter limits imposed by the data. The talk then considers scaling data and model size together, deriving compute-optimal relations for transformer-based jet taggers trained on as many as billions of simulated jets. It studies how training hyperparameters and input representations change these relations. The observed behaviour suggests that larger computing budgets could yield useful gains in particle physics, especially with a move toward more general-purpose foundation models.

Yixuan He (Arizona State University) considers graph neural networks for social, financial and biological networks in which edge signs and directions contain information needed for clustering and prediction. After an introduction to graph neural networks, the talk explains how signed and directed methods preserve relationships lost when every connection is treated as unsigned and undirected. The seminar then asks when a representation learned before the final task can transfer to a different graph or to a setting with few labels. Two theoretical perspectives address transfer across graph domains and the use of link prediction for linear community detection. TopoDIG and TopoSIGN combine encoders that preserve edge structure with persistent topological features, enabling pre-training and prompt-based adaptation on directed and signed networks.

Srinivasan Keshav (University of Cambridge) presents an interface designed to help ecologists use AI for remote sensing despite unfamiliarity with embeddings and algorithms. The Tessera Embeddings Explorer lets practitioners privately explore a region of interest, create habitat maps and perform regressions using their domain expertise and ground-truth knowledge. The talk demonstrates the tool and explains the design principles and the software architecture that follows from them, connecting usable ecological workflows with AI representations. Keshav is the Robert Sansom Professor of Computer Science at Cambridge.

Alok Puranik (Jane Street) examines data mixing for large language models: allocating model capacity and computation among training sources. A small model’s preferred data mix may differ from that of a larger model, yet the cost of large-model training forces many decisions to depend on smaller experiments. Scaling laws can predict performance as size and computation increase only when trends remain stable or change predictably. The talk uses experiments from Puranik’s work and frontier laboratories to examine data-mix questions that violate these assumptions, explain why small-scale results can mislead, and explore better methods for choosing training data. His research at Jane Street concerns scaling laws for sequence models in trading.

Navid Azizan presents methods that make neural models obey physical, safety and operational constraints at deployment. Hard-constrained neural networks, or HardNets, enforce input-dependent constraints by construction while preserving universal approximation within the feasible function class. Applications include models of chaotic dynamics with bounded trajectories, energy-constrained operator learning, safe reinforcement learning and control with formal guarantees. The talk then turns to enforcing constraints during sampling from pretrained diffusion and flow-matching models. Formulating generation as trajectory optimization allows receding-horizon control to guide outputs toward feasibility without retraining or imposing excessive restrictions on the sampling process. Examples from fluid dynamics, robot planning and control, PDE control and language-guided image editing illustrate how constraints can define admissible behavior while retaining expressive learning and generation.

Lenka Zdeborová introduces the Rules-and-Facts model, in which some observations follow a shared rule while others are isolated exceptions requiring memorization. The framework studies when a learner can acquire the rule and retain those exceptions simultaneously. Its results emphasize how capacity is organized and deployed, rather than capacity alone. The talk examines how regularization and the geometry of kernels or learned feature maps can reserve resources for memorization without undermining rule learning. It connects the balance between abstraction and memory to architectural and algorithmic choices, providing a theoretical account of neural networks that both generalize and recall specific facts.

MIT Biology Colloquium Series talk by Peter Koo (Cold Spring Harbor Laboratory), hosted by Yunha Hwang, on using AI to learn the gene regulatory code.

Zohar Ringel of the Hebrew University of Jerusalem presents a renormalisation-group approach to deep learning based on neural-network field theories. The seminar examines scaling laws, the removal of non-learnable field modes, benign overfitting and open questions about feature learning. The confirmed Quantum Matter seminar takes place in the Bob Room at Perimeter Institute on 8 September 2026, from 15:30 to 17:00 Toronto time.

Useful internal representations should explain the patterns of regularities and dependencies among observations. Probabilistic graphical models promise a principled way to uncover latent factors as such, but they are hard to scale to handle high-dimensional sensory observations and complicated dependencies structures. Neural-networks, on the other hand, excel at approximating complicated high-dimensional functions, but their internal representations do not easily lend themselves to a probabilistic interpretation. Despite some successes, a general unified approach is still missing for integrating the two approaches. I will describe a novel approach towards merging adaptive neural-network components into a probabilistic framework, based on three core ideas. The first is to train a set of networks to collectively perform inference, leveraging the ability of pattern-recognition methods to amortise complicated transformations. The second is to constrain the way in which the outputs of these networks are interpreted, transformed, and combined together. These constraints, together with the learning objective itself, are derived directly from probabilistic considerations encoded in a graphical model. Finally, the third core idea is that of recognition-parametrisation, allowing the inference ("recognition") procedure to directly define the model itself, without requiring an explicit "generative" decoder. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2026-03-04. Recording duration: 00:48:26.

1. Can we reconstruct images that a person saw, directly from their fMRI brain recordings? 2. Can we reconstruct the training data that a deep-network trained on, directly from the parameters of the network? The answer to both of these intriguing questions is “Yes!” In this talk I will present some of our work in both domains. I will then show how combining the power of Brains and Machines can lead to significant breakthroughs in both areas, and potentially bridge the gap between Minds and Machines. Finally, I will show how combining the power of Multiple Brains (with NO shared data) may lead to new breakthrough discoveries in Brain-Science, and allow mapping of information between different brains. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2026-02-18. Recording duration: 00:50:24.

Current machine learning systems consume vastly more energy than biological brains. Neuromorphic systems aim to overcome this difference by mimicking the brain’s information coding via discrete voltage spikes. However, it remains unclear how both artificial and natural networks of spiking neurons can learn energy-efficient information processing strategies. Here we propose Predictive Coding Light (PCL), a recurrent hierarchical spiking neural network for unsupervised representation learning. In contrast to previous predictive coding approaches, PCL does not transmit prediction errors to higher processing stages. Instead, it suppresses the most predictable spikes and transmits a compressed representation of the input. Using only biologically plausible spike-timing based learning rules, PCL reproduces a wealth of findings on information processing in visual cortex and permits strong performance in downstream classification tasks. Overall, PCL offers a new approach to predictive coding and its implementation in natural and artificial spiking neural networks

Robert Gower introduces Polar Express for the polar decomposition and matrix sign function, motivated by Muon neural-network training. Using only matrix-matrix products makes the method suited to high-throughput GPUs. Each iteration adapts its polynomial update through minimax optimization, building on Chen and Chow and Nakatsukasa and Freund. Worst-case error minimization gives rapid initial and asymptotic convergence. The talk addresses finite-precision implementation in bfloat16 and reports improved validation loss when training GPT-2 on one billion FineWeb tokens across several learning rates.

Lior Horesh presents tensor-tensor algebra designed to retain key properties of matrix algebra while representing multidimensional correlations. An Eckart–Young-like tensor representation theorem underpins computationally feasible, provably optimal decompositions. Matrix-mimetic operations allow existing computational workflows to be adapted to tensors. Examples include tensorized neural-network structures and tensor graph convolutional networks for time-evolving graphs. The discussion concludes with tensor group symmetries and extensions to equivariant learning.

David Baker presents recent advances in designing proteins for functions that natural evolution has not been required to solve. The approach begins with a desired structure or activity and uses deep learning to propose amino acid sequences expected to realise it. The resulting designs are translated into synthetic genes, produced as proteins and tested experimentally. The lecture connects computational design with laboratory characterisation, explaining how the combination can create new molecular functions relevant to medicine, technology and sustainability. Its focus is the construction and evaluation of proteins from scratch, extending the protein-design research recognised by the 2024 chemistry prize.

Predictive processing offers a unifying view of neural computation, proposing that brains continuously anticipate sensory input and update internal models based on prediction errors. In this talk, I will present converging evidence for the computational mechanisms underlying this framework across human neuroscience and deep neural networks. I will begin with recent work showing that large-scale distributed prediction-error encoding in the human brain directly predicts how sensory representations reorganize through predictive learning. I will then turn to PredNet, a popular predictive coding inspired deep network that has been widely used to model real-world biological vision systems. Using dynamic stimuli generated with our Spatiotemporal Style Transfer algorithm, we demonstrate that PredNet relies primarily on low-level spatiotemporal structure and remains insensitive to high-level content, revealing limits in its generalization capacity. Finally, I will discuss new recurrent vision models that integrate top-down feedback connections with intrinsic neural variability, uncovering a dual mechanism for robust sensory coding in which neural variability decorrelates unit responses, while top-down feedback stabilizes network dynamics. Together, these results outline how prediction error signaling and top-down feedback pathways shape adaptive sensory processing in biological and artificial systems.

Optical imaging is a major research tool in the basic sciences, and is the only imaging modality that routinely enables non-ionized imaging with subcellular spatial resolutions and high imaging speeds. In biological imaging applications, however, optical imaging is limited by tissue scattering to short imaging depths. This prevents large-scale bio-imaging by allowing visualization of only the outer superficial layers of an organism, or specific components isolated from within the organism and prepared in-vitro.

Synthetic face datasets are increasingly used to overcome the limitations of real-world biometric data, including privacy concerns, demographic imbalance, and high collection costs. However, many existing methods lack fine-grained control over identity attributes and fail to produce paired, identity-consistent images under structured capture conditions. In this talk, I will present FLUXSynID, a framework for generating high-resolution synthetic face datasets with user-defined identity attribute distributions and paired document-style and trusted live capture images. The dataset generated using FLUXSynID shows improved alignment with real-world identity distributions and greater diversity compared to prior work. I will also discuss how FLUXSynID’s dataset and generation tools can support research in face recognition and morphing attack detection (MAD), enhancing model robustness in both academic and practical applications.