The Information Bottleneck

Carnegie Mellon researcher and Abridge co-founder Zachary Lipton discusses where machine learning can help healthcare and why clinical work is harder to automate than software development. The conversation examines medical documentation, decision support, drug discovery, open models and model routing, then turns to academic research, doctoral training and the effects of automation on scientific careers. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Sara Hooker of Adaptation Lab explores AI systems that change with their users, tasks and environments. The discussion connects continual learning, efficient adaptation and automated science with the difficulty of evaluating tasks that lack easily checked answers. It also covers interfaces, distillation, open models, safety and regulation, multilingual tokenization, and possible successors to current Transformer systems. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Microsoft researcher Alexia Jolicoeur-Martineau explains the Tiny Recursive Model and the reasoning behind her work on small networks that repeatedly refine an internal state and proposed answer. The conversation examines truncated gradients, puzzle-solving benchmarks and the differences from autoregressive language generation. It also considers molecular modelling, limits of data scaling and which research directions she would prioritize. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Rohan Anil of Core Automation discusses the relationship between pretraining, reinforcement learning and learning after deployment. Drawing on his optimization work at Google and Anthropic, he examines why additional post-training, distillation and longer contexts may not solve continual learning. Other topics include second-order optimizers, distributed training, low-level coding agents and the difficulty of coordinating very large GPU clusters. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

John Langford of Microsoft AI Labs discusses whether compact implicit models of the world can improve data efficiency. The conversation examines belief-state compression, Transformer caches, JEPA-style objectives and his Next Latent research. It also covers the value of algorithmic research alongside scaling, agent-assisted experiments, open models, the history of CAPTCHA and the behavior of modern optimizers. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

INRIA researcher David Holzmüller explains how to choose among tabular foundation models, boosted trees and multilayer perceptrons. He discusses TabArena, TabICL and RealMLP, benchmark weaknesses, small-data performance, calibration and class imbalance. The conversation also examines text columns, explainability, time-series distinctions, synthetic pretraining and the trade-offs among computation, inference speed and predictive performance. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Charles Frye of Modal examines the infrastructure constraints behind large-scale AI research. With Ravid Shwartz Ziv and Allen Roush, he discusses underused hardware, saturated resources, sharing GPU capacity and deciding when to train a model. The conversation also covers inference and speculative decoding, hardware economics, virtualization and the security boundaries around agents and open models. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

EleutherAI executive director Stella Biderman discusses open models, independent research and the containment of potentially dangerous AI capabilities. She argues for stronger isolation in security evaluations and examines the influence of corporate power on scientific evidence. Other topics include sovereign AI, international access to open models, the role of interfaces in perceived progress, and filtering hazardous knowledge during pretraining. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

University of Freiburg professor and Prior Labs co-founder Frank Hutter explains how TabPFN changed deep learning for tabular data. The episode examines in-context learning from synthetic pretraining, forecasting, architectural approaches to larger tables and the continuing roles of boosted trees and language models. Hutter also discusses moving research into a company, team autonomy, model licensing and building research organizations outside established technology hubs. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Stanford researcher Surya Ganguli connects statistical physics, theoretical neuroscience and machine learning. He discusses scaling laws, data selection and the origins of diffusion models, then turns to neural experiments on perception, self-related processing and describing neuronal responses. The conversation closes with questions about human versus machine data efficiency and the different ways brains and artificial networks acquire useful algorithms. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Google DeepMind research director Brendan O'Donoghue examines discrete diffusion as an alternative to autoregressive text generation. He discusses sample diversity, reinforcement learning, latency and potential on-device or robotics applications. The episode also addresses the costs of serving diffusion models, hardware trade-offs between computation and memory bandwidth, and practical limits on progress toward more capable AI. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Nathan Lambert discusses post-training and the prospects for open AI models, drawing on his work on OLMo at Ai2 and his writing on reinforcement learning from human feedback. Topics include capability gaps, the economics of the open ecosystem, training environments, continual learning and skepticism about recursive self-improvement. The conversation also examines research culture, concentrated talent and whether increased model spending translates into better products. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Daphne Koller of insitro discusses the differences between finding predictive patterns and identifying interventions that improve disease outcomes. The conversation examines biological data scarcity, structure and causality, failures in drug development and the limits of speeding up wet-lab work with AI agents. It also considers foundation models for biology, generalization beyond observed data, human evidence and scientific discovery. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

PufferAI researcher Joseph Suarez argues that implementation and simulation bottlenecks have constrained reinforcement learning as much as algorithm choice. He discusses the design of useful simulators, CPU and GPU trade-offs, faster training pipelines and open-source research tools. The conversation ranges from game environments to potential applications in scientific simulation and biological modelling. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Florian Brand of Prime Intellect discusses how to evaluate agents when their tools and execution environments affect the result. The episode examines benchmark gaming, preventing shortcuts around scoring rules and the statistical limits of expensive evaluation runs. It also asks whether subjective impressions of reliability can be translated into useful measurements of model behavior. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Pleias co-founder Pierre-Carl Langlais describes language models built from documented open and public-domain sources alongside synthetic data. The conversation examines the SYNTH dataset, gaps in common web crawls, preservation of source material and ethical choices beyond copyright status. It also covers compact deployed models, benchmark incentives, reasoning-trace access and competing approaches to national AI development. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Dhruv Batra connects embodied AI research with browser agents at Yutori. He explains why navigation from pixels to actions links virtual robots and web interaction, then discusses reinforcement learning on live websites, simulation-to-reality gaps and teleoperation. The conversation also examines competing views of intelligence, claims about scaling and the implications of agent-mediated browsing for the web. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Berkeley professor Ion Stoica discusses how research software becomes useful infrastructure and, sometimes, a company. The conversation draws on systems projects and companies associated with his laboratory to examine adoption, maintenance and the role of students. It also covers the complexity of the AI software stack, GPU utilization, coding agents in distributed systems, reward design and regulating outcomes. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

NVIDIA researcher Jean-Francois Puget discusses evaluating agent skills and distinguishing real improvements from benchmark overfitting. Drawing on Kaggle competitions, he emphasizes validation that separates development feedback from final evaluation. The episode also considers coding agents, multi-agent software development, model-release claims and his team’s approach to the ARC-AGI competition. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0

Dimitris Papailiopoulos of Microsoft Research and the University of Wisconsin discusses how coding agents change scientific research. The conversation covers small arithmetic transformers, symbolic mathematical reasoning, interacting agents, and the role of human judgment and verification. It also examines diversity in machine-generated ideas, benchmark and harness overfitting, persistent agent errors, continual learning, world models, and where information theory helps explain AI. Hosted by Ravid Shwartz Ziv and Allen Roush; the full conversation is available on the publisher's YouTube channel.

Hosted by Ravid Shwartz Ziv, Allen Roush

Plays 0