Your Ranking Model Is Optimizing the Wrong Thing with Zoe Weil
Oct 15, 2026
The TWIML AI Podcast
How to Find the Agent Failures Your Evals Miss
Artificial Intelligence, Computer Science, Machine Learning and podcast with Scott Clark
The TWIML AI Podcast, Hosted by Sam Charrington
About
Scott Clark discusses operating language-model systems whose failures are difficult to anticipate with offline evaluations. He distinguishes telemetry, known-signal monitoring and production analytics, and describes using trace representations and clustering to discover emergent behaviour and feed improvements into evaluations and training.
Topics
Scott Clarklanguage-model systemsoffline evaluationstelemetryknown-signal monitoringproduction analyticstrace representationsclusteringEmergent Behaviourtraining
Related episodes
More from The TWIML AI Podcast
All episodesWhy Jev Is Changing How We Build With AI with Diogo Almeida
Oct 6, 2026The TWIML AI PodcastFrom Math Olympiads to Navier-Stokes: How Fast Is AI Progressing?
Sep 29, 2026The TWIML AI PodcastFrom Voice Agents to AI Avatars with Alexander Smola
Sep 16, 2026The TWIML AI Podcast
More with Sam Charrington
All their workDo AI Tokenomics Matter More Than Model Benchmarks? with Christopher Potts
Episode · Sep 9, 2026The TWIML AI PodcastWorld Models and the Future of Spatial AI with Justin Johnson
Episode · Sep 1, 2026The TWIML AI PodcastWhy the Next AI Breakthrough May Come from Physics with Max Welling
Episode · Aug 25, 2026The TWIML AI PodcastWhy Image Generation Needs More Than Bigger Models with Fatih Porikli
Episode · Aug 12, 2026The TWIML AI Podcast