Podcast, The Information Bottleneck, Episode 28
How to Control a Stochastic Agent with Stefano Soatto (VP AWS/ Pro. UCLA)
Hosted by Ravid Shwartz Ziv and Allen Roush
About
Stefano Soatto, vice president for AI at AWS and professor at UCLA, argues that building reliable AI agents is a control problem: language models behave as stochastic dynamical systems that must be controlled rather than merely prompted. He describes a coding approach between free-form and specification-driven coding, in which skeletons of AI functions are constrained by pre- and post-conditions so intent is verified before code is generated. The discussion also questions the divide between language models and world models, and distinguishes Shannon, algorithmic and conceptual information, explaining why synthetic data adds algorithmic rather than Shannon information.