48 min
Seminar, Hosted by Simons Institute for the Theory of Computing
Reliability Along The Way: Progress and Failure Signals In AI Agents
University of Wisconsin-Madison
Recording
Abstract
LLM agents are increasingly deployed on long-horizon tasks with tool use, irreversible actions and unpredictable feedback, yet there are few principled ways to tell, mid-episode, whether an agent is on track or quietly failing. Most uncertainty quantification (UQ) research still centers on single-turn question answering, a poor match for interactive agents. This talk presents a general formulation of agent UQ and the challenges unique to agentic settings, from choosing uncertainty estimators to modeling how uncertainty evolves over an interaction. It then shows that reinforcement-learning post-training already yields an implicit step-level signal, the progress advantage, which recovers the optimal advantage function without annotation or reward-model training. Across test-time scaling, UQ and failure attribution, this free byproduct outperforms confidence baselines and even dedicated trained reward models.