Skip to contentSkip to filters

We use cookies for analytics.

Reliability Along The Way: Progress and Failure Signals In AI Agents

Sharon Li

University of Wisconsin-Madison

Recording

48 min

Abstract

LLM agents are increasingly deployed on long-horizon tasks with tool use, irreversible actions and unpredictable feedback, yet there are few principled ways to tell, mid-episode, whether an agent is on track or quietly failing. Most uncertainty quantification (UQ) research still centers on single-turn question answering, a poor match for interactive agents. This talk presents a general formulation of agent UQ and the challenges unique to agentic settings, from choosing uncertainty estimators to modeling how uncertainty evolves over an interaction. It then shows that reinforcement-learning post-training already yields an implicit step-level signal, the progress advantage, which recovers the optimal advantage function without annotation or reward-model training. Across test-time scaling, UQ and failure attribution, this free byproduct outperforms confidence baselines and even dedicated trained reward models.

Topics