TRAILS, University of Maryland

Precedent-based reasoning, in law and in everyday life, rests on the idea that past decisions and their underlying reasons constrain future ones. Formal models of precedential constraint make this idea precise in terms of consistency: future decisions ought to be consistent with past ones. Ilaria Canavotto first introduces a standard model of precedential constraint and discusses how it could help people or AI systems identify the normative constraints that arise from a dataset of past decisions. She then presents a hierarchical generalisation of the model that captures more complex decisions proceeding through several levels of consideration, where decisions at one level determine the considerations relevant at the next. The talk draws on the papers "Piecemeal Knowledge Acquisition for Computational Normative Reasoning" and "Reasoning with hierarchies of open-textured predicates".

Large language models are increasingly used as agents that must learn from limited feedback, adapt to changing conditions and respond to other agents, so their usefulness depends on how well they decide over many rounds. Kaiqing Zhang uses regret, the cumulative performance measured against a benchmark chosen in hindsight, to evaluate this ability and to connect individual learning with equilibrium in repeated games. Experiments show that LLM agents can achieve low regret in some settings, while even GPT-4 incurs high regret on simple online learning problems. These failures motivate using regret as a training signal: by scoring entire decision trajectories, regret can guide training without expert action labels. The talk shows how minimising a regret-based loss can recover classical online learning algorithms, and how iteratively fine-tuning models on their own lowest-regret trajectories improves performance across models and tasks, including bandit problems presented in natural language. It draws on the papers "Do LLM Agents Have Regret? A Case Study in Online Learning and Games" and "Post-Training LLMs as Better Decision-Making Agents: A Regret-Minimization Approach".