45 min
Seminar, Hosted by Simons Institute for the Theory of Computing
Aligning Evaluation Incentives To Reduce Hallucination
RESI
Recording
Abstract
Large language models are typically evaluated with accuracy-based metrics that reward confident guessing while giving little or no credit for acknowledging uncertainty. Because evaluations influence which models and methods are selected, these incentives can work against techniques that reduce hallucinations at the cost of answering fewer questions. The talk discusses evaluation approaches that better reward useful uncertainty and reflect the differing costs of mistakes, correct answers and abstentions, together with open questions in designing evaluations whose incentives better match real-world objectives.
Topics
- hallucination
- uncertainty
- large language models
- accuracy-based metrics, search World Wide
- confident guessing, search World Wide
- hallucinations
- abstentions, search World Wide
- Evaluation Incentives, search World Wide
- evaluation approaches
- costs of mistakes, search World Wide
- real-world objectives, search World Wide