Skip to contentSkip to filters

We use cookies for analytics.

Understanding Language Model Hallucinations Through Parametric Knowledge Integration

Yue Dong

UC Riverside

Recording

47 min

Abstract

Language model hallucinations arise not only from gaps in knowledge but also from failures in how models retrieve, select and integrate information. This talk examines these failures from two complementary perspectives. It first presents a mechanistic account of non-factual hallucinations, identifying distinct failures in knowledge enrichment and answer extraction within the model's factual recall process. It then extends the analysis to long-context language models, examining how parametric knowledge interacts, and sometimes conflicts, with information provided in context. Stronger contextual retrieval does not necessarily lead to more reliable knowledge use and may even interfere with parametric recall. Together, these studies characterize hallucination as a failure of knowledge coordination, with implications for model interpretability, evaluation, mitigation and design.

Topics