ePoster

Expectation management in humans and LLMs

Benjamin Menasheand 2 co-authors

Presenting Author

Conference
COSYNE 2025 (2025)
Montreal, Canada

Conference

COSYNE 2025

Montreal, Canada

Resources

Authors & Affiliations

Benjamin Menashe, Austin Drake, Michal Ben-Shachar

Abstract

Mirative markers, such as “surprisingly”, are linguistic elements that encode a violation of expectations (Delancey, 1997, 2012; Aikhenvald, 2012). Such markers are used for expectation management during communication, informing the listener about an upcoming unexpected message. Sensitivity to mirative markers relies on two abilities: i) updating expectations upon recognizing a mirative marker, and ii) identifying expectation violations warranting the use of a mirative marker. The aim of this study is to quantify and compare sensitivity to mirative markers in humans and large language models (LLMs). In part 1, we used a sentence-completion task, where humans and LLMs were presented with sentence fragments and asked to continue them. We measured the impact of starting the sentence fragments with a mirative marker (“surprisingly”). Results show that for both humans and LLMs, mirative markers significantly increase response entropy and decrease top response probability, in line with theoretical proposals for mirative markers as operators on alternative expectations (Zanuttini \& Portner, 2003; Simeonova, 2015). Subsequent analyses using LLMs reveal that mirative markers’ effects are uncorrelated with contextual constraint and are limited in scope to two sentences ahead. In part 2, we created a novel task of mirative polarity selection where humans and LLMs are presented with a sentence pair and asked to select whether it was connected by a mirative marker (“surprisingly”) or by an anti-mirative marker (“unsurprisingly”). Results show that LLMs perform at an impressive human level. Subsequent analyses suggest that LLMs’ performance cannot be explained by simple heuristics or data contamination. However, LLMs, but not humans, are strongly influenced by the choice order presented. We conclude that both humans and LLMs use mirative markers as cues for calibrating their subsequent expectations during sentence processing. Moreover, both humans and LLMs accurately identify when mirative markers are appropriate, albeit with different biases.

Unique ID: cosyne-25/expectation-management-humans-llms-5da10bd1