Natural Language Processing

The 2026 Conference on Empirical Methods in Natural Language Processing, organized by ACL's SIGDAT, held October 24-29, 2026 in Budapest, Hungary, with a main conference plus workshops and tutorials.

Susan Athey examines how machine learning and digital interventions can help explain and improve workers’ transitions between jobs. One project uses Swedish administrative data to identify groups whose earnings and employment are less resilient after layoffs, revealing substantial differences among workers within the same firms and labour markets. A second line of work uses transformer models and large language models to represent careers and analyse gender wage gaps, identifying settings in which large unexplained differences persist. Two further projects develop and evaluate digital interventions intended to help disadvantaged workers enter expanding occupations in information technology and data science. The lecture connects new methods for measuring labour-market disadvantage with evidence about practical interventions.

LSTMs trained on next word prediction can accurately perform linguistic tasks that require tracking long-distance syntactic dependencies. Notably, model accuracy approaches human performance on subject-verb number agreement tasks including cases with interfering attractors (Gulordava et al., 2018). However, we do not have a mechanistic understanding of how LSTMs track syntactic structures to perform such linguistic tasks. Do LSTM language models learn abstract grammatical rules like humans, or do they rely on simple heuristics and patterns? Here, we test this using long-distance gender agreement in French, which requires understanding both hierarchical syntactic structure and the inherent gender of lexical units. Our model is able to reliably predict gender agreement in two contexts without attractor nouns: noun-adjective and noun-passive-verb agreement. Whereas model performance on test cases with attractor nouns resulted in more inaccuracies, suggesting that LSTMs may not be sensitive to abstract syntactic generalisations. While humans employ knowledge of the inherent gender properties of nouns, it appears that LSTMs heavily rely on clues from gendered articles in test phrases. Overall, we introduce gender agreement tests as a probing method that facilitates further investigation into the underlying mechanisms, internal representations, and linguistic capabilities of LSTM language models. Gulordava K, Bojanowski P, Grave E, Linzen T, Baroni M. 2018. Colorless green recurrent networks dream hierarchically. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1, pp. 1195–1205. Stroudsburg, PA: Assoc. Comput. Linguist.

Stanford linguist and computer scientist Chris Manning discusses what linguistics contributed to machine learning and why pragmatics and dialogue remain open problems for language models. The conversation examines early abstraction of verb categories in small transformers, how distributed representations shape learning, and whether language alone can support meaningful representations. It also considers language in world models, diffusion language models, and representation finetuning (ReFT), which steers frozen models through their hidden states. The final discussion asks where knowledge resides and whether concepts occupy linear subspaces.

Join Yasuo Tabei’s Succinct Information Processing Team to research efficient AI systems. Topics include memory and knowledge management for language models and agents; reusable skills from agent execution histories; algorithms that reduce computing and memory requirements; and AI operating under device, edge and communication constraints. The researcher will formulate problems, implement methods, evaluate reliability and efficiency, publish papers and release software. The Tokyo position uses renewable annual contracts, with maximum terms of seven years for research scientists and five for postdoctoral researchers. Applications begin with the employer’s HR inquiry form, followed by submission instructions.

Alok Puranik (Jane Street) examines data mixing for large language models: allocating model capacity and computation among training sources. A small model’s preferred data mix may differ from that of a larger model, yet the cost of large-model training forces many decisions to depend on smaller experiments. Scaling laws can predict performance as size and computation increase only when trends remain stable or change predictably. The talk uses experiments from Puranik’s work and frontier laboratories to examine data-mix questions that violate these assumptions, explain why small-scale results can mislead, and explore better methods for choosing training data. His research at Jane Street concerns scaling laws for sequence models in trading.

Tal Linzen, an associate professor of linguistics and data science at New York University and a research scientist at Google, will speak in MIT's Siegel Family Quest for Intelligence seminar series. His work combines behavioral experiments and computational methods to study language learning and comprehension, alongside large-language-model post-training, evaluation, and interpretability.