Shared computational principles for language processing in humans and deep language models

Ariel Goldstein,Daniel Friedman,Michael P Brenner ,Mariano Schain,Lora Fanda,Omer Levy,Patricia Dugan,Colton Casto,Werner Doyle,Sasha Devore,Kenneth A Norman,Gina Choe,Lucía Melloni ,Samuel A Nastase,Adeen Flinker,Bobbi Aubrey,Liat Hasenfratz,Catherine Kim,Alon Cohen,Avinatan Hassidim,Harshvardhan Gazula,Eliav Buchnik,Dotan Emanuel,Aren Jansen,Amy Price,Yossi Matias,Roi Reichart,Orrin Devinsky,Zaid Zada,Aditi Rao,Amir Feder,Uri Hasson

doi:10.1038/s41593-022-01026-4

Abstract

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models generate appropriate linguistic responses in a given context. In the current study, nine participants listened to a 30-min podcast while their brain responses were recorded using electrocorticography (ECoG). We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process the same natural narrative: (1) both are engaged in continuous next-word prediction before word onset; (2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise; (3) both rely on contextual embeddings to represent words in natural contexts. Together, our findings suggest that autoregressive DLMs provide a new and biologically feasible computational framework for studying the neural basis of language.

Highlights

The outstanding success of autoregressive deep language models (DLMs) is striking from theoretical and practical perspectives because they have emerged from a very different scientific paradigm than traditional psycholinguist models[1]
The human brain and autoregressive DLMs share three computational principles: (1) both are engaged in continuous context-dependent next-word prediction before word onset; (2) both match pre-onset predictions to the incoming word to induce post-onset surprise; (3) both represent words using contextual embeddings
These findings provide the missing evidence that the brain, like autoregressive DLMs, is constantly involved in next-word prediction before word onset as it processes natural language (Fig. 1)

Summary

Introduction

The outstanding success of autoregressive (predictive) DLMs is striking from theoretical and practical perspectives because they have emerged from a very different scientific paradigm than traditional psycholinguist models[1]. Autoregressive DLMs have proven to be extremely effective in capturing the structure of language[6–9] It is unclear, if the core computational principles of autoregressive DLMs are related to the way the human brain processes language. Past research has leveraged language models and machine learning to extract semantic representation in the brain[10–18] Such studies did not view autoregressive DLMs as feasible cognitive models for how the human brain codes language. Recent theoretical papers argue that there are fundamental connections between DLMs and how the brain processes language[1,19,20] In agreement with this theoretical perspective, we provide empirical evidence that the human brain processes incoming speech to an autoregressive DLM (Fig. 1). We provide new evidence that the brain is spontaneously engaged in next-word prediction before word onset during the processing of natural language. These findings provide the missing evidence that the brain, like autoregressive DLMs, is constantly involved in next-word prediction before word onset as it processes natural language (Fig. 1)

Methods

Results

Discussion

Conclusion