Large-scale replication study reveals a limit on probabilistic prediction in language comprehension.

Mante S Nieuwland ,Evelien Heyselaar ,Jyrki Tuomainen ,Heather J Ferguson ,Shirley Ann Rueschemeyer ,Eugenia Kulakova ,Nina Kazanina ,Katrien Segaert ,Dale J Barr ,Emily Darley ,E Matthew Husband ,Xiao Fu ,Stephen Politzer-Ahles ,Federica Bartolozzi ,Diane Mézière ,David I Donaldson ,Sarah Von Grebmer Zu Wolfsthurn ,Aine Ito ,Guillaume A Rousselet ,Simon Busch‐Moreno ,Vita Kogan ,Zdenko Kohút ,Falk Huettig

doi:10.7554/elife.33468

Abstract

Do people routinely pre-activate the meaning and even the phonological form of upcoming words? The most acclaimed evidence for phonological prediction comes from a 2005 Nature Neuroscience publication by DeLong, Urbach and Kutas, who observed a graded modulation of electrical brain potentials (N400) to nouns and preceding articles by the probability that people use a word to continue the sentence fragment ('cloze'). In our direct replication study spanning 9 laboratories (N=334), pre-registered replication-analyses and exploratory Bayes factor analyses successfully replicated the noun-results but, crucially, not the article-results. Pre-registered single-trial analyses also yielded a statistically significant effect for the nouns but not the articles. Exploratory Bayesian single-trial analyses showed that the article-effect may be non-zero but is likely far smaller than originally reported and too small to observe without very large sample sizes. Our results do not support the view that readers routinely pre-activate the phonological form of predictable words.

Highlights

Planck Institute for Psycholinguistics, Nijmegen, The Netherlands of Psychology, University of Birmingham, Birmingham, United Kingdom of Experimental Psychology, University of Bristol, Bristol, United Kingdom of Philosophy, Psychology and Language Sciences, University of Edinburgh, Edinburgh, United Kingdom
In the last few decades, the idea that people routinely and implicitly predict upcoming words during language comprehension has turned from a controversial hypothesis to a widelyaccepted assumption
Current theories of language comprehension[1,2,3] posit prediction, or context-based pre-activation, as an essential mechanism occurring at all levels of linguistic representation and facilitating the integration of words into the unfolding discourse representation

Summary

Introduction

Applying the replication analysis to our article data (Figure 1a), the original finding did not replicate: no laboratory observed a significant negative relationship between cloze and N400 at central-parietal electrodes. In the single-trial analysis (Fig. 1b-c), there was no statistically significant effect of cloze on article-N400s, with stricter control for prearticle voltage levels (Supplementary Fig. 1).

Results

Conclusion