Effects Of Word Predictability Research Articles

Large-scale evidence for logarithmic effects of word predictability on reading time

During real-time language comprehension, our minds rapidly decode complex meanings from sequences of words. The difficulty of doing so is known to be related to words' contextual predictability, but what cognitive processes do these predictability effects reflect? In one view, predictability effects reflect facilitation due to anticipatory processing of words that are predictable from context. This view predicts a linear effect of predictability on processing demand. In another view, predictability effects reflect the costs of probabilistic inference over sentence interpretations. This view predicts either a logarithmic or a superlogarithmic effect of predictability on processing demand, depending on whether it assumes pressures toward a uniform distribution of information over time. The empirical record is currently mixed. Here, we revisit this question at scale: We analyze six reading datasets, estimate next-word probabilities with diverse statistical language models, and model reading times using recent advances in nonlinear regression. Results support a logarithmic effect of word predictability on processing difficulty, which favors probabilistic inference as a key component of human language processing.

Stronger functional connectivity during reading contextually predictable words in slow readers

The effect of word predictability is well-documented in terms of local brain activation, but less is known about the functional connectivity among those regions associated with processing predictable words. Evidence from eye movement studies showed that the effect is much more pronounced in slow than in fast readers, suggesting that speed-impaired readers rely more on sentence context to compensate for their difficulties with visual word recognition. The present study aimed to investigate differences in functional connectivity of fast and slow readers within core regions associated with processing predictable words. We hypothesize a stronger synchronization between higher-order language areas, such as the left middle temporal (MTG) and inferior frontal gyrus (IFG), and the left occipito-temporal cortex (OTC) in slow readers. Our results show that slow readers exhibit more functional correlations among these connections; especially between the left IFG and OTC. We interpret our results in terms of the lexical quality hypothesis which postulates a stronger involvement of semantics on orthographic processing in (speed-)impaired readers.

A-maze of Natural Stories: Comprehension and surprisal in the Maze task

Behavioral measures of word-by-word reading time provide experimental evidence to test theories of language processing. A-maze is a recent method for measuring incremental sentence processing that can localize slowdowns related to syntactic ambiguities in individual sentences. We adapted A-maze for use on longer passages and tested it on the Natural Stories corpus. Participants were able to comprehend these longer text passages that they read via the Maze task. Moreover, the Maze task yielded useable reaction time data with word predictability effects that were linearly related to surprisal, the same pattern found with other incremental methods. Crucially, Maze reaction times show a tight relationship with properties of the current word, with little spillover of effects from previous words. This superior localization is an advantage of Maze compared with other methods. Overall, we expanded the scope of experimental materials, and thus theoretical questions, that can be studied with the Maze task.

Frequency and predictability effects in first and second language of different script bilinguals.

Reading efficiently in a second language (L2) is a crucial skill, but it is not universally achieved. Here we ask whether L2 reading efficiency is better captured as a language specific skill or whether it is mostly shared across L1 and L2, relying on general language abilities. To this end, we examined word frequency and predictability effects in sentence reading, and tested the same readers in L1 and L2, recording participants' eye-movements. Participants were 57 undergraduate bilingual speakers of Hebrew and English, languages that use different scripts, allowing for a clearer distinction between L1 and L2 processing. Both word frequency and word predictability effects were more pronounced in participants' L2 than in the L1, suggesting that both lower level and higher-order processes in reading are sensitive to language proficiency. Further, frequency effects in the L2 were linked with L2 proficiency but not general language abilities, and L2 predictability effects were not associated with either variable. Finally, readers' frequency and predictability effects in L1 and L2 were not associated with each other. Taken together, these results suggest that for different-script bilinguals, efficient reading in the L2 is a highly specific skill, dependent upon proficiency in that language, and drawing less on L1 and general language ability. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Robust Effects of Working Memory Demand during Naturalistic Language Comprehension in Language-Selective Cortex.

To understand language, we must infer structured meanings from real-time auditory or visual signals. Researchers have long focused on word-by-word structure building in working memory as a mechanism that might enable this feat. However, some have argued that language processing does not typically involve rich word-by-word structure building, and/or that apparent working memory effects are underlyingly driven by surprisal (how predictable a word is in context). Consistent with this alternative, some recent behavioral studies of naturalistic language processing that control for surprisal have not shown clear working memory effects. In this fMRI study, we investigate a range of theory-driven predictors of word-by-word working memory demand during naturalistic language comprehension in humans of both sexes under rigorous surprisal controls. In addition, we address a related debate about whether the working memory mechanisms involved in language comprehension are language specialized or domain general. To do so, in each participant, we functionally localize (1) the language-selective network and (2) the "multiple-demand" network, which supports working memory across domains. Results show robust surprisal-independent effects of memory demand in the language network and no effect of memory demand in the multiple-demand network. Our findings thus support the view that language comprehension involves computationally demanding word-by-word structure building operations in working memory, in addition to any prediction-related mechanisms. Further, these memory operations appear to be primarily conducted by the same neural resources that store linguistic knowledge, with no evidence of involvement of brain regions known to support working memory across domains.SIGNIFICANCE STATEMENT This study uses fMRI to investigate signatures of working memory (WM) demand during naturalistic story listening, using a broad range of theoretically motivated estimates of WM demand. Results support a strong effect of WM demand in the brain that is distinct from effects of word predictability. Further, these WM demands register primarily in language-selective regions, rather than in "multiple-demand" regions that have previously been associated with WM in nonlinguistic domains. Our findings support a core role for WM in incremental language processing, using WM resources that are specialized for language.

Effects of word predictability on eye movements during Arabic reading

Contextual predictability influences both the probability and duration of eye fixations on words when reading Latinate alphabetic scripts like English and German. However, it is unknown whether word predictability influences eye movements in reading similarly for Semitic languages like Arabic, which are alphabetic languages with very different visual and linguistic characteristics. Such knowledge is nevertheless important for establishing the generality of mechanisms of eye-movement control across different alphabetic writing systems. Accordingly, we investigated word predictability effects in Arabic in two eye-movement experiments. Both produced shorter fixation times for words with high compared to low predictability, consistent with previous findings. Predictability did not influence skipping probabilities for (four- to eight-letter) words of varying length and morphological complexity (Experiment 1). However, it did for short (three- to four-letter) words with simpler structures (Experiment 2). We suggest that word-skipping is reduced, and affected less by contextual predictability, in Arabic compared to Latinate alphabetic reading, because of specific orthographic and morphological characteristics of the Arabic script.

Word predictability effects are linear, not logarithmic: Implications for probabilistic models of sentence comprehension.

During language comprehension, we routinely use information from the prior context to help identify the meaning of individual words. While measures of online processing difficulty, such as reading times, are strongly influenced by contextual predictability, there is disagreement about the mechanisms underlying this lexical predictability effect, with different models predicting different linking functions - linear (Reichle, Rayner & Pollatsek, 2003) or logarithmic (Levy, 2008). To help resolve this debate, we conducted two highly-powered experiments (self-paced reading, N = 216; cross-modal picture naming, N = 36), and a meta-analysis of prior eye-tracking while reading studies (total N = 218). We observed a robust linear relationship between lexical predictability and word processing times across all three studies. Beyond their methodological implications, these findings also place important constraints on predictive processing models of language comprehension. In particular, these results directly contradict the empirical predictions of surprisal theory, while supporting a proportional pre-activation account of lexical prediction effects in comprehension.

A further look at ageing and word predictability effects in Chinese reading: Evidence from one-character words.

Older adults are thought to compensate for slower lexical processing by making greater use of contextual knowledge, relative to young adults, to predict words in sentences. Accordingly, compared to young adults, older adults should produce larger contextual predictability effects in reading times and skipping rates for words. Empirical support for this account is nevertheless scarce. Perhaps the clearest evidence to date comes from a recent Chinese study showing larger word predictability effects for older adults in reading times but not skipping rates for two-character words. However, one possibility is that the absence of a word-skipping effect in this experiment was due to the older readers skipping words infrequently because of difficulty processing two-character words parafoveally. We therefore took a further look at this issue, using one-character target words to boost word-skipping. Young (18–30 years) and older (65+ years) adults read sentences containing a target word that was either highly predictable or less predictable from the prior sentence context. Our results replicate the finding that older adults produce larger word predictability effects in reading times but not word-skipping, despite high skipping rates. We discuss these findings in relation to ageing effects on reading in different writing systems.

The Effect of Word Predictability on Phonological Activation in Cantonese Reading: A Study of Eye-Fixations and Pupillary Response.

This study aimed to investigate the effects of contextual predictability on orthographic and phonological activation during Chinese sentence reading by Cantonese-speaking readers using the error disruption paradigm. Participants' eye fixations and pupil sizes were recorded while they silently read Chinese sentences containing homophonic, orthographic, and unrelated errors. Sentences had varying amounts of contextual information leading up to target words such that some targets were more predictable than others. Results of the fixation time analysis indicated that orthographic effects were significant in first fixation and gaze duration, while phonological effects emerged later in total reading time. However, interactions between predictability and the homophonic condition were found in gaze duration. These results suggest that, while Cantonese readers activate word meanings primarily through orthography in early processing, early phonological activation can occur when facilitated by semantics in high-constraint sentence contexts. Analysis of pupillary response measurements revealed that participants' pupil sizes became larger when they read words containing orthographic errors, suggesting that orthographic error recovery processes significantly increase cognitive load.

Flexible parafoveal encoding of character order supports word predictability effects in Chinese reading: Evidence from eye movements.

Several eye-movement studies have revealed flexibility in the parafoveal processing of character-order information in Chinese reading. In particular, studies show that processing a two-character word in a sentence benefits more from parafoveal preview of a nonword created by transposing rather than replacing its two characters. One issue that has not been investigated is whether the contextual predictability of the target word influences this processing of character order information. However, such a finding would provide novel evidence for an early influence of context on lexical processing in Chinese reading. Accordingly, we investigated this issue in an eye-movement experiment using the boundary paradigm and sentences containing two-character target words with high or low contextual predictability. Prior to the reader's gaze crossing an invisible boundary, each target word was shown normally (i.e. a valid preview) or with its two characters either transposed or replaced by unrelated characters to create invalid nonword previews. These invalid previews reverted to the target word once the reader's gaze crossed the invisible boundary. The results showed larger preview benefits (i.e. a decrease in fixation times) for target words following transposed-character than substituted-character previews, revealing a transposed-character effect similar to that in previous research. In addition, a word predictability effect (shorter fixation times for words with high than low predictability) was observed following both valid and transposed-character previews, but not substituted-character previews. The findings therefore reveal that context can influence an early stage of lexical processing in Chinese reading during which character order is processed flexibly.

Dissociable effects of prediction and integration during language comprehension: evidence from a large-scale study using brain potentials.

Composing sentence meaning is easier for predictable words than for unpredictable words. Are predictable words genuinely predicted, or simply more plausible and therefore easier to integrate with sentence context? We addressed this persistent and fundamental question using data from a recent, large-scale (n = 334) replication study, by investigating the effects of word predictability and sentence plausibility on the N400, the brain's electrophysiological index of semantic processing. A spatio-temporally fine-grained mixed-effect multiple regression analysis revealed overlapping effects of predictability and plausibility on the N400, albeit with distinct spatio-temporal profiles. Our results challenge the view that the predictability-dependent N400 reflects the effects of either prediction or integration, and suggest that semantic facilitation of predictable words arises from a cascade of processes that activate and integrate word meaning with context into a sentence-level meaning. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

Older adults make greater use of word predictability in Chinese reading.

An influential account of normative aging effects on reading holds that older adults make greater use of contextual predictability to facilitate word identification. However, supporting evidence is scarce. Accordingly, we used measures of eye movements to experimentally investigate age differences in word predictability effects in Chinese reading, as this nonalphabetic language has characteristics that may promote such effects. Word-skipping rates were higher and reading times lower for more highly predictable words for both age groups. Effects of word predictability on word skipping did not differ across the 2 adult age groups. However, word predictability effects in reading time measures sensitive to both lexical identification (i.e., gaze duration) and contextual integration (i.e., regression-path reading times) were larger for the older than younger adults. Our findings therefore reveal that older Chinese readers make greater use of a word's predictability to facilitate both its lexical identification and integration with the prior sentence context. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Contextual predictability and phonetic attention

The interaction of contextual, high-level linguistic knowledge and the listener’s attention to low-level phonetic details has been the subject of a large body of research in speech perception for several decades. In the current paper, I investigate this interaction by considering the specific phenomenon of word predictability and its role in modulating the listener’s attention to subphonemic details of the acoustic signal. In the first experiment, subjects are presented with a discrimination task in which target words are presented in either predictable or unpredictable sentential context and then repeated in isolation, being either acoustically identical or subtly different. The subjects more accurately discriminate contextually unpredictable words, suggesting more attention to the phonetic details of words in unpredictable contexts. In the second experiment, considering the predictions of exemplar theory, I test whether this perceptual bias could result in changes in production. In this experiment, in which subjects heard and repeated sentences, I find a significant effect of word predictability on how close the subjects’ productions were to the model’s, which suggests a role of predictability on phonetic accommodation. The results of these experiments contribute to our understanding of stored exemplars and suggest the influence of contextual predictability in sound change.

Predictability effects during reading in the absence of parafoveal preview

ABSTRACTThe predictability of upcoming words facilitates both spoken and written language comprehension. One interesting difference between these language modalities is that readers’ routinely have access to upcoming words in parafoveal vision while listeners must wait for each fleeting word from a speaker. Despite readers’ potential glimpse into the future, it is not clear if and how this bottom-up information aids top-down prediction. The current study manipulated the predictability of target words and their location on a line of text. Targets were located in the middle of the line (preview available) or as the first word on a new line (preview unavailable). This represents an innovative method for manipulating parafoveal preview which utilises return sweeps to deny access to parafoveal preview of target words without the use of invalid previews. The study is the first to demonstrate gaze duration word predictability effects in the absence of parafoveal preview.

The Effects of Word Predictability and Contextual Uncertainty in the Processing of Korean Dative Sentences: An Eye-Movement Reading Comprehension Study

The Effects of Word Predictability and Contextual Uncertainty in the Processing of Korean Dative Sentences: An Eye-Movement Reading Comprehension Study

The predictability effect on eye movement in reading Korean dative sentences

Yun and Hong (2014) demonstrated the effect of role predictability and word predictability in the processing of Korean dative sentences. We aimed to investigate the nature of predictability effect that Yun and Hong observed, in particular, to explore whether the effect would reflect the early stage or the late stage of processing by observing readers` eye movement in reading. The significant effect of role predictability and word predictability were addictively observed in the measurement of early processing (i.e., first pass reading times) but not in the measurement of late processing (i.e., regression rates). Of interest, the effect of word predictability was mainly based on the processing of words of low role predictability. Our results indicate that when a thematic role for an upcoming word is not highly expected, word predictability contributes to the reduction of readers` integration difficulty, in a way that more predictable word is easier to be integrated into a sentence. We discuss our results in terms of Staub`s (2015) theoretical conclusion that the predictability effect depends on the process that multiple words are activated at once at graded degrees from the early stage. (Gachon University and Konkuk University)

The effects of word frequency and word predictability during first- and second-language paragraph reading in bilingual older and younger adults.

We used eye movement measures of paragraph reading to examine how word frequency and word predictability impact first-language (L1) and second-language (L2) word processing in matched bilingual older and younger adults, varying in amount of current L2 experience. Our key findings were threefold. First, across both early- and late-stage reading, word frequency effects were generally larger in older than in younger adults, whereas word predictability effects were generally age-invariant. Second, across both age groups and both reading stages, word frequency effects were larger in the L2 than in the L1, whereas word predictability effects were language-invariant. Third, graded differences in current L2 experience modulated L1 and L2 word processing in younger adults, but had no impact in older adults. Specifically, greater current L2 experience facilitated L2 word processing, but impeded L1 word processing among younger adults only. Taken together, we draw 2 main conclusions. First, bilingual older adults experience changes in word-level processing that are language-non-specific, potentially because lexical accessibility decreases with age. Second, bilingual older adults experience changes in word-level processing that are insensitive to graded differences in current L2 experience, potentially because lexical representations reach a functional ceiling over time. (PsycINFO Database Record

The effect of word predictability on reading time is logarithmic

It is well known that real-time human language processing is highly incremental and context-driven, and that the strength of a comprehender’s expectation for each word encountered is a key determinant of the difficulty of integrating that word into the preceding context. In reading, this differential difficulty is largely manifested in the amount of time taken to read each word. While numerous studies over the past thirty years have shown expectation-based effects on reading times driven by lexical, syntactic, semantic, pragmatic, and other information sources, there has been little progress in establishing the quantitative relationship between expectation (or prediction) and reading times. Here, by combining a state-of-the-art computational language model, two large behavioral data-sets, and non-parametric statistical techniques, we establish for the first time the quantitative form of this relationship, finding that it is logarithmic over six orders of magnitude in estimated predictability. This result is problematic for a number of established models of eye movement control in reading, but lends partial support to an optimal perceptual discrimination account of word recognition. We also present a novel model in which language processing is highly incremental well below the level of the individual word, and show that it predicts both the shape and time-course of this effect. At a more general level, this result provides challenges for both anticipatory processing and semantic integration accounts of lexical predictability effects. And finally, this result provides evidence that comprehenders are highly sensitive to relative differences in predictability – even for differences between highly unpredictable words – and thus helps bring theoretical unity to our understanding of the role of prediction at multiple levels of linguistic structure in real-time language comprehension.

Semantic similarity, predictability, and models of sentence processing

The effects of word predictability and shared semantic similarity between a target word and other words that could have taken its place in a sentence on language comprehension are investigated using data from a reading time study, a sentence completion study, and linear mixed-effects regression modeling. We find that processing is facilitated if the different possible words that could occur in a given context are semantically similar to each other, meaning that processing is affected not only by the nature of the words that do occur, but also the relationships between the words that do occur and those that could have occurred. We discuss possible causes of the semantic similarity effect and point to possible limitations of using probability as a model of cognitive effort.

Coregistration of eye movements and EEG in natural reading: Analyses and review.

Brain-electric correlates of reading have traditionally been studied with word-by-word presentation, a condition that eliminates important aspects of the normal reading process and precludes direct comparisons between neural activity and oculomotor behavior. In the present study, we investigated effects of word predictability on eye movements (EM) and fixation-related brain potentials (FRPs) during natural sentence reading. Electroencephalogram (EEG) and EM (via video-based eye tracking) were recorded simultaneously while subjects read heterogeneous German sentences, moving their eyes freely over the text. FRPs were time-locked to first-pass reading fixations and analyzed according to the cloze probability of the currently fixated word. We replicated robust effects of word predictability on EMs and the N400 component in FRPs. The data were then used to model the relation among fixation duration, gaze duration, and N400 amplitude, and to trace the time course of EEG effects relative to effects in EM behavior. In an extended Methodological Discussion section, we review 4 technical and data-analytical problems that need to be addressed when FRPs are recorded in free-viewing situations (such as reading, visual search, or scene perception) and propose solutions. Results suggest that EEG recordings during normal vision are feasible and useful to consolidate findings from EEG and eye-tracking studies.