Cloze Probability Research Articles

Quantification, prediction, and the online impact of sentence truth-value: Evidence from event-related potentials.

Do negative quantifiers like “few” reduce people’s ability to rapidly evaluate incoming language with respect to world knowledge? Previous research has addressed this question by examining whether online measures of quantifier comprehension match the “final” interpretation reflected in verification judgments. However, these studies confounded quantifier valence with its impact on the unfolding expectations for upcoming words, yielding mixed results. In the current event-related potentials study, participants read negative and positive quantifier sentences matched on cloze probability and on truth-value (e.g., “Most/Few gardeners plant their flowers during the spring/winter for best results”). Regardless of whether participants explicitly verified the sentences or not, true-positive quantifier sentences elicited reduced N400s compared with false-positive quantifier sentences, reflecting the facilitated semantic retrieval of words that render a sentence true. No such facilitation was seen in negative quantifier sentences. However, mixed-effects model analyses (with cloze value and truth-value as continuous predictors) revealed that decreasing cloze values were associated with an interaction pattern between truth-value and quantifier, whereas increasing cloze values were associated with more similar truth-value effects regardless of quantifier. Quantifier sentences are thus understood neither always in 2 sequential stages, nor always in a partial-incremental fashion, nor always in a maximally incremental fashion. Instead, and in accordance with prediction-based views of sentence comprehension, quantifier sentence comprehension depends on incorporation of quantifier meaning into an online, knowledge-based prediction for upcoming words. Fully incremental quantifier interpretation occurs when quantifiers are incorporated into sufficiently strong online predictions for upcoming words.

Studying Musical and Linguistic Prediction in Comparable Ways: The Melodic Cloze Probability Method.

Prediction or expectancy is thought to play an important role in both music and language processing. However, prediction is currently studied independently in the two domains, limiting research on relations between predictive mechanisms in music and language. One limitation is a difference in how expectancy is quantified. In language, expectancy is typically measured using the cloze probability task, in which listeners are asked to complete a sentence fragment with the first word that comes to mind. In contrast, previous production-based studies of melodic expectancy have asked participants to sing continuations following only one to two notes. We have developed a melodic cloze probability task in which listeners are presented with the beginning of a novel tonal melody (5–9 notes) and are asked to sing the note they expect to come next. Half of the melodies had an underlying harmonic structure designed to constrain expectations for the next note, based on an implied authentic cadence (AC) within the melody. Each such ‘authentic cadence’ melody was matched to a ‘non-cadential’ (NC) melody matched in terms of length, rhythm and melodic contour, but differing in implied harmonic structure. Participants showed much greater consistency in the notes sung following AC vs. NC melodies on average. However, significant variation in degree of consistency was observed within both AC and NC melodies. Analysis of individual melodies suggests that pitch prediction in tonal melodies depends on the interplay of local factors just prior to the target note (e.g., local pitch interval patterns) and larger-scale structural relationships (e.g., melodic patterns and implied harmonic structure). We illustrate how the melodic cloze method can be used to test a computational model of melodic expectation. Future uses for the method include exploring the interplay of different factors shaping melodic expectation, and designing experiments that compare the cognitive mechanisms of prediction in music and language.

Skipping syntactically illegal the previews: The role of predictability.

Readers tend to skip words, particularly when they are short, frequent, or predictable. Angele and Rayner (2013) recently reported that readers are often unable to detect syntactic anomalies in parafoveal vision. In the present study, we manipulated target word predictability to assess whether contextual constraint modulates the-skipping behavior. The results provide further evidence that readers frequently skip the article the when infelicitous in context. Readers skipped predictable words more often than unpredictable words, even when the, which was syntactically illegal and unpredictable from the prior context, was presented as a parafoveal preview. The results of the experiment were simulated using E-Z Reader 10 by assuming that cloze probability can be dissociated from parafoveal visual input. It appears that when a short word is predictable in context, a decision to skip it can be made even if the information available parafoveally conflicts both visually and syntactically with those predictions.

A “bag-of-arguments” mechanism for initial verb predictions

ABSTRACTPrevious studies have shown that comprehenders use rich contextual information to anticipate upcoming input on the fly, but less is known about how comprehenders integrate different sources of information to generate predictions in real time. The current study examines the time course with which the lexical meaning and structural roles of preverbal arguments impact comprehenders’ lexical semantic predictions about an upcoming verb in two event-related potential (ERP) experiments that use the N400 amplitude as a measure of online predictability. Experiment 1 showed that the N400 was sensitive to predictability when the verb's cloze probability was reduced by substituting one of the arguments (e.g. “The superintendent overheard which tenant/realtor the landlord had evicted … ”), but not when the verb's cloze probability was reduced by simply swapping the roles of the arguments (e.g. “The restaurant owner forgot which customer/waitress the waitress/customer had served … ”). Experiment 2 showed that argument substitution elicited an N400 effect even when the substituted argument appeared elsewhere in the sentence, indicating that verb predictions are specifically driven by the arguments in the same clause as the verb, rather than by a simple “bag-of-words” mechanism. We propose that verb predictions initially rely on a “bag-of-arguments” mechanism, which specifically relies on the lexical meaning, but not the structural roles, of the arguments in a clause.

The Effect of Lexical Predictability on Eye Movements in Reading: Critical Review and Theoretical Interpretation

AbstractA word's predictability in its context has a reliable influence on eye movements in reading. This article reviews the extensive literature that has investigated this influence, focusing on several specific empirical issues. These include assessment of cloze probability as the critical measure of predictability, the form of the relationship between predictability and reading time, the distributional effects of predictability, the interaction between predictability and word frequency, and the interaction between predictability and parafoveal preview. On the basis of this review, two theoretical conclusions are proposed. First, predictability effects in reading result from graded activation of potentially many words, as opposed to discrete prediction of a specific word. Second, this activation has the effect of facilitating either very early stages of lexical processing or pre‐lexical processing of visual features or letters.

P110. Networks for semantic integration in auditory speech comprehension

Introduction Semantic integration refers to the influence of previous context on the automatic integration of new information into a sentence during word comprehension. Neuroimaging studies have shown that sentence-level integration of semantic information is associated with activation of a left-lateralized fronto-temporal language network (Baumgaertner et al., 2002). The activation patterns are dependent on the difficulty of semantic integration which can be operationalised on the sentence-level through variation of the terminal word's expectancy, called probability. With this paradigm we attempted to characterize the properties of the language network relevant for semantic integration through multimodal neuroimaging examining structural and effective connectivity. Methods 17 young and healthy right-handed individuals were examined with functional MRI (fMRI) while they judged aurally presented sentences. Sentences were constructed following the pattern Der Schreiner baut den … (The Carpenter constructs the …) Tisch (desk=expected)/Turm (tower=unexpected)/Wein (wine=incongruent)/Kirst (pseudo word=anomalous). In a lexical decision task the subject has to decide if the last word of a sentence is a word (Tisch/Turm/Wein) or a pseudo word (Kirst). Dynamic causal modeling (DCM) for effective connectivity was calculated for 63 fully connected models between fMRI activation maxima in three regions. Additionally diffusion tensor imaging (DTI) sequences were acquired to assess structural connectivity between these regions. Results Behavioural data show the expected influence of cloze probability on lexical decision reaction times (expected nouns expected and incorrect>expected activation in the left inferior frontal gyrus (IFG) and the superior and medial temporal gyrus (STG/MTG) was increased ( p Discussion These results support the notion of a key contribution of the inferior frontal gyrus and the medial and superior temporal gyrus to semantic integration. DCM network analysis shows inhibitory influences on the pMTG as a consequence of increased difficulty of semantic integration. The functional relevance of these activation patterns and interactions will now be addressed in TMS experiments.

Corrigendum to “Effects of semantic constraint and cloze probability on Chinese classifier-noun agreement” [J Neurolinguistics (2014) 42–54

Corrigendum to “Effects of semantic constraint and cloze probability on Chinese classifier-noun agreement” [J Neurolinguistics (2014) 42–54

The influence of cloze probability and item constraint on cloze task response time

Abstract In research on the role of lexical predictability in language comprehension, predictability is generally defined as the probability that a word is provided as a sentence continuation in the cloze task (Taylor, 1953), in which subjects are asked to guess the next word of a sentence. The present experiments investigate the process by which subjects generate a cloze response, by measuring the latency to initiate a response in a version of the task in which subjects produce a spoken continuation to a visually presented sentence fragment. Higher probability responses were produced faster than lower probability responses. The latency to produce a response was also influenced by item constraint: A response at a given level of probability was issued faster when the context was more constraining, i.e., a single response was elicited with high probability. We show that these patterns are naturally produced by an activation-based race model in which potential responses independently race towards a response threshold. Implications for the interpretation of cloze probability as a measure of lexical predictability are discussed.

Auditory and motion metaphors have different scalp distributions: an ERP study.

While many links have been established between sensory-motor words used literally (kick the ball) and sensory-motor regions of the brain, it is less clear whether metaphorically used words (kick the habit) also show such signs of “embodiment.” Additionally, not much is known about the timing or nature of the connection between language and sensory-motor neural processing. We used stimuli divided into three figurativeness conditions—literal, metaphor, and anomalous—and two modality conditions—auditory (Her limousine was a privileged snort) and motion (The editorial was a brass-knuckle punch). The conditions were matched on a large number of potentially confounding factors including cloze probability. The electroencephalographic response to the final word of each sentence was measured at 64 electrode sites on the scalp of 22 participants and event-related potentials (ERPs) calculated. Analysis revealed greater amplitudes for metaphorical than literal sentences in both 350–500 ms and 500–650 ms timeframes. Results supported the possibility of different neural substrates for motion and auditory sentences. Greater differences for motion sentences were seen in the left posterior and left central electrode sites than elsewhere on the scalp. These findings are consistent with a sensory-motor neural categorization of language and with the integration of modal and amodal information during the N400 and P600 timeframes.

N-gram probability effects in a cloze task

What knowledge influences our choice of words when we write or speak? Predicting which word a person will produce next is not easy, even when the linguistic context is known. One task that has been used to assess context dependent word choice is the fill-in-the-blank task, also called the cloze task. The cloze probability of specific context is an empirical measure found by asking many people to fill in the blank. In this paper we harness the power of large corpora to look at the influence of corpus-derived probabilistic information from a word’s micro-context on word choice. We asked young adults to complete short phrases called n-grams with up to 20 responses per phrase. The probability of the responded word and the conditional probability of the response given the context were predictive of the frequency with which each response was produced. Furthermore the order in which the participants generated multiple completions of the same context was predicted by the conditional probability as well. These results suggest that word choice in cloze tasks taps into implicit knowledge of a person’s past experience with that word in various contexts. Furthermore, the importance of n-gram conditional probabilities in our analysis is further evidence of implicit knowledge about multi-word sequences and support theories of language processing that involve anticipating or predicting based on context.

A dataset of metaphors from the italian literature: exploring psycholinguistic variables and the role of context.

Defining the specific role of the factors that affect metaphor processing is a fundamental step for fully understanding figurative language comprehension, either in discourse and conversation or in reading poems and novels. This study extends the currently available materials on everyday metaphorical expressions by providing the first dataset of metaphors extracted from literary texts and scored for the major psycholinguistic variables, considering also the effect of context. A set of 115 Italian literary metaphors presented in isolation (Experiment 1) and a subset of 65 literary metaphors embedded in their original texts (Experiment 2) were rated on several dimensions (word and phrase frequency, readability, cloze probability, familiarity, concreteness, difficulty and meaningfulness). Overall, literary metaphors scored around medium-low values on all dimensions in both experiments. Collected data were subjected to correlation analysis, which showed the presence of a strong cluster of variables—mainly familiarity, difficulty, and meaningfulness—when literary metaphor were presented in isolation. A weaker cluster was observed when literary metaphors were presented in the original contexts, with familiarity no longer correlating with meaningfulness. Context manipulation influenced familiarity, concreteness and difficulty ratings, which were lower in context than out of context, while meaningfulness increased. Throughout the different dimensions, the literary context seems to promote a global interpretative activity that enhances the open-endedness of the metaphor as a semantic structure constantly open to all possible interpretations intended by the author and driven by the text. This dataset will be useful for the design of future experimental studies both on literary metaphor and on the role of context in figurative meaning, combining ecological validity and aesthetic aspects of language.

Effects of semantic constraint and cloze probability on Chinese classifier-noun agreement

This study aims to examine when and how readers make use of top-down information to predict or integrate upcoming words by utilizing the characteristics of Chinese classifier-noun agreement, as measured by event-related potentials (ERPs). Constraint strength of classifiers (strong and weak) and cloze probability of the pairing noun (high, low, implausible) was manipulated. Weakly constrained classifiers elicited a less positive P200 and an enhanced frontal negativity than strongly constrained classifiers, suggesting that readers used the preceding classifier to predict the upcoming noun, even before the pairing noun appeared. For ERPs elicited by the pairing nouns, there was a significant interaction between semantic constraint and cloze probability for the N400. For nouns following the weakly constrained classifiers, there was a graded cloze probability effect on the N400 (High < Low < Imp). For nouns following the strongly constrained classifiers, both low cloze and implausible nouns elicited larger N400s than high cloze nouns; however, there was no difference between low cloze and implausible nouns. The critical comparison for the constraint effect of low cloze nouns was found for the N400 but not for frontal

Predictability, plausibility, and two late ERP positivities during written sentence comprehension

Van Petten and Luka’s (2012, International Journal of Psychophysiology, 83(2), 176–190) literature survey of late positive ERP components elicited by more or less predictable words during sentence processing led them to propose two topographically and functionally distinct positivities: a parietal one associated with semantically incongruent words related to semantic reanalysis and a frontal one with unknown significance associated with congruent but lexically unpredicted words. With the goal of testing this hypothesis within a single set of experimental materials and participants, we report results from two ERP studies: Experiment 1, a post-hoc analysis of a dataset that varied on dimensions of both cloze probability (predictability) and plausibility, and Experiment 2, a follow-up study in which these factors were manipulated in a controlled fashion. In both studies, we observed distinct post-N400 positivities: a more anterior one to plausible, but not anomalous, low cloze probability sentence medial words, and a more posterior one to semantically anomalous sentence continuations. Taken together with an observed canonical cloze-modulated N400, these dual positivities indicate a dissociation between brain processes relating to written words׳ sentential predictability versus plausibility, clearly an important distinction for any viable neural or psycholinguistic model of written sentence processing.

From literal meaning to veracity in two hundred milliseconds.

Do the integration of semantic information and that of world knowledge occur simultaneously or in sequence during sentence processing? To address this question, we investigated event-related brain potentials elicited by the critical word of English sentences in three conditions: (1) correct; (2) semantic violation; (3) world knowledge violation (semantically correct but factually incorrect). Critically, we opted for low constraint sentence contexts (i.e., whilst being semantically congruent with the sentence context, critical words had low cloze probability). The processing of semantic violations differed from that of correct sentences as early as the P2 time-window. In the N400 time-window, the processing of semantic and world knowledge violations both differed significantly from that of correct sentences and differed significantly from one another. Overall, our results show that the brain needs approximately 200 ms more to detect a world knowledge violation than a semantic one.

Predictability speeds up the re-analysis process: An ERP investigation of gender agreement and cloze probability

To investigate the timing relationship between lexical access and later processes, the present study compared event-related potentials (ERPs) in response to spoken Dutch sentences that were either correct or contained gender agreement violations on the article or adjective preceding the noun. The target noun was either unpredictable from the preceding sentence information (low cloze) or was preceded by a highly constraining context (high cloze) to investigate whether contextual constraints influence morphological processing of grammatical gender in real-time. In line with previous findings, gender violations elicited a clear P600 indicating processes of repair or re-analysis evoked by the gender mismatch. Low cloze items, independent of the gender mismatch, elicited an increased N400 reflecting lexical access and semantic integration difficulty. Interestingly, an interaction between gender mismatch and cloze probability occurred in the early portion of the P600 time window, with the P600 starting significantly later in the low cloze conditions as compared to the high cloze conditions, although they had the same amplitude in the later part of the window. These results indicate that semantic expectancy facilitates the late processes sensitive to syntactic violations, suggesting an interplay between semantics and syntax in later stages of processing and supporting interactive accounts of language comprehension.

Hemispheric asymmetry in interpreting novel literal language: An event-related potential study

Conceptual mapping, or making connections between conceptual structure in different domains, is a key mechanism of creative language use whose neural underpinnings are not well understood. The present study involved the combination of event-related potentials (ERPs) with the divided visual field presentation technique to explore the relative contributions of the left and right hemispheres (LH and RH) to the construction of novel meanings in fully literal language. Electroencephalogram (EEG) was recorded as healthy adults read sentences that supported either a conventional literal reading of the sentence final word (“His main method of transportation is a boat,”), or a novel literal meaning derived from conceptual mapping (“The clever boys used a cardboard box as a boat,”). The novel and conventional conditions were matched for cloze probability (a measure of predictability based on the sentence context), lexical association between the sentence frame and the final word (using latent semantic analysis), and other factors known to influence ERPs to language stimuli. To compare effects of novelty to previously reported effects of predictability, a high-cloze conventional condition (“The only way to get around Venice is to navigate the canals in a boat.”) was included. ERPs were time-locked to sentence final words (“boat”) presented in either the left visual field, to preferentially stimulate the RH (lvf/RH), or in the right visual field, targeting the LH (rvf/LH). The N400 component of the ERP was affected by predictability in both presentation sides, but by novelty only in rvf/LH. Two distinct late frontal positive effects were observed. Word predictability modulated a frontal positivity with a LH focus, but semantic novelty modulated a frontal positivity focused in RH. This is the first demonstration that the frontal positivity may be composed of multiple overlapping components with distinct functional and anatomical characteristics. Extending contemporary accounts of the frontal positivity, we suggest that both frontal positivities reflect learning mechanisms involving prediction based on statistical regularities in language (LH) and world knowledge (RH).

Investigating the influence of proficiency on semantic processing in bilinguals: an ERP and ERD/S analysis.

In this study, we presented sentences either ending with high or low probability cloze words or semantically incongruent words to investigate the influence of L2 proficiency on electrophysiological correlates of semantic processing in bilinguals. Event-related potentials (ERPs) as well as the oscillatory dynamics of the EEG signal, specifically, frequency power changes expressed as event-related (de)synchronization (ERD/S), were analyzed. Replicating earlier results, we found an N400 on semantically incongruent words, as well as on low cloze probability words. For the bilinguals investigated in the present study, N400 latency in the low cloze probability condition was found to be modulated by L2 proficiency, indicating that L2 proficiency in our sample might have influenced the speed of semantic integration. Relative Theta power increased for all three word conditions, but no influence of proficiency was observed. Different from the ERP results, we found a stronger increase in theta power for low cloze probability words than for incongruent and high cloze probability words, especially over temporo-parietal brain areas. The spatial distribution of the theta ERD/S results also differed from the N400 topography. Whereas the N400 showed a typical topography with a maximum over temporo-posterior electrode positions, the theta ERD/S topography was maximal for high and low cloze probability words over left central-posterior electrode positions. These findings show that the ERP and ERD/S results are sensitive to semantic processing. The different pattern of the ERD/S results compared to the ERP results, functionally and with regard to topography, suggests that the ERD/S reflects a different aspect or stage of semantic processing, possibly the successful conceptualization of a sentence.

“If a lion could speak …”: Online sensitivity to propositional truth-value of unrealistic counterfactual sentences

People can establish whether a sentence is hypothetically true even if what it describes can never be literally true given the laws of the natural world. Two event-related potential (ERP) experiments examined electrophysiological responses to sentences about unrealistic counterfactual worlds that require people to construct novel conceptual combinations and infer their consequences as the sentence unfolds in time (e.g., “If dogs had gills…”). Experiment 1 established that without this premise, described consequences (e.g., “Dobermans would breathe under water …”) elicited larger N400 responses than real-world true sentences. Incorporation of the counterfactual premise in Experiment 2 generated similar N400 effects of propositional truth-value in counterfactual and real-world sentences, suggesting that the counterfactual context eliminated the interpretive problems posed by locally anomalous sentences. This result did not depend on cloze probability of the sentences. In contrast to earlier findings regarding online comprehension of logical operators and counterfactuals, these results show that ongoing processing can be directly impacted by propositional truth-value, even that of unrealistic counterfactuals.

Sentence comprehension in proficient adult cochlear implant users: On the vulnerability of syntax

Comprehending language via a cochlear implant, a neural prosthesis which stimulates the acoustic nerve electrically, poses a remarkable challenge to the brain. We investigated auditory sentence comprehension in a group of thirteen high proficient adult cochlear implant patients using event-related brain potentials. Four types of sentences were examined: correct sentences with either high (1) or low (2) expectancy of the sentence final word (cloze probability), semantically incorrect sentences (3), and sentences violating the argument structure of the verb (4). Participants judged the acceptability of the sentences. Relative to correct sentences with a high cloze probability all other conditions elicited a N400 effect in both the patient group and a matched control group, although the timing of the effect differed between the two groups. Moreover, whereas the argument structure violation elicited a late positivity in addition to the N400 effect in the control group, no such effect was observable in the cochlear implant group. We take these data to indicate that under adverse input conditions, processes of syntactic repair reflected in the P600 effect, are much more vulnerable than processes of semantic integration reflected in N400 effects.

Lexical versus conceptual anticipation during sentence processing: Frontal positivity and N400 ERP components

Although the sensitivity of the N400 to semantic processing is well established, late positive ERP components are also elicited during sentence comprehension. We suggest that there are multiple such components differing in scalp topography, and that a larger frontal positivity often follows the larger N400 elicited by congruent but unexpected sentence endings as compared to predictable endings. We evaluated the lexical versus conceptual specificity of this post-N400-positivity. High- and low-constraint sentences were completed by the words most preferred by a normative group (best completions), by words that were nearly synonymous to those best completions, and by other congruent words that were semantically dissimilar to the best completions. The N400 was sensitive to both the predictability (cloze probability) of the words and their semantic similarity to the best completion, consistent with a sensitivity to conceptual expectations that could be fulfilled by alternate words. In contrast, an anterior positivity was elicited by all final words that were not highly predictable, independent of whether they were semantically similar or dissimilar to the most preferred word, indicating a sensitivity to specific lexical expectations.