Adjacent Words Research Articles

Why do readers fail to notice word transpositions, omissions, and repetitions? A review of recent evidence and theory

AbstractMost readers have had the experience of initially failing to notice an omission or repetition of a function word, or a transposition of two adjacent words. In the present article, we review recent research investigating this phenomenon. We emphasize that failure to notice such errors is of substantial theoretical interest, given what we have learned about how systematically and incrementally readers inspect and process text. We endorse the idea that a process of rational inference may play a critical role, while we cast doubt on the idea that failure to notice errors arises from parallel processing of multiple words. We review a number of recent studies from our own laboratory that have investigated the relationship between eye movements during reading and noticing, or failing to notice, an error. While the conclusions from these studies are broadly consistent with a rational inference account, we find that when readers fail to notice an error, their eye movements generally show no indication that the error was registered at all. On its surface, this finding may be viewed as inconsistent with the idea that the rational inference process that enables readers to overlook errors is genuinely post‐perceptual. We suggest a mechanism by which eye movement control models could account for this finding.

Word Order in Complex Verb Phrases in Heritage Polish Spoken in Germany

This paper deals with word order in complex verb phrases consisting of auxiliaries and infinitive complements in heritage Polish. In Polish, infinitive complements normally follow auxiliaries, but discontinuous structures occur if required by the information structure. We investigate the occurrence and evaluation of adjacent and discontinuous word order patterns in relation to (a) the chronological age at testing and (b) the age of onset of the acquisition of the majority language, German. Therefore, we distinguish between simultaneous bilinguals (2L1, n = 61), early sequential bilinguals (cL2, n = 41) and an age-matched monolingual control group (ML, n = 50). The data consist of elicited oral narratives as well as acceptability judgments. We found that both 2L1 and cL2 bilinguals differ from the ML, but the difference depends on the age at testing and the type of data (oral production or evaluation). While 2L1 bilinguals show a u-curve development, which is shaped by the interplay of delayed acquisition in childhood and attrition in early adulthood, cL2 bilinguals started to prefer discontinuous structures rather early. Only in adulthood do both groups converge and exhibit an overuse and over-acceptance of discontinuous structures compared to the ML, which is due to cross-linguistic influence from German. However, language-internal factors (such as clause structure) also turned out to impact the distribution of adjacent and discontinuous structures in heritage (and monolingual) Polish.

Multi-level graph neural network for text sentiment analysis

Text sentiment analysis is a fundamental task in the field of natural language processing (NLP). Recently, graph neural networks (GNNs) have achieved excellent performance in various NLP tasks. However, a GNN only considers the adjacent words when updating the node representations of the graph, and thus the model can only focus on the local features while ignoring global features. In this paper, we propose a novel multi-level graph neural network (MLGNN) for text sentiment analysis. To consider both local features and global features, we apply node connection windows with different sizes at different levels. Particularly, we integrate a scaled dot-product attention mechanism as a message passing mechanism into our method for fusing the features of each word node in the graph. The experimental results demonstrated that the proposed model outperformed other models in text sentiment analysis tasks.

Relative Clause Sentence Processing in Korean-Speaking School-Aged Children With and Without Specific Language Impairment.

Purpose The goal of this study was to examine online and off-line sentence processing using Korean language relative clause sentences between children with specific language impairment (SLI) and children with typical development (TD). Method Twenty-four children with TD and 19 children with SLI participated in this study. Children completed online and off-line sentence-processing tasks using relative clause sentences. The response time (RT) data obtained from the online processing task were analyzed at each word position and between adjacent words for items answered both correctly and incorrectly on the off-line comprehension task. A linear mixed-effects model and a generalized linear mixed-effects model were used to analyze the performances on the online/off-line sentence-processing task between the two groups. Results The results revealed that the processing pattern of RTs on the online processing task differed between the two groups, such that the SLI group did not show the predicted RT increase while the TD group did. Also, the SLI group processed each word with comparable or faster reading rates than the TD group. On the off-line comprehension task, the SLI group performed poorly compared to the TD group. Conclusions Processing of syntactically complex sentences differed between the TD and SLI groups, such that the SLI group had lower accuracy on the off-line comprehension task and was less efficient on the online processing task as compared to the TD group. These results mainly support the syntactic deficit account in children with SLI.

Fast syntax in the brain: Electrophysiological evidence from the rapid parallel visual presentation paradigm (RPVP).

In 2 ERP experiments participants read 4-word sequences presented for 200 ms (RPVP paradigm) and were required to decide whether the word sequences were grammatical or not. In Experiment 1, the word sequence consisted of either a grammatically correct sentence (e.g., she can sing now) or an ungrammatical scrambled sequence (e.g., sing can now she). A reduced N400 effect was obtained in the grammatically correct sequences compared to the ungrammatical sequences. In Experiment 2, the critical comparison was between 2 types of ungrammatical sequences: transposed-word sequences (e.g., you that read wrong, transposing 2 adjacent central words can form a grammatical sequence) and control sequences (e.g., you that read worry, transposing any 2 adjacent central words still forms an ungrammatical sequence). An N400 reduction was observed in the transposed-word sequences relative to the control sequences. We interpret these N400 effects as evidence that an elementary syntactic representation can be rapidly constructed on the basis of parallel processing of word identities and their parts-of-speech. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Repetition causes confusion: Insights to word segmentation during Chinese reading.

Since there are no spaces between words to mark word boundaries in Chinese, it is common to see 2 identical neighboring characters in natural text. Usually, this occurs when there are 2 adjacent words containing the same character (we will call such a coincidental sequence of 2 identical characters repeated characters). In the present study, we examined how Chinese readers process words when there are repeated characters. In 3 experiments, we compared how Chinese readers process 4-character strings including 2 repeated characters (e.g. , pinyin: xíngdòng dòngjī, meaning behavioral motivation) with a control condition where none of the characters repeat (e.g. , pinyin: xíngdòng yùwàng, meaning behavioral desire). In Experiment 1, the 4-character strings were presented for 40 ms and participants were asked to report as many characters as possible. Participants reported the second and third characters less accurately in the repeated condition than the control condition. In Experiments 2A and 2B, we embedded 2 different types of 4-character strings, compound Chinese characters and simple Chinese characters, into the same sentence frames, and asked participants to read these sentences normally. Gaze duration and total time on the second word were significantly longer in the repeated condition. These results suggest that the repeated characters increased the difficulty of word processing. Moreover, the results are consistent with the predictions of serial models, which assumes that words are processed serially in reading. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The stylistic functions and types of Zeugma

This article is devoted to the stylistic functions and types of zeugma. Zeugma is a fun rhetorical device, an important figure of speech, wherein a single word is used to denote two or more words in a sentence and is fraught with literal and metaphorical undertones. Like most other figures of speech, even this rhetoric device draws its name from the Greek word ‘zeugma’, meaning ‘yoking’ or ‘bonding.’ Zeugma is the use of a word in the same grammatical, but different semantic relations to two adjacent words in the context, the semantic relations being on the one hand literal and on the other, transferred. Dora was plunging at once into privileged intimacy and into the middle of the room. This stylistic device is particularly favored in English emotive prose and in poetry. The revival of the original meanings of words must be regarded as an essential quality of any work in the belles-lettres style. A good writer always keeps the chief meanings of words from fading away, provided the meanings are worth being kept fresh and vigorous.

Research of Attention-Based Bi-GRU-CRF for Slot Filling

Slot Filling (SF) is a critical part of spoken language understanding (SLU) which targets to capture semantic constituents from a specific utterance. It is considered as a sequence labeling issue. Currently, recurrent neural networks have shown promising effectiveness in this issue. Considering the effects of interrelated information within adjacent words and labels, we present that a novel approach consists of bi-directional gate recurrent unit (Bi-GRU), attention mechanism, and conditional random field (CRF). Our model can utilize interrelated information from words in the neighborhood, highlight key information, and exploit the dependencies within labels corresponding to surrounding words. The empirical experiments illustrate that our model significantly boosts the F1 score with around 1% and 5.1% relative enhancement on two public benchmark dataset ATIS and SNIPS separately.

Formulaicity in constrained communication An intermodal approach

In this exploratory study bordering on corpus linguistics, formulaic language and studies on constrained communication (focusing on translation, interpreting, and L2) we aim to verify whether constrained texts found in the Polish-English component of an intermodal EPTIC corpus differ from native texts in terms of use of adjacent word combinations commonly known as bigrams and whether similar patterns can be found across spoken and written registers. To that end, we fit a Poisson regression model with fixed and random effects. The results show that the translated language variety contributes to the higher number of the most frequent bigram types in both spoken and written registers, and that the number of frequent bigrams in texts generally increases when the speech/source speech is delivered impromptu, but the effect is significant only for the written register. The findings reveal the considerable impact of individual variation on formulaicity as most of the bigram variation within both models is explained by text-specific random variables rather than fixed variables.

Poisson-Gamma Latent Dirichlet Allocation Model for Topics with Word Dependencies

This paper introduces the Poisson-Gamma Latent Dirichlet Allocation (PGLDA) model for modeling word dependencies in topics modeling. The Poisson document length distribution has been used extensively in the past for modeling topics with the expectation that its effect will fizzle out at the end of the model definition. This procedure often leads to downplaying the effect of word correlation with topics and thus reducing the precision or accuracy of retrieved documents in such a situation. Therefore, we propose a new class of model that relaxes the words independence assumption in the existing Latent Dirichlet Allocation (LDA) model by introducing the Gamma distribution that can capture the correlation between adjacent words in a document. The Poisson document length distribution and Gamma correlation distribution are then convoluted to form a new mixture distribution for modeling word dependencies. Model parameter estimation was achieved via Laplacian approximation of the log-likelihood. The new model was then evaluated using the 20 Newsgroups and AG's News datasets. The applicability of the model was assessed using the F1 score. The results of the evaluation showed appreciable supremacy of PGLDA over LDA.

Общеотрицательные предложения с приглагольным типом отрицания в английском языке

The research objective includes identifying structural-semantic models of general negative sentences in the English language. The authors examine the category of adverbial negation, analyse the structure of a general negative sentence and reveal sentential meaning of its elements, which constitutes scientific originality of the study. The conducted research allows concluding that general negative sentences can express direct negation (action itself is completely negated) and shifted negation (negation does not affect the verb but is shifted to adjacent words). Adverbial negation can affect a wide range of the sentence elements: the verb itself and a group of adjacent words.

Adjacent and Non-Adjacent Word Contexts Both Predict Age of Acquisition of English Words: A Distributional Corpus Analysis of Child-Directed Speech.

Children show a remarkable degree of consistency in learning some words earlier than others. What patterns of word usage predict variations among words in age of acquisition? We use distributional analysis of a naturalistic corpus of child-directed speech to create quantitative features representing natural variability in word contexts. We evaluate two sets of features: One set is generated from the distribution of words into frames defined by the two adjacent words. These features primarily encode syntactic aspects of word usage. The other set is generated from non-adjacent co-occurrences between words. These features encode complementary thematic aspects of word usage. Regression models using these distributional features to predict age of acquisition of 656 early-acquired English words indicate that both types of features improve predictions over simpler models based on frequency and appearance in salient or simple utterance contexts. Syntactic features were stronger predictors of children's production than comprehension, whereas thematic features were stronger predictors of comprehension. Overall, earlier acquisition was predicted by features representing frames that select for nouns and verbs, and by thematic content related to food and face-to-face play topics; later acquisition was predicted by features representing frames that select for pronouns and question words, and by content related to narratives and object play.

On the noisy spatiotopic encoding of word positions during reading: Evidence from the change-detection task

The present study builds on our prior work showing evidence for noisy word-position coding in an immediate same-different matching task. In that research, participants found it harder to judge that two successive brief presentations of five-word sequences were different when the difference was caused by transposing two adjacent words compared with different word replacements – a transposition effect. Here we used the change-detection task with a 1-s delay introduced between sequences – a task thought to tap into visual short-term memory. Concurrent articulation was used to limit the contribution of active rehearsal. We used standard response-time (RT) and error-rate analyses plus signal detection theory (SDT) measures of discriminability (d’) and bias (c). We compared the transposition effects for ungrammatical word sequences and nonword sequences observed with these different measures. Although there was some evidence for transposition effects with nonwords, the effects were much larger with word sequences. These findings provide further support for the hypothesized noisy assignment of word identities to spatiotopic locations along a line of text during reading.

A deep learning-based social media text analysis framework for disaster resource management

Social media has evolved itself as a significant tool used by people for information spread during emergencies like natural or man-made disasters. Real-time analysis of this huge collected data can play a vital role in crisis estimation, response and assistance exercises. We propose a novel prototype system that analyzes the emergency-related tweets to classify them as need or available tweets. The presented system also takes care of non-English tweets as there is no boundary of language for social media users. Several classifiers along with different learning methodologies are used to show their usefulness for an efficient solution. Here, a new supervised learning technique based on word embedding is incorporated in the novel hybrid model that comprises of LSTM and CNN. The system will further give a ranked list of tweets, along with a relevance score for each tweet with respect to the topic. Finally for each of the identified need tweets, its corresponding availability tweets are mapped. For the mapping task, a novel two-word sliding window approach is proposed to generate the combine embedding of two adjacent words. The experimental results show significant improvement in the performance. We evaluate our proposed system with FIRE-2016 and CrisisLex datasets to illustrate its effectiveness during mobilization of needful resources.

A transposed-word effect in Chinese reading.

Studies using a grammaticality decision task suggest surprising flexibility in the processing of the relative order of words in sentences when reading alphabetic scripts like French. In these studies, participants made rapid grammaticality decisions for ungrammatical stimuli created by transposing two adjacent words in either a grammatical or an ungrammatical base sentence, which were intermixed with equal numbers of grammatically correct stimuli. The key finding was that participants made more errors and were slower to reject transposed-word stimuli created from grammatical than ungrammatical base sentences. This suggested that flexibility in the processing of word order allowed participants to access representations of the base grammatical sentences, interfering with their decisions to correctly reject transposed-word stimuli. With the present research, we investigated if a similar transposed-word effect is observed for a non-alphabetic script (Chinese) that uses few grammatical markers and primarily conveys grammatical structure via word order. Such scripts may require stricter processing of word order during reading and so provide a strong test of the cross-linguistic generality of the transposed-word effect. We report three experiments using the same design and procedure as previous research, while varying the length of the transposed words across experiments. In all three experiments, participants made more errors and were slower to reject transposed-word stimuli derived from grammatical than ungrammatical base sentences. This replicates previous findings with alphabetic scripts and provides novel evidence for a transposed-word effect in Chinese reading. We consider the implications for models of reading in alphabetic and non-alphabetic scripts.

Attribute Sentiment Scoring with Online Text Reviews: Accounting for Language Structure and Attribute Self-Selection

The authors address two significant challenges in using online text reviews to obtain finegrained attribute level sentiment ratings. First, in contrast to methods that rely on word frequency, they develop a deep learning convolutional-LSTM hybrid model to account for language structure. The convolutional layer accounts for spatial structure (adjacent word groups or phrases) and LSTM accounts for sequential structure of language (sentiment distributed and modified across non-adjacent phrases). Second, they address the problem of missing attributes in text in constructing attribute sentiment scores—as reviewers write only about a subset of attributes and remain silent on others. They develop a model-based imputation strategy using a structural model of heterogeneous rating behavior. Using Yelp restaurant review data, they show superior attribute sentiment scoring accuracy with their model. They find three reviewer segments with different motivations: status seeking, altruism/want voice, and need to vent/praise. Reviewers write to inform and vent/praise, but not based on attribute importance. The heterogeneous model-based imputation performs better than other common imputations; and importantly leads to managerially significant corrections in restaurant attribute ratings. More broadly, our results suggest that social science research should pay more attention to reduce measurement error in variables constructed from text.

Constrained LSTM and Residual Attention for Image Captioning

Visual structure and syntactic structure are essential in images and texts, respectively. Visual structure depicts both entities in an image and their interactions, whereas syntactic structure in texts can reflect the part-of-speech constraints between adjacent words. Most existing methods either use visual global representation to guide the language model or generate captions without considering the relationships of different entities or adjacent words. Thus, their language models lack relevance in both visual and syntactic structure. To solve this problem, we propose a model that aligns the language model to certain visual structure and also constrains it with a specific part-of-speech template. In addition, most methods exploit the latent relationship between words in a sentence and pre-extracted visual regions in an image yet ignore the effects of unextracted regions on predicted words. We develop a residual attention mechanism to simultaneously focus on the pre-extracted visual objects and unextracted regions in an image. Residual attention is capable of capturing precise regions of an image corresponding to the predicted words considering both the effects of visual objects and unextracted regions. The effectiveness of our entire framework and each proposed module are verified on two classical datasets: MSCOCO and Flickr30k. Our framework is on par with or even better than the state-of-the-art methods and achieves superior performance on COCO captioning Leaderboard.

The Role of Attention in Word Recognition: Results from OB1-Reader.

When reading, orthographic information is extracted not only from the word the reader is looking at, but also from adjacent words in the parafovea. Here we examined, using the recently introduced OB1‐reader computational model, how orthographic information can be processed in parallel across multiple words and how orthographic information can be integrated across time and space. Although OB1‐reader is a model of text reading, here we used it to simulate single‐word recognition experiments in which parallel processing has been shown to play a role by manipulating the surrounding context in flanker and priming paradigms. In flanker paradigms, observers recognize a central word flanked by other letter strings located left and right of the target and separated from the target by a space. The model successfully accounts for the finding that such flankers can aid word recognition when they contain bigrams of the target word, independent of where those flankers are in the visual field. In priming experiments, in which the target word is preceded by a masked prime, the model accounts for the finding that priming occurs independent of whether the prime and target word are in the same location or not. Crucial to these successes is the key role that spatial attention plays within OB1‐reader, as it allows the model to receive visual input from multiple locations in parallel, while limiting the kinds of errors that can potentially occur under such spatial pooling of orthographic information.

Assistant diagnosis with Chinese electronic medical records based on CNN and BiLSTM with phrase-level and word-level attentions

BackgroundInferring diseases related to the patient’s electronic medical records (EMRs) is of great significance for assisting doctor diagnosis. Several recent prediction methods have shown that deep learning-based methods can learn the deep and complex information contained in EMRs. However, they do not consider the discriminative contributions of different phrases and words. Moreover, local information and context information of EMRs should be deeply integrated.ResultsA new method based on the fusion of a convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) with attention mechanisms is proposed for predicting a disease related to a given EMR, and it is referred to as FCNBLA. FCNBLA deeply integrates local information, context information of the word sequence and more informative phrases and words. A novel framework based on deep learning is developed to learn the local representation, the context representation and the combination representation. The left side of the framework is constructed based on CNN to learn the local representation of adjacent words. The right side of the framework based on BiLSTM focuses on learning the context representation of the word sequence. Not all phrases and words contribute equally to the representation of an EMR meaning. Therefore, we establish the attention mechanisms at the phrase level and word level, and the middle module of the framework learns the combination representation of the enhanced phrases and words. The macro average f-score and accuracy of FCNBLA achieved 91.29 and 92.78%, respectively.ConclusionThe experimental results indicate that FCNBLA yields superior performance compared with several state-of-the-art methods. The attention mechanisms and combination representations are also confirmed to be helpful for improving FCNBLA’s prediction performance. Our method is helpful for assisting doctors in diagnosing diseases in patients.

Using virtual reality to study reading: An eye-tracking investigation of transposed-word effects

Abstract Eye movements were recorded using a virtual reality (VR) set-up as participants made grammatical decisions to sequences of words. Ungrammatical decisions were harder to make to transposed-word sequences (The white was cat big) compared with control sequences where transposing two adjacent words never produces a correct sentence (The black ran dog fat). Crucially, we found significant transposed-word effects independently of the order in which the two critical words were fixated (i.e., was before cat vs. cat before was), thus falsifying a reading out-of-order account of transposed-word effects.