Low-frequency Words Research Articles

Fixation-related NIRS indexes retinotopic occipital processing of parafoveal preview during natural reading

While word frequency and predictability effects have been examined extensively, any evidence on interactive effects as well as parafoveal influences during whole sentence reading remains inconsistent and elusive. Novel neuroimaging methods utilize eye movement data to account for the hemodynamic responses of very short events such as fixations during natural reading. In this study, we used the rapid sampling frequency of near-infrared spectroscopy (NIRS) to investigate neural responses in the occipital and orbitofrontal cortex to word frequency and predictability. We observed increased activation in the right ventral occipital cortex when the fixated word N was of low frequency, which we attribute to an enhanced cost during saccade planning. Importantly, unpredictable (in contrast to predictable) low frequency words increased the activity in the left dorsal occipital cortex at the fixation of the preceding word N-1, presumably due to an upcoming breach of top-down modulated expectation. Opposite to studies that utilized a serial presentation of words (e.g. Hofmann et al., 2014), we did not find such an interaction in the orbitofrontal cortex, implying that top-down timing of cognitive subprocesses is not required during natural reading. We discuss the implications of an interactive parafoveal-on-foveal effect for current models of eye movements.

Multi-Channel Reverse Dictionary Model

A reverse dictionary takes the description of a target word as input and outputs the target word together with other words that match the description. Existing reverse dictionary methods cannot deal with highly variable input queries and low-frequency target words successfully. Inspired by the description-to-word inference process of humans, we propose the multi-channel reverse dictionary model, which can mitigate the two problems simultaneously. Our model comprises a sentence encoder and multiple predictors. The predictors are expected to identify different characteristics of the target word from the input query. We evaluate our model on English and Chinese datasets including both dictionary definitions and human-written descriptions. Experimental results show that our model achieves the state-of-the-art performance, and even outperforms the most popular commercial reverse dictionary system on the human-written description dataset. We also conduct quantitative analyses and a case study to demonstrate the effectiveness and robustness of our model. All the code and data of this work can be obtained on https://github.com/thunlp/MultiRD.

The Differential Effects of Subtitles on the Comprehension of Native English Connected Speech Varying in Types and Word Familiarity

Connected speech produced by native speakers poses a challenge to second language learners. Video subtitles have been found to assist the decoding of English connected speech for learners of English as a foreign language (EFL). However, the presence of subtitles may divert the listeners’ attention to the visual cues while paying less attention to the speech signals. To test this proposal, we employed a bi-modal audio-visual listening test and examined whether EFL listeners were able to correctly identify the connected speech when misleading subtitles were present. We further tested whether connected speech with words of lower frequency further reduced the accuracy rate. Twenty-eight adolescent EFL learners, all with more than 10 years of experiences in learning English in schools, were tested with three major types of connected speech phonological processes, namely assimilation, elision, and juncture. The results of statistical analyses showed that matched and mismatched subtitles facilitated the comprehension of both familiar and unfamiliar connected speech. Error analyses revealed the degree of item-specific variations across the three types of connected speech processes as well as across the three subtitling conditions. This research provides insights on the immediate and long-term impact of subtitles on the decoding of English connected speech.

Linear and Nonlinear Age Trajectories of Language Use: A Laboratory Observation Study of Couples' Conflict Conversations.

This study investigated linear and nonlinear age effects on language use with speech samples that were representative of naturally occurring conversations. Using a corpus-based approach, we examined couples' conflict conversations in the laboratory. The conversations, from a total of 364 community-dwelling German-speaking heterosexual couples (aged 19-82), were videotaped and transcribed. We examined usage of lower-frequency words, grammatical complexity, and utterance of filled pauses (e.g., äh ["um"]). Multilevel models showed that age effects on the usage of lower-frequency words were nonsignificant. Grammatical complexity increased until middle age (i.e., 54) and then declined. The utterance of filled pauses increased until old age (i.e., 70) and then decreased. Results are discussed in relation to cognitive aging research.

Assessing Physiological Stress Responses to Word Retrieval in Individuals with Aphasia: A Preliminary Study

Objective: Word retrieval difficulty can be distressing for individuals with aphasia. However, physiological stress responses to word retrieval are relatively unstudied. This study aims to measure the physiological response (i.e., heart rate and respiratory rate) and perceived stress in individuals with aphasia during a naming task to understand whether word retrieval may be a stressor to them. Method: An aphasia group (n = 7) and a healthy group (n = 38) participated in 4 experimental conditions, including rest, counting, and high-frequency and low-frequency word conditions, while their physiological parameters were measured throughout the sessions. Stress ratings were also obtained for each condition. Results: Significant differences in heart rate were found while participants were engaged in speech and/or language tasks, compared to the rest condition, for both groups. Significant differences in respiratory rate were found between the rest and low-frequency word conditions in the aphasia group. No significant difference was found in heart rate between 2 groups, but a difference in respiratory rate was found between 2 groups in the low-frequency word condition. There were some discrepancies between stress level and physiological parameters. Conclusions: Individuals with aphasia perceived greater stress in the naming tasks, however, the perceived stress was not necessarily parallel to the physiological parameters.

A hybrid representation-based simile component extraction

Simile, a special type of metaphor, can help people to express their ideas more clearly. Simile component extraction is to extract tenors and vehicles from sentences. This task has a realistic significance since it is useful for building cognitive knowledge base. With the development of deep neural networks, researchers begin to apply neural models to component extraction. Simile components should be in cross-domain. According to our observations, words in cross-domain always have different concepts. Thus, concept is important when identifying whether two words are simile components or not. However, existing models do not integrate concept into their models. It is difficult for these models to identify the concept of a word. What’s more, corpus about simile component extraction is limited. There are a number of rare words or unseen words, and the representations of these words are always not proper enough. Exiting models can hardly extract simile components accurately when there are low-frequency words in sentences. To solve these problems, we propose a hybrid representation-based component extraction (HRCE) model. Each word in HRCE is represented in three different levels: word level, concept level and character level. Concept representations (representations in concept level) can help HRCE to identify the words in cross-domain more accurately. Moreover, with the help of character representations (representations in character levels), HRCE can represent the meaning of a word more properly since words are consisted of characters and these characters can partly represent the meaning of words. We conduct experiments to compare the performance between HRCE and existing models. The experiment results show that HRCE significantly outperforms current models.

A multi-label text classification method via dynamic semantic representation model and deep neural network

The increment of new words and text categories requires more accurate and robust classification methods. In this paper, we propose a novel multi-label text classification method that combines dynamic semantic representation model and deep neural network (DSRM-DNN). DSRM-DNN first utilizes word embedding model and clustering algorithm to select semantic words. Then the selected words are designated as the elements of DSRM-DNN and quantified by the weighted combination of word attributes. Finally, we construct a text classifier by combining deep belief network and back-propagation neural network. During the classification process, the low-frequency words and new words are re-expressed by the existing semantic words under sparse constraint. We evaluate the performance of DSRM-DNN on RCV1-v2, Reuters-21578, EUR-Lex, and Bookmarks. Experimental results show that our method outperforms the state-of-the-art methods.

Lexical richness of newspaper editorials published in Southeast Asian countries

This paper investigates the lexical richness of newspaper editorials written by the writers from ESL (English as a Second Language) and EFL (English as a Foreign Language) of Southeast Asian countries. Using editorial texts published on the same day in two major online newspapers from Malaysia and the Philippines as representative of ESL countries, and two major online newspapers published in Indonesia and Thailand that represent EFL countries, this paper compares the production of Type Token Ratio (TTR) as a measurement of the lexical richness. This study displays a profile of lexical richness gained by submitting the texts into a vocabulary profiler program namely Lexical Frequency Profile (LFP) proposed by Laufer and Nation (1995) to highlight the emergence of the high-frequency word list (K1 and K2 words) and low-frequency word list (AWL and Off-list words). In general, the results show that in all terms of word lists, ESL texts have more varied vocabulary than EFL texts as indicated by the TTR scores (ESL: 0.51; EFL: 0.49). Although the gap of the TTR scores between ESL and EFL texts is slightly insignificant, a bigger TTR score indicates a high lexical richness, while a smaller TTR score shows a low lexical richness. The higher score of TTR in ESL texts could be understood since English plays an important role in education, governance policy and popular culture in those countries (i.e. Malaysia and the Philippines), meanwhile, in Indonesia and Thailand, it plays a lesser role.

영어 어말 설단폐쇄음의 탈락 양상: 고빈도 단어를 중심으로

Word frequency exerts a strong influence on coronal stop deletion. In general, high frequency words undergo word-final coronal stop deletion at a higher rate than low frequency words. This deletion process is undergone in consonant cluster simplification for economy of articulation. This study aims to investigate aspects of English word-final coronal stop deletion across a word boundary in high frequency words. Data were extracted from the Buckeye Corpus of Spontaneous American English Speech depending on morphological, phonological, phonetic and sociolinguistic variations. The results showed that different phonological environments favored one type of coronal stop variations such as [t], flapping, glottaling and deletion. Deletion was most prevalent for coronal stops in C/t,d/#C environment while coronal stops frequently underwent flapping in V/t,d/#V environment. In C/t,d/#C environment adjacent identical features were prohibited (Obligatory Contour Principle). More specifically, when the preceding consonants were more similar to coronal stops, the rate of coronal stop deletion was higher. The following consonants also influenced the coronal stop deletion. The rate of deletion was higher before sonorants than before obstruents. Therefore, actual education of coronal stop deletion should be done depending on various environments and factors.

Holistic word processing is correlated with efficiency in visual word recognition.

Holistic processing of visual words (i.e., obligatory encoding of/attending to all letters of a word) could be a marker of expert word recognition. In the present study, we thus examined for the first time whether there is a direct relation between the word-composite effect (i.e., all parts of a visual word are fully processed when observers perform a task on a word part) and fast access to the orthographic lexicon by visual word experts (i.e., fluent adult readers). We adopted an individual differences approach and used the word-frequency effect (i.e., faster recognition of high- than low-frequency words) in an independent lexical decision task as a proxy of fast access to lexical orthographic representations. Fluent readers with larger word-composite effect showed smaller word-frequency effect. This correlation was mainly driven by an association between a larger composite effect and faster lexical decision on low-frequency words, probably because these lexical representations are less stable and integrated/unitized, hence allowing differentiating among fluent readers. We thus showed that holistic processing of visual words is indeed related to higher efficiency in visual word recognition by skilled readers.

Greining á orðanotkun í lesskilningsog náttúruvísindahlutum PISA 2018: Samanburður á íslensku þýðingunni og enska textanum

Markmið þessarar rannsóknar var að kanna samræmi í orðtíðni í íslenskum og enskum textum á lesskilnings- og náttúruvísindahluta PISA-prófanna 2018. Ef þýðing texta í alþjóðlegu prófi er þyngri eða léttari en sami texti á upprunalega málinu getur það haft áhrif á skilning og skekkt samanburð milli tungumála.Greindir voru tveir textar úr lesskilningshluta PISA 2018 og tveir úr náttúruvísindahlutanum. Notaður var orðtíðnilisti Íslenskrar risamálheildar og enskur orðtíðnilisti sem byggist á tveimur málheildum og er aðgengilegur í gegnum hugbúnaðinn VocabProfile. Orðin voru flokkuð eftir tíðni í fimm flokka. Ef munur var á tíðniflokki orða á íslensku og ensku var kannað hvort til væri samheiti fyrir íslenska orðið í sama tíðniflokki og það enska og lengd samheita borin saman.Niðurstöður benda til að hlutfall algengustu orða sé lægra í textum íslensku þýðingarinnar en í ensku frumtextunum og að hlutfall orða í flokki sjaldgæfustu orðanna sé umtalsvert hærra í íslensku textunum en þeim ensku. Þá virðist dreifing orða á milli orðtíðniflokka vera jafnari í ensku en íslensku þýðingunni. Fram komu vísbendingar um ákveðið ósamræmi og ójafnvægi sem fólst í að tveir þriðju hlutar þeirra íslensku orða, sem féllu í annan tíðniflokk en ensku orðin, voru sjaldgæfari en samsvarandi ensk orð. Í ljós kom að fækka hefði mátt orðum í ólíkum orðtíðniflokkum með því að nota íslenskt samheiti í sama orðtíðniflokki og enska orðið og að draga hefði mátt enn frekar úr ósamræminu með því að velja samheiti úr nærliggjandi tíðniflokki. Hlutfall íslenskra samheita sem voru algengari og lengri var yfir 30% í textunum fjórum. Niðurstöðurnar gefa tilefni til að endurskoða þurfi leiðbeiningar OECD og beina því til þýðenda að þeir taki mið af orðtíðnilistum við val á orðum.

An association-constrained LDA model for joint extraction of product aspects and opinions

The Latent Dirichlet Allocation (LDA) model, which is a document-level probabilistic model, has been widely used in topic modeling. However, an essential issue of the LDA is its shortage in identifying co-occurrence relationships (e.g., aspect-aspect, aspect-opinion, etc.) in sentences. To address the problem, we propose an association constrained LDA (AC-LDA) for effectively capturing the co-occurrence relationships. Specifically, based on the basic features of the syntactic structure in product reviews, we formalize three major types of word association combinations and then carefully design corresponding identifications. For reducing the influence of global aspect words on the local distribution, we apply an important constraint on global aspects. Finally, the constraint and related association combinations are merged into the LDA to guide the topic-words allocation in the learning process. Based on the experiments on real-world product review data, we demonstrate that our model can effectively capture the relationships hidden in local sentences and further increase the extraction rate of fine-grained aspects and opinion words. Our results confirm the superiority of the AC-LDA over the state-of-the-art methods in terms of the extraction accuracy. We also verify the strength of our method in identifying irregularly appeared terms, such as non-aspect opinions, low-frequency words, and secondary aspects.

The effect of contextual plausibility on word skipping during reading

Recent eye-movement evidence suggests readers are more likely to skip a high-frequency word than a low-frequency word independently of the semantic or syntactic acceptability of the word in the sentence. This has been interpreted as strong support for a serial processing mechanism in which the decision to skip a word is based on the completion of a preliminary stage of lexical processing prior to any assessment of contextual fit. The present large-scale study was designed to reconcile these findings with the plausibility preview effect: higher skipping and reduced first-pass reading times for words that are previewed by contextually plausible, compared to implausible, sentence continuations that are unrelated to the target word. Participants' eye movements were recorded as they read sentences containing a short (3–4 letters) or long (6 letters) target word. The boundary paradigm was used to present parafoveal previews which were either higher or lower frequency than the target, and either plausible or implausible in the sentence context. The results revealed strong, independent effects of all three factors on target skipping and early measures of target fixation duration, while frequency and plausibility interacted on later measures of target fixation duration. Simulations using the E-Z Reader model of eye-movement control in reading demonstrated that plausibility effects on skipping are potentially consistent with the assumption that higher-level contextual information only affects post-lexical integration processes. However, no current model of eye movements in reading provides an explicit account of the information or processes that allow readers to rapidly detect an integration failure.

Are Some Words Worth More than Others?

Current evaluation metrics for language modeling and generation rely heavily on the accuracy of predicted (or generated) words as compared to a reference ground truth. While important, token-level accuracy only captures one aspect of a language model's behavior, and ignores linguistic properties of words that may allow some mis-predicted tokens to be useful in practice. Furthermore, statistics directly tied to prediction accuracy (including perplexity) may be confounded by the Zipfian nature of written language, as the majority of the prediction attempts will occur with frequently-occurring types. A model's performance may vary greatly between high- and low-frequency words, which in practice could lead to failure modes such as repetitive and dull generated text being produced by a downstream consumer of a language model. To address this, we propose two new intrinsic evaluation measures within the framework of a simple word prediction task that are designed to give a more holistic picture of a language model's performance. We evaluate several commonly-used large English language models using our proposed metrics, and demonstrate that our approach reveals functional differences in performance between the models that are obscured by more traditional metrics.

Improving Chinese Word Representation with Conceptual Semantics

The meaning of a word includes a conceptual meaning and a distributive meaning. Word embedding based on distribution suffers from insufficient conceptual semantic representation caused by data sparsity, especially for low-frequency words. In knowledge bases, manually annotated semantic knowledge is stable and the essential attributes of words are accurately denoted. In this paper, we propose a Conceptual Semantics Enhanced Word Representation (CEWR) model, computing the synset embedding and hypernym embedding of Chinese words based on the Tongyici Cilin thesaurus, and aggregating it with distributed word representation to have both distributed information and the conceptual meaning encoded in the representation of words. We evaluate the CEWR model on two tasks: word similarity computation and short text classification. The Spearman correlation between model results and human judgement are improved to 64.71%, 81.84%, and 85.16% on Wordsim297, MC30, and RG65, respectively. Moreover, CEWR improves the F1 score by 3% in the short text classification task. The experimental results show that CEWR can represent words in a more informative approach than distributed word embedding. This proves that conceptual semantics, especially hypernymous information, is a good complement to distributed word representation.

Word frequency effects on judgments of learning: More than just beliefs

Judgments of learning (JOLs) are usually higher for high-frequency words than for low-frequency words, which has been attributed to beliefs about how word frequency affects memory. The main goal of the present study was to explore if identifying word frequency as a relevant cue is necessary for it to affect JOLs. The idea is that for one to base judgments in beliefs of how a variable affects memory, one must first consider that variable. In Experiments 1 and 2, participants studied a list of high- and low-frequency words, made immediate JOLs, and answered questions aimed at identifying the cues used to make those JOLs. The results showed that identifying word frequency as a cue was not necessary for effects on JOLs to occur, suggesting that some participants could not have used beliefs about how word frequency affects memory when making JOLs. In Experiment 3, we measured processing fluency of high- and low-frequency words through a lexical decision task. Participants identified high-frequency words quicker than low-frequency words, suggesting the former to be more fluently processed. In Experiment 4, we explored if response times in a lexical decision task mediated the effect of word frequency on JOLs. Results showed a significant mediation of 8–13%, depending on the analysis technique. We argue that theory-driven processes do not fully account for word frequency effects on JOLs.

Word Frequency Is Associated With Cognitive Effort During Verbal Working Memory: A Functional Near Infrared Spectroscopy (fNIRS) Study.

PurposePsycholinguistic models traditionally view verbal working memory capacity as independent from linguistic features; connectionist models suggest otherwise. Moreover, lexical processing studies show high frequency words differ in cognitive effort from low frequency words, although these effects during concurrent processing of words in working memory are unknown. This novel study examines potential differences in cognitive effort, as measured by differences in HbO2 and Hb, for high frequency versus low frequency words during a working memory paradigm.MethodsA total of 21 neurologically typical participants (age 18–23) completed an auditory, n-back, working memory task comparing performance with high- as compared to low- frequency words. Hemodynamic changes in the prefrontal cortex were recorded with a continuous-wave functional near-infrared spectroscopy (fNIRS) device. Behavioral data (accuracy, reaction time) were recorded using E-prime.ResultsDifferences in word frequency were evident at both behavioral and neurological levels. Participants were more accurate, albeit slower in identifying the target two back in a sequence for low- as compared to high-frequency words. Patterns of hemodynamic changes were also significantly different between HF and LF conditions.ConclusionThe results from this study indicate that the behavioral and neurological signatures inherent in holding high- versus low-frequency words in working memory differs significantly. Specifically, the findings from this study indicated that words differing in frequency place different demands on cognitive processing load in memory updating tasks.

Converging semantic and phonological information in lexical retrieval and selection in young and older adults.

The present experiments investigated the influence of combined phonological and semantic information on lexical retrieval, metacognitive retrieval states, and selection in an immediate multiple-choice task. Younger and older adults attempted to retrieve words (e.g., abdicate) from low-frequency word definitions. Retrieval was preceded by primes that were "both" semantically and phonologically related (e.g., abandon), phonologically related (e.g., abdomen), semantically related (e.g., resign), or unrelated (e.g., pink). Younger and older adults benefited from phonological primes in retrieval, and also showed reduced, but reliable, facilitation from "both" primes. Younger and older adults also indicated that they were likely to "know" the answer more often after any related primes compared with unrelated primes. Because there was no facilitation in actual retrieval after semantic primes, this reflects a false "knowing" response. After each retrieval attempt, participants were given the correct answer along with the 4 primes in a multiple-choice test. Both younger and older adults were likely to false alarm to the "both" and semantic alternatives. When instructed that the prime was not the answer, younger adults decreased their false alarms, but not the older adults. With masked, briefly presented primes, younger adults mimicked the false alarms shown by older adults, suggesting that the high false alarm rates in older adults reflect an inability to discriminate the source of activation. The present experiments provide strong evidence for age-invariant phonological facilitation, and also suggest that overlapping semantic information moderates the facilitatory effect of phonological information on retrieval, and also produces age-related differences on an immediate multiple-choice task. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Referent's Lexical Frequency Predicts Mismatch Negativity Responses to New Words Following Semantic Training.

Lexical ERPs (event-related potentials) obtained in an oddball paradigm were suggested to be an index of the formation of new word representations in the brain in the learning process: with increased exposure to new lexemes, the ERP amplitude grows, which is interpreted as a signature of a new memory-trace build-up and activation. Previous learning studies using this approach have, however, mostly used meaningless novel word forms; it therefore remains uncertain whether the increased amplitude simply reflects increased familiarity with the new stimulus or is indeed a reflection of a complete word representation. Here, we used the oddball paradigm to measure the mismatch negativity (MMN) responses to novel word forms before and after semantic training, during which they were associated with previouslyfamiliar words of either high or low frequency of occurrence. Following training, the amplitude of the MMN to novel words was enhanced. Furthermore, these changes were dependent on the frequency of the reference which novel items became associated with: namely, the MMN amplitude became greater and the latency shorter for the item which was assigned the high-frequency meaning. Even though the amount of training was the same for both types of items, the low-frequency stimulus did not achieve similar significant changes. Our results suggest that the new surface form becomes linked to the existing representation, which then automatically activates in full when the respective stimulus is present at the input. This finding indicates that the learning-related MMN dynamics, manifest as a response increase after learning, likely reflects the formation and activation of a complete lexicosemantic memory circuits for words.

The Clock Counts - Length Effects in English Dyslexic Readers.

In reading, length effects (LEs) are defined as an increment in the time taken to read as a function of word length and may indicate whether reading is proceeding in an efficient whole word fashion or by serial letter processing. LEs are generally considered to be a pathognomonic symptom of developmental dyslexia (DD) and predominantly have been investigated in transparent orthographies where reading impairment is characterized as slow and effortful. In the present study a sample of 18 adult participants with DD were compared to a matched sample of typical developing readers to investigate whether the LE is a critical aspect of DD in an opaque orthography, English. We expected that the DD group would present with marked LEs, in both words and non-words, compared to typical developing readers. The presence of LEs in the DD group confirmed our prediction. These effects were particularly strong in low frequency words and in non-words, as observed in reading speed. These preliminary findings may have important theoretical implications for current understanding of DD.