Text Complexity Research Articles

ReaderBench: Multilevel analysis of Russian text characteristics

This paper introduces an adaptation of the open source ReaderBench framework that now supports Russian multilevel analyses of text characteristics, while integrating both textual complexity indices and state-of-the-art language models, namely Bidirectional Encoder Representations from Transformers (BERT). The evaluation of the proposed processing pipeline was conducted on a dataset containing Russian texts from two language levels for foreign learners (A - Basic user and B - Independent user). Our experiments showed that the ReaderBench complexity indices are statistically significant in differentiating between the two classes of language level, both from: a) a statistical perspective, where a Kruskal-Wallis analysis was performed and features such as the “nmod” dependency tag or the number of nouns at the sentence level proved the be the most predictive; and b) a neural network perspective, where our model combining textual complexity indices and contextualized embeddings obtained an accuracy of 92.36% in a leave one text out cross-validation, outperforming the BERT baseline. ReaderBench can be employed by designers and developers of educational materials to evaluate and rank materials based on their difficulty, as well as by a larger audience for assessing text complexity in different domains, including law, science, or politics.

Word-formation complexity: a learner corpus-based study

This article explores the word-formation dimension of learner text complexity which indicates how skilful the non-native speakers are in using more and less complex - and varied - derivational constructions. In order to analyse the association between complexity and writing accuracy in word formation as well as interactive effects of task type, text register, and native language background, we examine the materials of the REALEC corpus of English essays written by university students with Russian L1. We present an approach to measure derivational complexity based on the classification of suffixes offered in Bauer and Nation (1993) and then compare the complexity results and the number of word formation errors annotated in the texts. Starting with the hypothesis that with increasing complexity the number of errors will decrease, we apply statistical analysis to examine the association between complexity and accuracy. We found, first, that the use of more advanced word-formation suffixes affects the number of errors in texts. Second, different levels of suffixes in the hierarchy affect derivation accuracy in different ways. In particular, the use of irregular derivational models is positively associated with the number of errors. Third, the type of examination task and expected format and register of writing should be taken into consideration. The hypothesis holds true for regular but infrequent advanced suffixal models used in more formal descriptive essays associated with an academic register. However, for less formal texts with lower academic register requirements, the hypothesis needs to be amended.

Computational linguistics and discourse complexology: Paradigms and research methods

The dramatic expansion of modern linguistic research and enhanced accuracy of linguistic analysis have become a reality due to the ability of artificial neural networks not only to learn and adapt, but also carry out automate linguistic analysis, select, modify and compare texts of various types and genres. The purpose of this article and the journal issue as a whole is to present modern areas of research in computational linguistics and linguistic complexology, as well as to define a solid rationale for the new interdisciplinary field, i.e. discourse complexology. The review of trends in computational linguistics focuses on the following aspects of research: applied problems and methods, computational linguistic resources, contribution of theoretical linguistics to computational linguistics, and the use of deep learning neural networks. The special issue also addresses the problem of objective and relative text complexity and its assessment. We focus on the two main approaches to linguistic complexity assessment: “parametric approach” and machine learning. The findings of the studies published in this special issue indicate a major contribution of computational linguistics to discourse complexology, including new algorithms developed to solve discourse complexology problems. The issue outlines the research areas of linguistic complexology and provides a framework to guide its further development including a design of a complexity matrix for texts of various types and genres, refining the list of complexity predictors, validating new complexity criteria, and expanding databases for natural language.

Research on Long Text Classification Model Based on Multi-Feature Weighted Fusion

Text classification in the long-text domain has become a development challenge due to the significant increase in text data, complexity enhancement, and feature extraction of long texts in various domains of the Internet. A long text classification model based on multi-feature weighted fusion is proposed for the problems of contextual semantic relations, long-distance global relations, and multi-sense words in long text classification tasks. The BERT model is used to obtain feature representations containing global semantic and contextual feature information of text, convolutional neural networks to obtain features at different levels and combine attention mechanisms to obtain weighted local features, fuse global contextual features with weighted local features, and obtain classification results by equal-length convolutional pooling. The experimental results show that the proposed model outperforms other models in terms of accuracy, precision, recall, F1 value, etc., under the same data set conditions compared with traditional deep learning classification models, and it can be seen that the model has more obvious advantages in long text classification.

Modelling the lexical complexity of homogenous texts: a time series approach.

Lexical complexity of homogeneous texts, especially when produced by an institutional author over time, exhibits a generally observed increasing trend with local random fluctuations. Such an irreversible entropic process fits very cogently into the dynamical complexity system theory, where the social, economic, and cultural missions such texts set to serve constitute the underlying driving momentum for the texts to adapt themselves from low to high complexity. Structural equations have been shown effective in modeling such macroscopic behavior of the entropic process of the homogeneous texts. The current work formulates the problem from a time series modeling approach applied to a large sociolinguistic corpus in written Chinese. The findings show that such an alternative approach not only produces as valid models with strong goodness of fit as the structural equation approach, but also exhibits, by design, additional benefits in explaining the entropic process of homogeneous texts in the dynamical complexity system framework. Some technical challenges, such as phase change in model calibration, are also solved with less cost using the newly proposed approach. Further directions are pointed out to more fully compare these approaches in the setup of the current study and corpus linguistics in general.

A data-driven global observatory addressing worldwide challenges through text mining and complex data visualisation

Observing the world on a global scale can help us understand better the context of problems that engage us all. In this paper, we propose a data-driven global observatory that puts together the different perspectives of media, science, statistics and sensing over heterogeneous data sources and text mining algorithms. We also discuss the implementation of this global observatory in the context of epidemic intelligence, monitoring the impact of the COVID-19 pandemic, and in the context of climate change, with a specific focus on water resource management. We will also discuss the value of this global solution in local contexts and priorities.

Quantifying French Document Complexity

Measuring a document's complexity level is an open challenge, particularly when one is working on a diverse corpus of documents rather than comparing several documents on a similar topic or working on a language other than English. In this paper, we define a methodology to measure the complexity of French documents, using a new general and diversified corpus of texts, the "French Canadian complexity level corpus", and a wide range of metrics. We compare different learning algorithms to this task and contrast their performances and their observations on which characteristics of the texts are more significant to their complexity. Our results show that our methodology gives a general-purpose measurement of text complexity in French.

Textual Syntactic Complexity and Its Role In Second Language Reading Outcomes In Indonesia

This paper examines the role of syntactic complexity in L2 reading outcomes across different EFL proficiency levels in an Indonesian university. Indonesian university students (N = 148) at Intermediate and Advanced levels of proficiency read four English passages differing in syntactic complexity. The latter was measured by several widely used text modelling tools. Participants read two low and two high complexity texts and completed a post-test comprehension test. Syntactic complexity had a statistically significant but low magnitude effect size, accounting for 2%-5% of the variance of reading performance between the L2 English proficiency levels. There were also noticeable differences in text analysis measures across the different complexity tools. The usefulness of syntactic complexity as an isolated dimension of text complexity is evaluated. The contribution of this study to the field both in theory and practice is presented.

Textual Syntactic Complexity and Its Role In Second Language Reading Outcomes In Indonesia

This paper examines the role of syntactic complexity in L2 reading outcomes across different EFL proficiency levels in an Indonesian university. Indonesian university students (N = 148) at Intermediate and Advanced levels of proficiency read four English passages differing in syntactic complexity. The latter was measured by several widely used text modelling tools. Participants read two low and two high complexity texts and completed a post-test comprehension test. Syntactic complexity had a statistically significant but low magnitude effect size, accounting for 2%-5% of the variance of reading performance between the L2 English proficiency levels. There were also noticeable differences in text analysis measures across the different complexity tools. The usefulness of syntactic complexity as an isolated dimension of text complexity is evaluated. The contribution of this study to the field both in theory and practice is presented.

Impact of the Text Simplification on Understanding.

The reduction of the linguistic complexity of medical texts to make them more understandable to a larger population is an important task. The simplification of texts involves several steps, among which our study focuses on the definition of complex constructions and on study of the impact of the simplification. For this study, we selected 20 texts from the medical domain on different topics, namely drugs, diseases, substances, and medical institutions. We identified complex linguistic constructions and carried out their manual simplification at syntactic, lexical and semantic levels. We then designed a questionnaire to test comprehension of the texts and conducted a study with 26 participants. The results of this study shows that simplified texts obtained higher number of correct answers than technical texts. This difference is statistically significant. The self-evaluation questionnaire, done at the beginning of the test, indicates that the participants tend to overestimate their understanding of medical information. Besides, there is no correlation between the time taken to complete the interview and the correct answers provided.

Prewriting in Voice Chat: Exploring the Effects of Collaborative Prewriting on EFL Learners’ Performance

Addressing the lack of studies on L2 online and collaborative prewriting, this study examines L2 collaborative prewriting through voice chat. It explores the effects of an online prewriting condition (individual vs collaborative) on performance in subsequent individual writing as assessed by text length, accuracy, complexity, and analytical rating scales of content, organization, and language. One hundred and twenty-six Thai university students were divided into a control (individual prewriting) group and an experimental (collaborative prewriting) group. The utilized prewriting task was structured, focusing on ideas generation, selection, and organization, and the main task was problem-solution paragraph writing. Independent-sample t-tests revealed that accuracy was comparable across groups; however, the experimental group significantly outperformed the control group on text length, some lexical and syntactic complexity measures, and all rating scores, and the effect size concerning text length was large. Overall, these findings suggest potential benefits of collaboration in voice-chat prewriting.

Text Complexity Analysis of Chinese and foreign academic English writing via mobile devices based on neural network and deep learning

Text Complexity Analysis of Chinese and foreign academic English writing via mobile devices based on neural network and deep learning

Linguistic Indicators for Text Complexity in Picture Books for Young Chinese Children Learning English as a Foreign Language

We examined the linguistic features of texts in twenty-nine picture books used in an early English as a Foreign Language program in China. We used the software CLAN to automatically extract indices of linguistic complexity that are typically used to analyze child-directed speech and tested if these indices aligned with expert judgments on the books’ appropriate grade level (Kindergarten-1 through Kindergarten-3). Of the eleven characteristics investigated, seven showed significant between-level differences with moderate effect sizes. Across all levels, vocabulary complexity (i.e., frequency of types, frequency of tokens, and vocabulary diversity) and syntactic complexity (i.e., number of verbs per utterance, number of Developmental-Sentence-Scoring-eligible utterances, mean length of utterance in morphemes, and total number of non-zero morphemes) increased, also in alignment with experts’ judgments. Indices of child language development can thus be used to estimate text complexity in picture books. The study contributes to a better understanding of children’s picture book difficulty and has methodological implications for investigating text characteristics for very young children learning English as a foreign language.

Improving access to COVID-19 information by ensuring the readability of government websites.

Issue addressedThis study evaluated the readability of web pages from two public‐facing Victorian government websites that were responsible for communicating key health messages relating to the COVID‐19 pandemic in 2020.MethodsWebpages were downloaded and filtered to identify relevant materials (English language materials containing HTML files that referred to COVID‐19). The files were converted to text files and two Python packages, SpaCy and TextStat were used to obtain the data presented here. In addition to running two well‐established readability tests, SMOG Index (Simple Measure of Gobbledygook) and Flesch Reading Ease formula, we also calculated the figures for sentence length and word length, which drive the readability measures and allow a disaggregated view of the data. Type token ratio measures were conducted as a reflection of the breadth of vocabulary used in the web pages.ResultsDerived measures of text complexity were higher than recommended levels of text complexity for health promotion materials, which are generally set at senior primary school levels. This did not vary depending on the intended audience (public or professional). A senior secondary reading level was required for effective engagement with the text published on both sites.ConclusionsImproving the readability of materials on key government websites where information about COVID‐19 is being communicated to the public, represents a low cost and potentially effective means of improving public understanding of the pandemic and the steps individuals need to take to protect themselves and the community.So what?Given the challenges widely identified in ensuring compliance with protective behaviours, confidence in seeking vaccination and increasing distrust of government, it would be strategic to improve public communication to ensure health messages are simple and readily understood.SummaryThe complexity and readability of text contained in web pages during 2020 from two Victorian government departments were evaluated. Communication regarding the restrictions and the management of risks associated with COVID‐19 was the main focus of these 367 individual web pages from the Department of Health and Human Services (DHHS) and the Department of Education and Training (DET). Results indicated that across both sites and on both readability measures used, an education level equivalent to senior secondary school would be required to readily understand the contents.

The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study

Abstract This study explores the interaction effect between source text (ST) complexity and machine translation (MT) quality on the task difficulty of neural machine translation (NMT) post-editing from English to Chinese. When investigating human effort exerted in post-editing, existing studies have seldom taken both ST complexity and MT quality levels into account, and have mainly focused on MT systems used before the emergence of NMT. Drawing on process and product data of post-editing from 60 trainee translators, this study adopted a multi-method approach to measure post-editing task difficulty, including eye-tracking, keystroke logging, quality evaluation, subjective rating, and retrospective written protocols. The results show that: 1) ST complexity and MT quality present a significant interaction effect on task difficulty of NMT post-editing; 2) ST complexity level has a positive impact on post-editing low-quality NMT (i.e., post-editing task becomes less difficult when ST complexity decreases); while for post-editing high-quality NMT, it has a positive impact only on the subjective ratings received from participants; and 3) NMT quality has a negative impact on its post-editing task difficulty (i.e., the post-editing task becomes less difficult when MT quality goes higher), and this impact becomes stronger when ST complexity increases. This paper concludes that both ST complexity and MT quality should be considered when testing post-editing difficulty, designing tasks for post-editor training, and setting fair post-editing pricing schemes.

What does research say about the texts we use in elementary school?

Much of the attention given to literacy, of late, has focused on ensuring that students can read, without consideration to what they are given to read. Kristin Conradi Smith & Elfrieda H. Hiebert discuss four general aspects of the texts used in elementary classrooms – text complexity, text type (narrative versus informational), text format (paper versus screen), and diversity of people presented in the text. They review recent research on each of these text characteristics and then make suggestions for how to apply the research in the classroom.

Reading Impairment in Adolescents with Fetal Alcohol Spectrum Disorders

ABSTRACT Purpose To date, research on effects of prenatal alcohol exposure (PAE) has focused on a broad range of cognitive impairments, but relatively few studies have examined effects of PAE on development of reading skills. Although PAE has been linked to poorer reading comprehension, it remains unclear whether this impairment is attributable to deficits in phonological processing, word reading, oral language skills, and/or executive functioning. Methods A comprehensive reading battery was administered to 10 adolescents with fetal alcohol syndrome (FAS); 16 with partial FAS; 30 nonsyndromal heavily exposed; 49 controls. Results PAE was related to poorer reading comprehension but not to single-word reading or phonological processing, suggesting that the mechanics of reading are intact in adolescents with fetal alcohol spectrum disorders at this age. PAE-related impairment in reading comprehension was mediated, in part, by deficits in mastery of oral language skills, including vocabulary, language structure, and verbal fluency. Conclusions These results are consistent with research showing that reading comprehension in adolescence relies increasingly on linguistic comprehension abilities, especially once word reading becomes automatic and text complexity increases. Our findings suggest that reading-impaired adolescents with PAE will benefit from intervention programs targeting vocabulary knowledge, language structure, verbal fluency, and reading comprehension skills.

Syntactic Complexity in Journal Research Article Abstracts Written in English

The reader’s ability to connect new information to existing knowledge is crucial when reading a text. Nonetheless, text complexity, in many ways, is more linguistic than cognitive. It encompasses the degree of sophistication, and how challenging a reading section is. Depending on the section, such difficulty may appear on the vocabulary level, in the organizational structure, or with coherence, and cohesion. In the globalized world of scientific communication, research articles published in non-Anglophone academic journals require English abstracts to access an international database and citation possibilities. This paper describes the syntactic complexity in journal research article abstracts. A corpus of abstracts written in English and published in Anglophone and non-Anglophone contexts were sampled. The English sub-corpora underwent software-based text analysis using fourteen syntactic complexity measures with the second language (L2) Syntactic Complexity Analyzer (Lu, 2010). Significant differences appeared in only four of the fourteen syntactic indices between texts in Anglophone and non-Anglophone journals, and out of these fourteen measures, non-native groups reported thirteen lower mean values. The study affords insights for L2 writing research to produce accurate texts in content and structure. Ideally, findings will uncover pedagogical implications and applications for academic writing instructions.

Readability as a measure of textual complexity: determinants and evidence in Brazilian companies

ABSTRACT The aim of this article was to evaluate the effect of company earnings and of harmonization with IFRS on the readability of Management Reports in the Brazilian stock market. There is a gap to be filled both in the elaboration and adaptation of readability measures to the context studied, as the studies tend to replicate the original formulas, and in identifying the determinants of the readability of Brazilian company reports, as the research in this field remains in its infancy and the results are inconclusive. The results provide indications for investors to identify complex textual information and may help public policymakers to establish a simple writing manual, along the lines of the SEC’s 1998 Plain English Handbook. The modified metrics and the one developed overcome the criticisms regarding the use of readability formulas in accounting research and could be used in substitution of the original metrics in future studies. An econometric model was used that presents the determinants of readability. Readability was calculated for the Results Analysis section of the Management Report. The resulting construct is understood via three attributes: persistence, current performance, and the reference benchmark. Harmonization with IFRS is a dummy variable, which delimits the pre- and post-IFRS periods. The hypotheses were tested in a sample of Brazilian companies made up of 714 company-year observations covering the period from 2006 to 2019. The descriptive results show that there is an apparent improvement in the readability of the reports in the pre- and post-IFRS period comparison. The econometric evidence shows that, in general, companies with persistent and positive earnings present less complex reports and are more likely to have highly readable reports, because managers publish reports with better readability to signal positive results to the market.

A content spectral-based text representation

In the last few years, text analysis has grown as a keystone in several domains for solving many real-world problems, such as machine translation, spam detection, and question answering, to mention a few. Many of these tasks can be approached by means of machine learning algorithms. Most of these algorithms take as input a transformation of the text in the form of feature vectors containing an abstraction of the content. Most of recent vector representations focus on the semantic component of text, however, we consider that also taking into account the lexical and syntactic components the abstraction of content could be beneficial for learning tasks. In this work, we propose a content spectral-based text representation applicable to machine learning algorithms for text analysis. This representation integrates the spectra from the lexical, syntactic, and semantic components of text producing an abstract image, which can also be treated by both, text and image learning algorithms. These components came from feature vectors of text. For demonstrating the goodness of our proposal, this was tested on text classification and complexity reading score prediction tasks obtaining promising results.