Reading Comprehension Assessment Research Articles

A Study on the Predictors of Written Expression Level of Turkish Learners as a Foreign Language

In the context of teaching Turkish to foreigners, this study aimed to assess whether attitudes towards the target language, grammatical knowledge, speaking skills, reading comprehension, listening comprehension, and prior writing knowledge could predict success in writing in the target language. To investigate this, a relational survey method and a quantitative research design were employed. The study sample comprised 147 B1 level-foreign learners studying Turkish as a foreign language at the Turkish Language Teaching Center (TÖMER) in Ankara during the 2022-2023 academic year. The study utilized a multiple-choice test to assess reading comprehension, an informative listening text titled “Ay’a Yolculuk (Journey to the Moon)” along with a 20-question short-answer test to evaluate listening comprehension, a rating scale for assessing speaking skills, and a cloze test for measuring Turkish grammar knowledge. Moreover, in this study, a short-answer test was employed to assess the participants’prior knowledge of writing. An attitude scale was used to measure students’attitudes towards learning Turkish, while a 6+1 analytical assessment scale was used to evaluate students’ writing skills. When the predictive impact of the chosen independent variables on writing skills was examined, these variables collectively accounted for 66% of the variation in writing achievement. The ranking of the influence levels of these variables on writing achievement (from most to least influential) was as follows: prior knowledge, grammatical knowledge, reading comprehension, attitude towards the target language, speaking achievement, and listening achievement.

Exploring the distribution and cognitive profiles of poor readers across varying levels of reading difficulty: implications for identification and support

BackgroundThis study explored the impact of different cut‐off points used to identify children with reading difficulties on the distribution of these children across the three poor reader groups predicted by the simple view of reading (dyslexia, specific comprehension difficulty and mixed difficulty). Additionally, the study investigated whether the cognitive profiles of these poor reader groups remained consistent across varying levels of reading impairment.MethodsThis study included 209 primary school children from New Zealand, in Years 4–6 and aged 8–10 years, who experienced reading comprehension difficulties. Using a two‐step cluster analysis, participants were assigned to one of three poor reader groups: dyslexia, specific comprehension difficulty (SCD) and mixed difficulty. We examined the distribution of children across these groups at eight levels of reading comprehension difficulty and conducted strengths and weaknesses profiles at four levels of reading ability across 14 tests that assessed a range of reading‐related skills. Additionally, we compared the performance of children in these groups who performed above and below the 10th percentile on a reading comprehension assessment across these tests.ResultsThe results showed that the proportion of children assigned to each group varied across the reading levels, with children at the lower end of the reading comprehension continuum more likely to exhibit the mixed difficulty profile than the SCD and dyslexia profiles. Overall, the research found that dyslexia, SCD and mixed difficulty profiles could be identified at all levels of reading comprehension difficulty, though weaknesses in their reading‐related skills increased as reading comprehension difficulties also increased.ConclusionsOur findings suggest that struggling readers demonstrate distinct cognitive profiles across all levels of reading ability. These results have important implications for identifying and supporting struggling readers, as well as enhancing our understanding of the underlying mechanisms of reading difficulties. The possible relationship between the results obtained in this study and tiered models of reading support is discussed.

Exploring Students' Reading Fluency and Accuracy through the Implementation of ORVT

Reading is an indispensable foundational skill crucial for success in life. However, numerous Indonesian students encounter difficulties with reading comprehension, as highlighted in the PISA National Report. Addressing this challenge necessitates a focus on enhancing both reading fluency and accuracy. The Oral Reading Verification Test (ORVT) is a valuable tool to assess reading ability and guide interventions. This study used quantitative methods, focusing on improving students' reading comprehension through the ORVT. A total of 61 junior high school students from grades VI, VII, and IX participated in the study, with assessment results revealing progress in comprehension and specific error patterns. Pronunciation errors topped the list in grades 7, 8, and 9, with an average percentage of 51.71%. The three classes' achievement on the reading comprehension assessment reached an average of 85.1%, understanding well (complete). The results showed improvement in reading comprehension, highlighting areas of strength and improvement across different aspects of oral reading skills (highly satisfactory). This study underscores the effectiveness of ORVT in improving reading skills and emphasizes the importance of targeted support for reading comprehension.

Improvement Of Reading Comprehension Skills Through the Exelearning.Net Interactive Model in Upper Basic Students

The purpose of this study was to investigate the impact of the interactive model known as "Exelearning.net" on improving reading comprehension skills among students of Basic Superior in Santa Elena, Ecuador. An experimental study with a quantitative approach was conducted, involving a sample of 70 students divided into two groups of 35. Pre and post-tests were administered to assess reading comprehension. Statistical analyses revealed significant improvements in the dimensions of literal, inferential, and critical comprehension, with higher scores in the experimental group than in the control group. The results showed a significance level of < 0.001, indicating that the application of the interactive model ExeLearning.net had a positive impact on strengthening students' reading comprehension skills. These findings suggest that using interactive programs can facilitate the relationship between reading and text comprehension.

Formative Feedback on Student-Authored Summaries in Intelligent Textbooks Using Large Language Models

AbstractAs intelligent textbooks become more ubiquitous in classrooms and educational settings, the need to make them more interactive arises. An alternative is to ask students to generate knowledge in response to textbook content and provide feedback about the produced knowledge. This study develops Natural Language Processing models to automatically provide feedback to students about the quality of summaries written at the end of intelligent textbook sections. The study builds on the work of Botarleanu et al. (2022), who used a Longformer Large Language Model (LLM) to develop a summary grading model. Their model explained around 55% of holistic summary score variance as assigned by human raters. This study uses a principal component analysis to distill summary scores from an analytic rubric into two principal components – content and wording. This study uses two encoder-only classification large language models finetuned from Longformer on the summaries and the source texts using these principal components explained 82% and 70% of the score variance for content and wording, respectively. On a dataset of summaries collected on the crowd-sourcing site Prolific, the content model was shown to be robust although the accuracy of the wording model was reduced compared to the training set. The developed models are freely available on HuggingFace and will allow formative feedback to users of intelligent textbooks to assess reading comprehension through summarization in real time. The models can also be used for other summarization applications in learning systems.

Developing digital academic skills in preservice teachers: performance and metacognitive perspectives

ABSTRACT The purpose of this research was to evaluate an instructional change in an academic literacy course that transitioned from traditional print- to digital-based instruction. The learning outcomes of 71 preservice teachers who attended the digitally-modified course were compared to the learning outcomes of 55 students who attended the same course with print-based instruction. Results showed that the print and digital cohorts developed comparable academic literacy skills, with higher confidence ratings in the digital cohort for both reading and writing performance. Interestingly, the digital cohort was calibrated in their self-evaluation of writing quality, but not in their assessment of reading comprehension. Taken together, this study emphasises the importance of incorporating strategies for deep processing of digital materials and tools to facilitate digital academic writing skills in teacher education. In addition, future teachers should be made aware of the metacognitive biases that accompany digital learning.

Improvement in simultaneous processing through metacognitive instruction

This study presents the improvement generated in simultaneous processing by the application of metacognitive instruction to 3rd grade primary education students. A pre-post-follow-up test design was used. The sample consisted of 30 students from one classroom. Two groups were formed; an intervention group comprising students in the classroom who presented difficulties in simultaneous processing, showing a cognitive weakness that compromises reading comprehension, and a comparison group comprising the rest of the students in the class. The instruments used were the Das-Naglieri Cognitive Assessment System (DN CAS), Assessment of Reading Comprehension (ACL) test and the Reading Awareness Scale (ESCOLA). The groups were equivalent in socioeconomic and cultural status and in successive processing. Reading and study variables were used as differentiators: simultaneous processing and reading awareness. A metacognitive training program based on the Planning Facilitation method was applied to the intervention group. The comparison group followed the usual curricular activities of the classroom and for the education level, according to the traditional teaching model. The program was implemented in 48 sessions over 12 weeks. After the intervention, the group with weakness improved significantly in simultaneous processing and reading awareness; the comparison group did not show any change in their initial scores. The metacognitive instruction that facilitates discussion between students and reflective verbalization about reading tasks promotes improvement in reading awareness and in the simultaneous processing necessary for reading comprehension.

Investigating the Relationship Between the Scores of the Persian Reading Comprehension Assessment Task and the Reading and Dyslexia Test

Background and Objectives: Today, comparing the scores of tests and evaluation scales with each other and the correlation between them is done to check and compare the test’s accuracy in diagnosing reading problems. The present study investigated the correlation between the scores of the Persian reading comprehension assessment task (PRCAT) and the text comprehension subtest of the reading and dyslexia test (NEMA) in fourth and fifth-grade elementary school students. Methods: In this study, 24 students from the fourth and fifth grades of elementary schools in Tehran City, Iran with normal reading skills participated. The set of items for the Persian reading comprehension assessment task and the text comprehension subtest of the NEMA test were taken from the students. Results: The results showed a positive correlation between the scores of the PRCAT and the text comprehension subtest of NEMA, and the correlation coefficient between them is 0.622 (r=0.622, P<0.01). Conclusion: The results showed that the PRCAT has an acceptable validity compared to the text comprehension subtest of the NEMA test.

Improving cognitive-state analysis from eye gaze with synthetic eye-movement data

Eye movements can be used to analyze a viewer’s cognitive capacities or mental state. Neural networks that process the raw eye-tracking signal can outperform methods that operate on scan paths preprocessed into fixations and saccades. However, the scarcity of such data poses a major challenge. We therefore develop SP-EyeGAN, a neural network that generates synthetic raw eye-tracking data. SP-EyeGAN consists of Generative Adversarial Networks; it produces a sequence of gaze angles indistinguishable from human ocular micro- and macro-movements. We explore the use of these synthetic eye movements for pre-training neural networks using contrastive learning. We find that pre-training on synthetic data does not help for biometric identification, while results are inconclusive for the detection of ADHD and gender classification. However, for the eye movement-based assessment of higher-level cognitive skills such general reading comprehension, text comprehension, and the distinction of native from non-native readers, pre-training on synthetic eye-gaze data improves the models’ performance and even advances the state-of-the-art for reading comprehension. The SP-EyeGAN model, pre-trained on GazeBase, along with the code for developing your own raw eye-tracking machine learning model with contrastive learning, is available at https://github.com/aeye-lab/sp-eyegan.

Developing Materials for Assessing Reading Literacy and Comprehension of early Graders in Bulgaria and Italy

The paper presents the bilateral project Assessing reading literacy and comprehension of early graders in Bulgaria and Italy, its aims and expected results, as well as the principles underlying the creation of texts assessing reading literacy in Italian and Bulgarian. The ultimate goal of the project is to improve the literacy skills of primary school children through education. In order to contribute to the achievement of this goal, a thorough investigation focused on the assessment of reading literacy and comprehension among early school children in Bulgaria and Italy. The article focuses on the principles for preparing materials in Italian and Bulgarian for assessing reading literacy. The linguistic features used to predict reading ability are divided into four main groups: raw text, lexical, morpho-syntactic and syntactic features.

Gaining a deeper understanding of the deep cloze reading comprehension test: examining potential contributors and consequences

AbstractThe deep cloze test was developed by Jensen and Elbro (Read Writ Interdiscip J 35(5):1221–1237, 2022. https://doi.org/10.1007/s11145-021-10230-w) to assess reading comprehension at the level of global situational understanding. In two independent studies, we examined potential contributors to students’ scores on the deep cloze reading comprehension test, as well as the predictability of students’ scores on this measure for their course achievement and integrated text understanding measured with an open-ended written comprehension assessment. Results showed that students’ language background, word recognition skills, and working memory resources explained unique portions of the variance in students’ scores on the deep cloze reading comprehension test. Further, scores on this test were positively correlated with students’ course achievement and uniquely predicted their integrated text understanding when language background, working memory, and prior topic knowledge were controlled for. Taken together, our findings support an interpretation of the deep cloze reading comprehension test as an effective and efficient measure of situation level understanding that draws on language skills, word level processes, and working memory resources and also can be used to predict students’ performance on important criterial tasks requiring deeper level understanding.

Automated Think-Aloud Protocol for Identifying Students with Reading Comprehension Impairment Using Sentence Embedding

The think-aloud protocol is a valuable tool for investigating readers’ cognitive processes during reading. However, its reliance on experienced human evaluators poses challenges in terms of efficiency and scalability. To address this limitation, this study proposes a novel application of natural language processing to automate the think-aloud protocol. Specifically, we use a sentence embedding technique to encode the stimulus text and corresponding readers’ responses into high-dimensional vectors, and the similarity between these embeddings serves as a feature. The properties of the feature are investigated for word-frequency-based and contextualized embedding models. Differences in the sentence embedding-based feature between poor comprehenders and normal readers are investigated. Using these features, seven machine learning models were trained to classify readers into normal and abnormal groups. The highest F1 score of 0.74 was achieved with the contextualized embedding and random forest classifier. This highlights the effectiveness of the embedding technique in extracting useful features for automating the think-aloud protocol for assessing reading comprehension abilities. The potential benefits of this automated approach include increased efficiency and scalability, ultimately facilitating the early diagnosis of reading comprehension impairment and individualized interventions.

The Correlation between Reading Strategies and Reading Comprehension

Indonesian students continue encounter difficulties when it comes to reading English academic materials as English is their foreign language. The study aims to explore the reading strategies employed by English as a Foreign Language (EFL) students in order to improve their reading comprehension and to determine whether a significant correlation exists between reading strategies and reading comprehension. The study involved 60 participants from one senior high school in Batam, Indonesia. The study utilized Survey of Reading Strategies (SORS) and reading comprehension test. The Kolmogorov-Smirnov test was performed to determine the normality of the data distribution while Pearson correlation was utilized to investigate the interrelation among the variables. The results showed that students most commonly employed problem-solving strategies when reading and there was a significant correlation between reading strategy and reading comprehension. The findings suggest that students who employ a variety of reading strategies, including problem-solving strategies, support reading strategies, and global reading strategies, perform better on reading comprehension assessments. The study made a significant addition to the field of English language teaching by highlighting the need of raising EFL students’ awareness and teaching them proper reading strategy for comprehending scholarly texts.

Impact of Questioning Strategies on Reading Comprehension of Pakistani O-Level Students

Introduction: This research investigates the impact of questioning strategies on the reading comprehension skills of Pakistani O-Level students. The study employed a true experimental research design, dividing 30 students from FFC Grammar School and College, Goth Machi, Sadiqabad, into Control and Experimental groups based on age and capability. Methodology: Random sampling formed the Control and Experimental groups, with both undergoing pre and post-tests to assess reading comprehension levels. While the Control group received traditional teaching methods, the Experimental group received treatment involving diverse questioning techniques during reading activities. Results/Findings: Statistical analysis revealed significant differences favoring the Experimental group. The implementation of varied questioning techniques led to improved comprehension of complex texts among students in this group. Future Direction: This study underscores the efficacy of well-planned questioning approaches in enhancing reading comprehension. Further research could delve deeper into refining and optimizing these strategies to foster even better academic outcomes for O-level students in Pakistan.

Efficacy of a metacognitive intervention programme on reading comprehension in third grade students

This study examined the efficacy of a metacognitive intervention programme in a classroom group of 33 third-grade students. Two groups were formed: the intervention group with students in the classroom who showed reading delay and cognitive attention processes (n=8) and the comparison group (n=25). In the intervention group, a planning facilitation programme was implemented that aimed to improve inhibition processes and, correspondingly, overall reading comprehension indices. The Das-Naglieri: Cognitive Assessment System (D.N: CAS; Naglieri & Das, 1997) was used to measure attention processes. For reading processes, the Catalan et al. (2010) Reading Comprehension Assessment (ACL) test was used. The measurements were taken at pre-test, post-test, and follow-up. The results showed that the intervention group equalled the comparison group in reading comprehension and significantly improved in inhibition processes in the post and follow-up measures. It is concluded that the programme has had an impact on improving inhibition processes when facing reading activities.

An Empirical Study on Enhancing English Reading Skills among Female Middle School Students in Government Schools in Saudi Arabia and its Direct Impact on Academic Achievement

This study aimed to address the issue of low English reading proficiency among female middle school students in government schools in Saudi Arabia. The study targeted a sample of year-seven students; forty-two students were under direct observation in classroom and participated in interviews, questionnaire, and assessment forms. The primary focus of this study was to employ reading comprehension assessments as the major influencing factor.
 
 The main research question was how reading comprehension assessments was used to enhance the English reading proficiency of year 7 female students. This study primarily examined reading comprehension assessments, gathered existing literature on the topic, and formulated a hypothesis to help resolve the problem. The result of this study shows clear indication that reading comprehension assessments had a positive impact on students’ attitude toward the English language subject. Students’ reading proficiency improved, and their confidence in reading in front of their peers increased. Incorporating various activities contributed to achieving lesson objectives, deepened comprehension, and enhanced reading comprehension skills. Integrating modern technology into education stimulated students’ motivation to learn.

Intensive reading strategies in EFL reading comprehension improvement

This study aims to investigate the efficacy of using intensive reading strategies in improving reading comprehension in the EFL classroom. Because of the children’s poor reading comprehension, classroom exercises are devised. It is a quantitative research method with a quasi-experimental design. A survey is administered to two groups of students to assess their reading comprehension assessments. Following that, both the control and experimental groups complete a pre-test. The experimental group then participated in a classroom intervention that focused on using intensive reading tactics to increase their reading comprehension. The target population is then given a post-test, and the findings were children examined to assess the improvement in students’ reading comprehension before and after treatment. It is determined that intensive reading tactics are beneficial in boosting reading comprehension; thus, they should be used to assist students in their English language learning journey.

The Correlation between Students’ Reading Habit and Their Reading Comprehension Achievement

This study has objectives to discover the significant correlation and influence between reading habits and their reading comprehension successes of students. This study took a quantitative approach, implementing a correlational research method. The research population came from a total of 416 eleventh grade students at SMAN 21 Palembang. The cluster random selection technique was used to choose three classes of 104 students each. The data were gathered through the use of a questionnaire designed to assess reading habits, as well as a test aimed at assessing reading comprehension achievement. The results demonstrated that there was an important relationship between students' reading habits and their reading comprehension achievement, with the r obtained (.321) was being greater than the r-table (.192) and the significant value achieved .001, which was less than .05. Furthermore, the regression analysis result displayed that reading habit significantly contributed to reading comprehension achievement, with the R-square value .118, which stating that students' reading habit contribute 11.8% to their reading comprehension achievement.

Modeling item-level heterogeneous treatment effects: A tutorial with the glmer function from the lme4 package in R.

Recent advancements in education scholarship have introduced Item Response Theory (IRT) models to address treatment heterogeneity at the assessment item level. These models for item-level heterogeneous treatment effects (IL-HTE) enable detailed analyses of treatments that may have varying impacts on individual items within an assessment. This article offers a comprehensive tutorial for applied researchers interested in implementing IL-HTE analysis in R, utilizing the lme4 package. Using empirical data from a second-grade reading comprehension assessment as a running example, this tutorial emphasizes model-building strategies, interpretation techniques, visualization methods, and extensions. By following this tutorial, researchers will gain practical insights into utilizing IL-HTE analysis for enhanced understanding and interpretation of treatment effects at the item level.

QUA-RC: the semi-synthetic dataset of multiple choice questions for assessing reading comprehension in Ukrainian

In this article we present the first dataset of multiple choice questions for assessing reading comprehension in Ukrainian. The dataset is based on the texts from the Ukrainian national tests for reading comprehension, and the MCQs themselves are created semi-automatically in three stages. The first stage was to use GPT-3 to generate the MCQs zero-shot, the second stage was to select MCQs of sufficient quality and revise the ones with minor errors, whereas the final stage was to expand the dataset with the MCQs written manually. The dataset is created by the Ukrainian language native speakers, one of whom is also a language teacher. The resulting corpus has slightly more than 900 MCQs, of which only 43 MCQs could be kept as they were generated by GPT-3.