Progress In International Reading Literacy Study Research Articles

Reading Comprehension Questions in Fourth Grade Arabic Language Textbooks in Oman: An Analysis From the Perspective of Progress in International Reading Literacy Study

The study aimed to determine the extent to which the reading comprehension questions in the book, I Love My Language for the Fourth Grade, Part One and Two, consider the reading comprehension processes. It also measured the level of consideration of the reading comprehension questions for the reading comprehension processes by the international study Progress in International Reading Literacy Study (PIRLS). The descriptive analytical method was used, and its tool was a content analysis card, which included four processes with 237 questions. The results showed that the book I Love My Language for the fourth grade included the four reading comprehension processes and carefully analyzed most of its sub-processes. It is recommended to reconsider the balance of processes in the book. This is important in light of the reading comprehension processes and the diversity of reading comprehension questions. It is necessary to take into account the four comprehension processes and their sub-processes in the two parts measured by the international study "PIRLS".

The teacher’s role in teaching reading comprehension skills to Grade 9 English Home Language learners

Background: The high number of poor-performing learners in reading comprehension in South African schools is a concerning factor that calls for studies to mitigate this problem. The Progress in International Reading Literacy Study (PIRLS) of 2021 indicates that about 81% of South African Grade 4 learners who participated, failed to reach the required benchmark in the test. Research indicates that teachers are not conversant with theory and explicit teaching of reading comprehension strategies.Objectives: This study aimed to investigate the role played by teachers in developing Grade 9 learners’ reading comprehension skills in English Home Language and establishing which reading comprehension skills and strategies are taught to learners.Method: This study used a qualitative approach. Four data sets were utilised to collect data including a document analysis, a questionnaire for learners, interviews with teachers of English and lesson observations.Results: The findings of the study show that there is little guidance on how to teach reading comprehension skills. It also emerged that instead of teaching reading comprehension skills, teachers merely ‘test’ learners’ answering skills.Conclusion: This study suggests that there is a need for re-training and assisting teachers in explicitly teaching reading comprehension skills to learners. Learners may benefit from the enhanced expertise of teachers and become more proficient in reading comprehension.Contribution: The study has provided invaluable insights into the current teaching practice of comprehension skills with actionable insights for teachers, teacher educators, and policymakers to support learners’ academic success and bridge the gap between praxis and theory.

Does Early Childhood Education Matter on Reading Achievement Among Arab Countries

Our research focuses on the impact of Early Childhood Education (ECE) in the reading achievement of pupils in primary schools in MENA countries. The study employed the international data from PIRLS 2016 to identify the effect of early literacy practices on later reading attainment. Leveraging data from the 2016 Progress in International Reading Literacy Study (PIRLS) by the International Association for the Evaluation of Educational Achievement, we utilize ordinary least squares (OLS) regression and quantile regression to estimate education production functions for each country individually. This approach enables us to assess how coefficients on explanatory variables vary across the spectrum of test scores. The results show that in MENA countries, early childhood education, which develops pupils’ skills acquisition before formal schooling, improves their reading performance. In sum, The Early childhood education effect can be reinforced through government policy and family background. In addition, we recommend a universal access to early childhood education., concentration of early childhood services in the private sector makes the programs confined only to wealthy people who can afford them, especially people living in urban cities and improve the quality of early education by implementing appropriate curricula and modern teaching.

Technology factors related to the differences in paper and online reading scores in PIRLS 2016

The Progress in International Reading Literacy Study (PIRLS) was conducted in paper and online reading modes in 2016 using the same samples of students in a number of countries. Differences in reading literacy scores were found in several European countries. In some countries, the differences favored the electronic reading mode. Yet in others, the paper reading mode was favored. As the electronic reading mode differs substantially in the cognitive demands compared to the paper mode, it can be expected that the differences between the two modes are related to the variables related to technology: availability and access, general use, use for educational purposes in class or out-of-school, and self-efficacy with technology. This study investigates the Information and Communication Technology (ICT) factors related to the differences in paper and online reading in six European countries participating in both modes in PIRLS 2016. This study uses linear regression models as the application of multilevel modeling is not suitable because of the low between-school variances across countries. The results from this study show limited support for the relative effect that the student individual, school, and classroom ICT variables have on the differences between paper and electronic reading. Access to technology is related to mode differences only in Italy, and the use of computer devices in and out of school is related to the mode differences in Italy and Portugal. Student self-efficacy is related to the mode differences in Portugal and Slovenia. School resources show significant effects in Denmark (computers to students ratio) and Italy (instruction affected by digital resource shortages). None of the classroom variables showed any significant relationship in any of the countries. In addition, socio-economic status (which is proxied by the variable on home resources for learning) is a significant predictor in half of the countries. In addition to these findings, the general technological context within countries is discussed as part of an evaluation of the difference in reading in the two test delivery modes. The general uptake of technology in different social and economic aspects, as measured by the Digital Economy and Society Index (DESI), follows the differences between the two reading modes.

Comparative Study of GenAI (ChatGPT) vs. Human in Generating Multiple Choice Questions Based on the PIRLS Reading Assessment Framework

ABSTRACTHuman‐generated multiple‐choice questions (MCQs) are commonly used to ensure objective evaluation in education. However, generating high‐quality questions is difficult and time‐consuming. Generative artificial intelligence (GenAI) has emerged as an automated approach for question generation, but challenges remain in terms of biases and diversity in training data. This study aims to compare the quality of GenAI‐generated MCQs with humans‐created ones. In Part 1 of this study, 16 MCQs were created by humans and GenAI individually with alignment to the Progress in International Reading Literacy Study (PIRLS) assessment framework. In Part 2, the quality of MCQs generated was assessed based on the clarity, appropriateness, suitability, and alignment to PIRLS by four assessors. Wilcoxon rank sum tests were conducted to compare GenAI versus humans generated MCQs. The findings highlight GenAI's potential as it was difficult to differentiate from human created questions and offer recommendations for integrating AI technology for the future.

Teacher-centered analysis with TIMSS and PIRLS data: weighting approaches, accuracy, and precision

This paper extends existing work on teacher weighting in student-centered surveys by looking into aspects of practical implementation of deriving and using weights for teacher-centered analysis in the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS). The formal conditions to compute teacher-centered weights are detailed, including mathematical equations. We provide a proposal on how to define the targeted populations as well as how to collect data that is needed to derive teacher-centered weights, yet currently unavailable. We also tackle the issue of teacher nonresponse by proposing a respective adjustment factor, as well as mentioning the challenge of multiple selection probabilities when teachers teach in multiple schools. The core part of the paper focuses on studying the level of accuracy that can be expected when estimating teacher population characteristics. We use TIMSS 2019 data and simulate likely scenarios regarding the variance in weights. The results show that (i) the different weighting scenarios lead to relatively similar estimates; however, the differences between the scenarios are sufficient to justify the recommendation to use correctly derived teacher weights; (ii) differences between estimated standard errors based on complex sampling and corresponding estimates based on simple random sampling are sufficiently consistent to support use of a procedure to estimate standard errors that accounts for both sample weights and the complex sampling design; (iii) sample sizes and variance in weights significantly limit estimate precision, so that total population estimates with sufficient precision are available in the majority of countries but subpopulation features are generally not sufficiently precise. To provide a critical evaluation of our results, we recommend implementation of the proposed method in one or more countries. This recommended study will permit examination of logistical considerations in implementation of required changes in data acquisition and will provide data to replicate the analysis with teacher-centered weights.

A psychometric evaluation of an item bank for an English reading comprehension tool using Rasch analysis

This study reports the psychometric evaluation of an item bank for an Assessment for Learning (AfL) tool to assess primary school students’ reading comprehension skills. A pool of 46 primary 1 to 6 reading passages and their accompanying 522 multiple choice and short answer items were developed based on the Progress in International Reading Literacy Study (PIRLS) assessment framework. They were field-tested at 27 schools in Singapore involving 9834 students aged between 7 and 13. Four main comprehension processes outlined in PIRLS were assessed: focusing on and retrieving explicitly stated information, making straightforward inferences, interpreting and integrating ideas and information, and evaluating and critiquing content and textual elements. Rasch analysis was employed to examine students’ item response patterns for (1) model and item fit; (2) differential item functioning (DIF) about gender and test platform used; (3) local item dependence (LID) within and amongst reading passages; and (4) distractor issues about options within the multiple-choice-type items. Results showed that the data adequately fit the unidimensional Rasch model across all test levels with good internal consistency. Psychometric issues found amongst items were primarily related to ill-functioning distractors and local dependence on items. Problematic items identified were reviewed and subsequently amended by a panel of assessment professionals for future recalibration. This psychometrically and theoretically sound item bank is envisaged to be valuable to developing comprehensive classroom AfL tools that provide information for the English reading comprehension instructional design in the Singaporean context.

Enrolment in the first stage of early childhood education and students’ academic performance: a cross-country analysis

ABSTRACT Parents have the option of enrolling their children in the first stage of early childhood education (from 0 to 3 years of age). However, not all parents decide to do so, waiting until the second stage of early childhood education to enrol them in the education system (from 3 to 5 years of age), or even until compulsory education when their children are around 6. We intend to analyse the influence of students’ enrolment in the first stage of early childhood education on their fourth-grade reading scores. This analysis has been performed using data from the Progress in International Reading Literacy Study (PIRLS) 2011 and 2016 for 39 countries and an instrumental variable approach to go beyond simple correlation. We find that attending the first stage of early childhood education has a positive influence on students’ reading scores in 18 countries, whereas it presents a null influence in 16 countries.

Re‐examining the relationship between patience, risk‐taking, and human capital investment across countries*

SummaryHanushek et al. (2022) show that students in countries in which people are more patient and less risk‐taking perform better in the Programme for International Student Assessment (PISA) test. In this paper, we probe the robustness of this study. Our narrow replication shows that most of the results are robust to alternative model specifications. Our broad replication shows that the main results are robust to measuring student performance with data from the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) instead of PISA.

Exploring school factors of academic achievement in Macau: The application of educational data mining in Progress in International Reading Literacy Study (PIRLS) 2016 assessment

International large-scale assessments, provide structured and static data, and due to their extensive databases, they can be considered as a valuable resource for Big Data in Education. In this paper, we propose an educational data mining approach to detect and analyze factors linked to academic performance in Macau schools using data from the Progress in International Reading Literacy Study (PIRLS) 2016. We conducted a secondary data analysis based on a set of socioeconomic, process, and outcome variables from PIRLS and other sources, and built decision trees to obtain a predictive model of school performance. By doing so, we were able to identify the school and student-level variables that are most significant in predicting student performance in Macau. These findings will be useful for informing educational policy decisions and shedding light on the causes of poor performance in Macau schools. Overall, our study highlights the potential of educational data mining approaches in analyzing large-scale assessment data and generating insights for educational research and practice.

The effect of peer bullying on academic achievement: A meta‐analysis study related to results of TIMSS and PIRLS

AbstractFocusing on the variables that can affect both academic achievement and the well‐being of students has been crucial for their development, making schools effective and designing educational policy as well as curriculum. The study has aimed to investigate the effect of peer bullying on academic achievement and to determine moderators in the relationship between these two terms by using TIMSS (The Trends in International Mathematics and Science Study) and PIRLS (Progress in International Reading Literacy Study) data through the meta‐analysis method. Based on seven international survey data sets from TIMSS (2003, 2007, 2011, and 2015) and PIRLS (2006, 2011, and 2016), the current research has focused on 793 independent findings, including 3,503,005 students in 77 countries. In the analysis process, the mean effect size was measured by using the coefficient of standardized means (Cohen's d) based on the random effect model. In addition, the significance of the moderator variables was calculated by using the Q statistic. The findings of the study revealed that peer bullying had a significant effect on academic achievement, and in this effect, grades of the students and course types played moderator roles, while the culture of the countries was not a moderator. It could be concluded that the effect of peer relationships not only on the social development of students but also on their academic performance should not be ignored.

Investigating the relationship between parental attitudes toward reading, early literacy activities, and reading literacy in Arabic among Emirati children

Emirati children’s reading skills have consistently lagged behind global standards on international standardized tests. Given the United Arab Emirates’s Vision 2031, which aims for a world-class education system, and given the importance of Arabic literacy in preserving national identity and cultural heritage, there is an urgent need to investigate the factors that influence Arabic literacy among Emirati children. Therefore, this study, using data from the Progress in International Reading Literacy Study (PIRLS) 2016 database, aimed to examine the relationships between parents’ attitudes toward reading, early literacy activities, early literacy tasks, and Arabic literacy. Specifically, the study aimed to (1) determine the predictive value of parental attitudes toward reading for Arabic reading literacy among Emirati children; (2) examine how early literacy activities and tasks contribute to Arabic reading literacy; and (3) examine the mediating role of early literacy activities and tasks in the relationship between parental attitudes and Arabic reading literacy. The results of the study suggest that parental attitudes toward reading are an important predictor of their children’s Arabic reading literacy. Participation in early literacy activities before primary school and mastery of early literacy tasks at the beginning of primary school were both associated with reading proficiency in Arabic. Furthermore, both early literacy activities and early literacy tasks significantly mediated the relationship between parental attitudes toward reading and children’s reading proficiency in Arabic. The findings are critical for educators to adapt teaching methods, curricula, and parental involvement programs to more effectively support Arabic literacy development. The study highlights the need for an inclusive approach involving parents, educators, and policymakers to create an enabling environment for the development of Arabic literacy, which is essential for academic success and cultural preservation.

Regularized Mislevy-Wu Model for Handling Nonignorable Missing Item Responses

Missing item responses are frequently found in educational large-scale assessment studies. In this article, the Mislevy-Wu item response model is applied for handling nonignorable missing item responses. This model allows that the missingness of an item depends on the item itself and a further latent variable. However, with low to moderate amounts of missing item responses, model parameters for the missingness mechanism are difficult to estimate. Hence, regularized estimation using a fused ridge penalty is applied to the Mislevy-Wu model to stabilize estimation. The fused ridge penalty function is separately defined for multiple-choice and constructed response items because previous research indicated that the missingness mechanisms strongly differed for the two item types. In a simulation study, it turned out that regularized estimation improves the stability of item parameter estimation. The method is also illustrated using international data from the progress in international reading literacy study (PIRLS) 2011 data.

Literacy as Epistemology and Educational Policy: An Exploration of a Large Swedish Professional Development Programme for Teachers

ABSTRACT From once being synonymous with reading and writing, literacy, as a concept has expanded in the last three decades to today being associated with multiple meaning making forms and competences. Impactful, in this shift, has been the research orientation of New Literacy Studies. Besides academia, the literacy concept has also had a major influence on the global assessment frameworks of Program for International Student Assessment (PISA) and Progress in International Reading Literacy Study (PIRLS). This study is interested in the epistemological potential of this perception of literacy, but also on how it affects educational policy. Material examined is a large Swedish governmentally-initiated professional development programme for teachers, launched in 2014. Two main variants of the literacy concept can be identified in the programme, one more closely aligned with New Literacy Studies, and the other more influenced by PISA and PIRLS. Findings show, strong impact from the global assessment version of the literacy concept, resulting in a mainly skills-based and instrumental educational discourse, far from the concept of Bildung, traditionally characteristic of the Nordic school systems. Further, of the two variants, New Literacy has a broader epistemological potential, and is, as such, more closely related to Bildung.

Comparing parental and school pressure in terms of their relations with students' well-being

Towards the end of the elementary school years, children experience increased learning demands and pressure to perform, which can threaten their well-being. However, while parents and teachers are considered the main sources of academic pressure, a systematic comparison of the two has not yet been conducted. Therefore, this study examined the relation between those sources of academic pressure and five different facets of well-being among elementary school children. Our sample comprised 342 fourth graders (45.9 % female, Mage = 10.29) participating in the field test of the Progress in International Reading Literacy Study (PIRLS) 2021 in Germany. Controlling for gender and academic achievement, parental and school pressure were negatively related to several facets of students' well-being, with our findings showing that parental pressure was associated with a greater number of aspects of well-being. However, a direct comparison of the two sources of academic pressure did not reveal significant differences in the relations with the various facets of students' well-being. Limitations of the study and implications for research and practice are discussed. Educational relevance statementStudents' well-being is an important educational goal and research on this topic is of great relevance. This study represents an important contribution to the current literature as it examines the relationship between students' well-being and academic pressure among elementary school children. Studies of this age group in this context are rare but nonetheless of great importance. In addition, the present study included two different sources of academic pressure (parents, individuals in the school environment). Direct comparison of different sources of academic pressure in the elementary school context provides a more nuanced picture of how it affects students during their early school career.We found that both, parental and school pressure are negatively related to several facets of well-being. However, the results did not show a preponderance of one source of academic pressure over the other in terms of its association with students' well-being.The findings are of great practical importance, as they indicate that the behavior of teachers and parents is related to the well-being of students at a sensitive stage of their school career and general development. Therefore, their awareness of this issue should be raised. In addition, students' resilience and stress management skills should be strengthened so that they can better cope with academic pressure.

Scope of Questionnaires in the Pirls 2021 Study

This scientific article explores the relationship between the living environment and various factors with student learning based on the findings of the Progress in International Reading Literacy Study (PIRLS) conducted in 2021. The study aims to investigate the influence of the living environment and other relevant aspects on students' academic achievements. The research employed a comprehensive questionnaire survey as the primary method to collect data from a diverse sample of students across different countries. The results demonstrate a significant correlation between the living environment and students' learning outcomes, highlighting the crucial role played by factors such as household socio-economic status, parental involvement, access to educational resources, and school environment. The implications of these findings underscore the importance of creating conducive living conditions and providing adequate support systems for students to maximize their learning potential. This research provides valuable insights for policymakers, educators, and stakeholders to design evidence-based interventions and policies aimed at enhancing student learning outcomes on a global scale. Highlights: The study examines the impact of the living environment and various factors on student learning based on PIRLS 2021 data. Results highlight a significant correlation between the living environment and students' academic achievements, emphasizing the role of socio-economic status, parental involvement, access to educational resources, and school environment. Findings have implications for policymakers, educators, and stakeholders in designing interventions and policies to enhance student learning outcomes globally. Keywords: Living environment, Student learning, PIRLS 2021, Factors, Academic achievements

The Impact of the Presence of Prosodic Features (Tone Markings) on Comprehending Setswana Words in Reading

The paper is about the impact of prosodic features, more especially tone marking, in extracting the meaning in Setswana readings, because of its contribution to meaning in Setswana. The research was conducted in response to the 2016 Progress in International Reading Literacy Study (PIRLS), which found that more than 80% of Grade 4 students tested in an African Language were unable to read for comprehension. The semiotic theoretical framework, which is concerned with how signs are used for interpretation, served as the foundation for this study. The focus of the study was on the Intermediate Phase learners from eight schools where Setswana is taught as a First Additional Language and/or as a Home Language. Qualitative and quantitative method approaches were used to collect data. Participants were chosen through random sampling. The study discovered that when words are not marked, learners are unable to determine which meaning is intended. It was discovered that in reading the most difficult aspect is comprehending what has been read. The study proposes that new means of emphasizing prosodic markers (such as tones, accent, stress, and rhythm) be given special focus in the South African education system so that learners may be able to extract meaning and comprehend what they read.

Associations between teacher quality, instructional quality and student reading outcomes in Nordic PIRLS 2016 data

Progress in International Reading Literacy Study (PIRLS) focuses on the reading proficiency of students mostly in the fourth year of schooling. A wide selection of studies has shown that family background and early literacy activities at home have substantial associations with student achievement in reading literacy. However, research focusing on teacher qualities and teaching processes is inadequate. In this study, we focus on associations of teacher quality (formal qualifications and professional identity) and instructional quality (classroom management, cognitive activation and teacher support) with cognitive and affective-motivational student outcomes (variables Reading Achievement, Students Confident in Reading, and Students Like Reading). We analyzed PIRLS 2016 data from four Nordic countries (Denmark, Finland, Norway, and Sweden), consisting altogether of 923 teachers and 17,161 students. Using path analysis, we considered selected background variables from teacher and student questionnaires in relation to the outcomes. Overall, the associations of student outcomes with teacher quality and instruction quality were found to be weak in all the countries, and there was little variation between the countries. The strongest association observed in all countries was the positive relation between Teacher Support Perceived by Students and Students Like Reading. Further, a positive Working Atmosphere in the Classroom tended to promote Reading Achievement and Students Confident in Reading. Teacher’s Specialization in reading and the language of the test was positively associated with Teacher’s Self-Efficacy in teaching reading, which in turn was related to measures of instructional quality. The implications for practice are discussed.

Associations between having breakfast and reading literacy achievement among Nordic primary school students

ABSTRACT Having breakfast is associated with improved diet quality, cognitive- and academic performance and can therefore positively impact learning and health, although the impact on reading literacy is unknown in the Nordic countries. The aim of this study was to assess the association between having breakfast often versus rarely and reading literacy achievement based on Progress in International Reading Literacy Study (PIRLS) from 2016. The sample consisted of Danish (N = 3508), Finnish (N = 4896), Norwegian (N = 4232) and Swedish (N = 4525) students, 10–11 years old. Students self-reported their frequency of having breakfast. Linear regression analysis (adjusted for socio-economic status and gender) showed that those who often vs. rarely had breakfast achieved a higher reading literacy score. The results suggest that having breakfast may be important for reading literacy achievement even after adjusting for socioeconomic status. This potential relationship between breakfast intake and students’ academic achievement should be given priority for further research and practice as breakfast is a modifiable factor that can be both improved and be intervened on.

Moderate Gaming and Internet Use Show Positive Association with Online Reading of 10-Year-Olds in Europe

• The paper examined the relationship between four screen activities and online reading scores of 10-year-olds in six high-income European countries. • Playing computer games and surfing the web showed a right-skewed inversed U-shape with moderate use having a mildly positive association with online reading when compared to both no-use and heavy use (above 2 hours daily). • Online chatting and watching videos showed a negative association with online reading above the threshold of 1 hour daily. • There were not significant differences between top and bottom performers apart from heavy gaming associated with 26-point (or over a quarter of a standard deviation) lower score for low-performers and 6-point lower score for high-performers. • The paper documents the emergence of online-offline reading gaps of the same children: 10-year olds who reported no screen use scored 6-11 points lower on online than offline test. Analogically, children who spent more time online scored higher on online tests than on offline tests. The paper analyses how four screen activities relate to reading scores using the representative sample of 21,217 ten-year-olds who sat online and offline Progress in International Reading Literacy Study (PIRLS) test in six high-income European countries. In regression models, gaming and Internet use showed a right-skewed inverted U-shape relationship to online reading with moderate use (30-60 minutes daily) showing a positive association when compared to both no-use and heavy use (above 2 hours). Online chatting and watching videos showed negative relationship to online reading above the threshold of approximately one hour daily. Quantile regression showed that all four types of screen time had similar influence on top and bottom performers except for gaming over 2 hours daily which was associated with 26-point (or over a quarter of a standard deviation) lower score for low-performers and 6-point lower score for top-performers. The paper documents the emergence of online-offline reading gaps: children who reported no screen use scored 6-11 points lower on online than offline test. Similarly, children who spent more time online scored higher on online tests than on offline tests. Whenever the heavy screen use yielded significant results, it was associated with higher online score (ranging from 8 to 16 points) when compared to offline score. A common finding for all screen activities, testing modes and groups of performers is an adverse effect on reading of more than two hours daily of screen time.