Programme For International Student Assessment Research Articles

The influence of ICT competencies on English digital reading achievement

Due to the prevalence of information and communication technologies (ICT) among secondary school students and the increasing significance of digital reading, exploring the impact of ICT on students’ digital reading performance in English seems necessary. The current research aimed to examine the effects of ICT competencies factors on secondary school students’ English digital reading performance. Data were obtained from 7703 15-year-old students from three East Asian economies, Macau, Hong Kong, and Singapore, from the Programme for International Student Assessment (PISA) 2018 dataset. The results of hierarchical regression analysis revealed that students’ ICT literacy was negatively correlated with their English digital reading performance, while the influence of attitudes toward ICT varied across the three East Asia economies. Additionally, males had lower English digital reading achievement, while students with higher socioeconomic status (SES) had higher English digital reading performance.

Towards Defining Climate Adaptation Literacy in the Context of Urban Resilience

Abstract The climate crisis intensifies rapidly, straining the urban environment and society. Addressing climate change requires collaborative efforts, integrating physical resilience with knowledge empowerment. Formal and informal knowledge are vital for effective adaptation. In the updated Intergovernmental Panel on Climate Change (IPCC) AR6 Report an emphasis on knowledge and its role in reducing response risk was highlighted. At the urban scale, adaptation knowledge is essential, bridging social and technical systems. Here, community knowledge evolves from a passive component within a greater adaptation strategy to an active, integral element. This paper develops a unique definition of climate adaptation literacy for the urban built environment based on an extensive literature review on urban climate adaptation, guided by the PISA (Program for International Student Assessment) 2025 Framework. The results of this work highlight existing gaps in climate adaptation knowledge and present urban climate adaptation literacy as a means to measure the response risk. These insights aid the formulation of resilience policies and urban adaptation strategies and contribute to the safety, resilience, sustainability, and functionality of the material and non-material values of the urban environment

An adaptation of Random Forest to estimate convex non‐parametric production technologies: an empirical illustration of efficiency measurement in education

AbstractThis paper presents a novel approach to conduct non‐parametric estimations of production technologies that adhere to the basic assumptions of production theory axioms, including free disposability in inputs and outputs and convexity. The methodology is rooted in adapting the highly effective machine learning techniques associated with Random Forest and the use of splines. The new method features a piecewise linear estimator analogous to data envelopment analysis (DEA); however, it distinguishes itself by addressing DEA's overfitting and lack of robustness via randomization of data and input variables in the construction of the models. In this paper, the virtues of employing machine learning techniques for assessing the efficiency of public services, particularly in the realm of educational institutions, are underscored. The new approach has the capability to predict outputs based on inputs, even for units not included in the observed sample. Furthermore, it enables the identification of the most relevant inputs in relation to output production. To demonstrate the advantages of our method, an estimation of the educational production function is conducted for Spanish regions utilizing data sourced from the Program for International Student Assessment.

Evaluation of Reading Empowerment Asserting Connection at Home (REACH): Performance of Grade 7 Learners

In the Philippines, reading is a persistent challenge that requires continuous remediation with the results of the Program for International Student Assessment (PISA) from both 2018 and 2022, and the country has been at the bottom of the rankings. Thus, the goal of this study is to strengthen reading remediation. This study focused on the evaluation of a reading program implemented; the Reading Empowerment: Asserting Connection at Home (REACH), and its relationship to the performance of Grade 7 learners in relation to reading. The participants of this study were 96 secondary school English teachers because they are field implementers of the reading program and were selected through a proportional stratified random sampling to ensure that each cluster was represented. The study collected MPS results from three successive school years and evaluated the implementation of the reading program. The questionnaire was validated and researcher-made. Based on the findings, the hypotheses were partially supported. It was revealed that there was a significant relationship between the REACH evaluation and the school’s reading performance, particularly between the Product evaluation and the school’s reading performance during SY 2022-2023, and between the overall REACH evaluation and the school’s reading performance during SY 2023-2024. Furthermore, there was a positive relationship between the REACH Evaluation in terms of Context and Psychological Component. It was recommended that curriculum implementers continue the implementation of Reading Empowerment: Asserting Connection at Home (REACH), but this time, we focused on strengthening the other components of the CIPP Evaluation to improve the results of the reading program.

Examining the associations between high achievement in reading and school climate: evidence from five South American countries

An emergent literature has shown that some students overcome adversities related to their low Socio-Economic Status (SES) by attaining high academic achievement. One of the aims of this literature is to identify the factors that explain the capacity of students from low SES backgrounds to attain high academic achievement. However, upon reviewing this literature, I observed that few studies have comparatively investigated the associations between the capacity of students from low SES backgrounds to attain high achievement in reading and school climate characteristics. Additionally, I found that not many studies have researched whether student SES moderates these associations. This paper contributes to fill these two gaps by estimating associations between high achievement in reading and school climate characteristics. Moreover, it examines whether student SES moderates these associations. To do this, this research estimated logit and heterogeneous choice models using representative samples of students from Argentina, Brazil, Chile, Colombia, and Uruguay who participated in the 2018 Programme for International Student Assessment (PISA). The results show that high achievement in reading is positively correlated to certain teaching practices, such as perceived teacher enthusiasm during the instruction and adapting instruction to students’ needs. Conversely, it is negatively correlated to teacher feedback and teacher-directed instruction. Furthermore, the findings indicate that high achievement in reading is negatively correlated to the scarcity and low quality of educational material. Interestingly, the results show that student SES does not moderate these associations. In conclusion, these findings indicate that interventions targeting these areas of school climate may help to increase students’ probability of attaining high achievement in reading, regardless of their SES. Likewise, these results suggest that if these interventions are focused in low SES schools, they may contribute to bridge the gap in reading skills between students from low and middle or high socioeconomic backgrounds.

A framework for comparing large-scale survey assessments: contrasting India’s NAS, United States’ NAEP, and OECD’s PISA

Large-scale survey assessments (LSAs) are important tools for measuring educational outcomes and shaping policy decisions. We present a framework for comparing LSAs to facilitate studying the impact of design choice on the precision of results, contrasting India’s National Achievement Survey (NAS), the United States’ National Assessment of Educational Progress (NAEP), and the OECD’s Programme for International Student Assessment (PISA). Our framework focuses on four key elements: sampling design, assessment design, analysis methodology, and reporting. The notion of total survey error, which is the accumulation of errors across the four key elements, can be used for both designing and evaluating LSAs. As example, we compare statistics that are commonly (but not always) reported from NAS, NAEP, and PISA to summarize outcomes related to sampling, measurement, and reporting. Our examination reveals several key similarities and differences among the three assessments, thereby highlighting the nuanced ways in which each LSA is tailored to meet the specific needs of their purpose and the challenges they face.

META-ANALYSIS: THE IMPROVEMENT NUMERACY LITERACY THROUGH PROBLEM BASED LEARNING

Numeracy literacy is a very important skill in the 21st century. However, the numeracy literacy rate in Indonesia is still low, this can be seen from the results of the PISA test and various observational studies in schools. One way to overcome this problem is by applying the Problem Based Learning (PBL) model. The aim of this research is to determine the improvement numeracy literacy through the PBL model. The research method used is meta analysis with a sequence of preparation, implementation and analysis stages. The analyzed articles were 10 articles, with the criteria of each article discussing the PBL model for numeracy literacy, and there was information about the mean and standard deviation. The results obtained are that students' numeracy literacy through the PBL model has increased, where . It was concluded that there was a significant difference between the pre-test and post-test literacy numeracy scores before and after the PBL model. The average pre-test and post-test scores are 62.54 and 77.92 respectively, numeracy literacy in the post-test is higher than in the pre-test, so it can be concluded that the PBL model can increase students' numeracy literacy. The articles analyzed have similarities and differences in results and views because many aspects are applied. There were 2 different studies found, both of which received an Effect Size (ES) value of 0.09 and were categorized as small, so further research related to PBL on numeracy literacy, especially at the elementary school level, considering that in this meta analysis only used 1 data with the elementary school level whose results were not significant.

Methodological aspects of the highly adaptive testing design for PISA.

This methods paper describes the methodological and statistical underpinnings of the highly adaptive testing design (HAT), which was developed for the Programme for International Student Assessment (PISA). The aim of HAT is to allow for a maximum of adaptivity in selecting items while taking the constraints of PISA into account with appropriate computer algorithms. HAT combines established methods from the area of computerized adaptive testing (a) to improve item selection when items are nested in units, (b) to make use of the correlation between the dimensions measured, (c) to efficiently accomplish constraint management, (d) to control for item position effects, and (e) to foster students' test-taking experience. The algorithm is implemented using the programming language R and readers are provided with the necessary code. This should facilitate future implementations of the HAT design and inspire other adaptive testing designs that aim to maximize adaptivity while meeting constraints.

English digital reading achievement for East Asian students: identifying the key predictors using a machine learning approach

ABSTRACT Students’ digital reading literacy has attracted increasing attention in the current digital era; however, few studies have been conducted to explore how factors at different levels influence students’ digital reading achievement. Grounded by socio-ecological theory, the current comprehensively explored the relative importance of 24 individual, microsystem, and mesosystem variables in predicting English digital reading achievement through the machine learning approach (i.e. random forest regression). The secondary data were retrieved from the Program for International Student Assessment (PISA) 2018, including 7,703 15-year-old students from Macao, Hong Kong, and Singapore. Our study identified 12 key factors that best predicted East Asian students’ English digital reading achievement. Among them, students’ socioeconomic status, subject-related ICT use during lessons, and interest in ICT ranked as the top three factors. The disparities in the roles played by disciplinary climate, gender, being bullied, immigrant status, and home language among the three economies were discussed.

The role of item format in the PISA 2018 mathematics literacy assessment: A cross-country study

When construct-irrelevant sources affect item difficulty, validity of the assessment is compromised. Using responses of 260000 students from 71 countries to the Programme for International Student Assessment (PISA) 2018 mathematics assessment and cross-classified mixed effects models, we examined three validity concerns associated with the construct-irrelevant factor, item format: whether the format influenced item difficulty, whether item format’s impact on difficulty varied across countries, undermining PISA’s foundational goal of meaningful country comparisons, and whether item format effects differed between genders, affecting assessment fairness. Item format contributed to a substantial average of 12 % of variance in item difficulties. The effect of item format was non-uniform across countries, with 30 % of the variance in item difficulties being due to format in lower-performing countries, and 10 % in higher-performing countries, challenging the comparability of educational outcomes. The impact of gender on item format differences was minor. Implications for secondary research and assessment design are discussed.

Understanding the relationship between competitive climates and reading achievement in East Asia: a situated expectancy-value theory perspective

ABSTRACT Competitive climates have been found to be critical for language learning. However, most of these studies primarily focused on examining the direct association between competitive climates and language performance. Little attention has been given to exploring how competitive climates interplay with motivational beliefs in influencing language learning, especially in the Confucian culture. To address these gaps, this study aimed to investigate how expectancy-value beliefs mediated the relationship between competitive climates and reading achievement in East Asian regions. The study utilized data from the Programme for International Student Assessment (PISA) 2018, with 41,872 participants from six East Asian regions, including Chinese Taipei, Hong Kong, Macao, Japan, Korea, and Mainland China. The results revealed that competitive climates could enhance students’ reading achievement by fostering expectancy and value beliefs, demonstrating the positive influence of competitive climates on reading achievement within East Asian regions. The theoretical and practical implications are discussed.

The Gender-Equality Paradox in Intraindividual Academic Strengths: A Cross-Temporal Analysis

Independent of overall achievement, girls’ intraindividual academic strength is typically reading, whereas boys’ strength is typically mathematics or science. Sex differences in intraindividual strengths are associated with educational and occupational sex disparities in science, technology, engineering, and mathematics (STEM) fields. Paradoxically, these sex differences are larger in more gender-equal countries, but the stability of this paradox is debated. We assessed the stability of the gender-equality paradox in intraindividual strengths, and its relation to wealth, by analyzing the academic achievement of nearly 2.5 million adolescents across 85 countries and regions in five waves (from 2006 to 2018) of the Programme for International Student Assessment (PISA). Girls’ intraindividual strength in reading and boys’ strength in mathematics and science were stable across countries and waves. Boys’ advantage in science as an intraindividual strength was larger in more gender-equal countries, whereas girls’ advantage in reading was larger in wealthier countries. The results have implications for reducing sex disparities in STEM fields.

Artificial Neural Network and Adaptive Neuro Fuzzy Inference System Hybridized Models in the Sustainable Integration of Language and Mathematics Skills: The Case of Singapore and Hong Kong

The four basic language skills, listening, speaking, reading, and writing play, a crucial role in the development of an individual’s skills in other disciplines. The current study aims to underpin the relationship between language skills and mathematics skills by focusing on the language and mathematics curricula of two consistently high-achieving countries, Hong Kong and Singapore, in the Program for International Student Assessment (PISA) rankings. In the current study, the convergent parallel mixed method was utilized that qualitative and quantitative data were composed together. Primarily, the outcomes of four language skills were determined in the native language teaching curricula of the two countries. The topics and themes related to four basic language skills were determined from the two mathematics curricula. The curricula were examined by document analysis from qualitative research methods. The analysis was conducted by examining the native language teaching and the mathematics curricula of both countries by the content analysis method. Later, the findings of the document analysis were used to develop machine learning models to find a possible positive relationship between language skills and the PISA scores. Although a number of previous studies have found a reasonable relationship between language skills and mathematics skills, the current study results were contradictory to the ones performed previously in the literature, and considering the curricula no positive relationship between the language and mathematics skills was found. The findings of the current study were further supported by the artificial neural network (ANN) and Adaptive Neuro Fuzzy Inference System (ANFIS) model performance metrics. Compared to an acceptable level of 0.80, significantly low R2 values of 0.35 and 0.39 for the ANN and ANFIS models, respectively, indicated very little relationship between the language and mathematics skills.

Mathematical literacy learning environment for inclusive education teachers: A framework

The Program for International Student Assessment (PISA) 2022 study revealed low mathematical literacy scores among students, including those in inclusive education settings. A significant factor contributing to this issue is the inadequate capacity of teachers to effectively teach mathematical literacy. In response, this study aimed to design a mathematical literacy learning environment framework to enhance teachers' theoretical and practical understanding of teaching mathematical literacy in schools. The research employed a design research methodology, specifically the development study type, which involved preliminary research, prototype development, and evaluation stages. This paper focuses on the preliminary research phase, where data were collected through literature reviews and focus group discussions (FGDs) with teachers from inclusive schools. The collected data were analyzed qualitatively and descriptively. The study successfully developed a framework for a mathematical literacy learning environment that integrates courses, social media, and community engagement. This framework is intended to serve as a comprehensive reference for improving teachers' capacity to teach and assess students' mathematical literacy in a more holistic manner.

Assessment of student ICT competence according to mathematics, science, and reading literacy: evidence from PISA 2018

To assess the integration of information and communication technologies (ICTs) in teaching and learning, a new construct for ICT competence was developed by the Programme for International Student Assessment (PISA) in 2022. The framework for ICT assessment involves four major dimensions; access to ICT, use of ICT, students’ ICT competencies, and subject-related ICT use. In this paper, we analysed this ICT assessment in terms of validity construction of test scores with data from PISA 2018 and investigated ICT competence in relation to the variables of mathematics, science, and reading literacy. The analyses were based on public data from PISA 2018 for Turkey (N = 6890). After removal of univariate and multivariate outliers from the Turkish sample, factor analysis, correlation analysis, and structural equation modelling were performed on 5438 data items. In the study, we confirmed the four-dimensional structure of ICT competence for the Turkish sample. In this study, the construct validity of the PISA 2018 application ICT test was verified using the Turkish sample. The ESCS variable was included as a control variable in the analyses due to its significant impact on the relevant variables. The analysis showed that the mathematical literacy scores of the PISA 2018 Turkish students were positively low affected by the availability of ICT, negatively low affected by the entertainment of ICT, and positively low affected by the interest and perceived competence in ICT, use of ICT for educational purposes at school and outside school, and the use of ICT related to the subject. Considering the ICT variables that affected the PISA 2018 students’ science literacy scores; the entertainment of ICT and the interest and perceived competence in ICT significantly low affected the students’ use of ICT on the subject. On the other hand, the availability of ICT negative significantly low affect their science literacy scores. While the entertainment of ICT negatively low affected the science literacy scores, interest and perceived competence in ICT and use of ICT for educational purposes at school and outside school positively low affected the students’ use of ICT on the subject. Finally, when we look at the ICT variables that low affected the students’ reading literacy scores; and use of ICT for educational purposes at school and outside school and the interest and perceived competence in ICT significantly positive low affected the use of ICT on the subject. On the other hand, the availability of ICT negative significantly low affect reading literacy scores. While the entertainment of ICT negatively low affected reading literacy scores; interest and perceived competence in ICT positively low affected the use of ICT on the subject.

Factors influencing students’ reading literacy in Morocco: A multilevel analysis

Morocco is a multilingual country with a diverse culture. When it participated in the Program for International Student Assessment (PISA) for the first time, its reading literacy scores ranked relatively low in the test. Employing hierarchical linear modeling (HLM), the present study examined the effects of both student- and school-level factors on students’ reading literacy in Morocco. Student-level (N = 5800) and school-level (N = 179) data were drawn from the PISA 2018. The results showed that students’ reading literacy differed significantly across schools, accounting for about 40.5% of the overall variation. Further, the results showed that at the student level, when the variables of gender and family indices (i.e., economic, social, and cultural status) were controlled, language spoken at home, understanding and remembering of meta-cognition, self-concept of reading, expected occupational status, adaptation to instruction, teacher support in test language lessons, and duration in early childhood education and care all had significant effects on students’ reading literacy. At the school level, school location, school type, school size, shortage of educational material, and proportion of highly educated teachers had a crucial influence on student reading literacy, either directly or indirectly. To date, the reasons for these effects have not been systematically investigated. Morocco was the only African country to take part in the PISA test and it would be worthwhile to explore the factors affecting reading literacy in this low-performing country. Further, the results may be applicable to some extent to low reading performance in other countries.

Self-Efficacy and Life Satisfaction in Adolescence: Testing the Presumed Role of Individualism–Collectivism

Previous studies suggest a positive association between self-efficacy and subjective well-being. However, these investigations depend on the dichotomy between East Asian and Western cultures, often without directly examining the influence of cultural orientation. This raises questions: How do the psychological benefits of adolescent self-efficacy vary across countries, and to what extent does this variance relate to countries’ cultural orientations? To address these questions, the current study analyzed a sample of 15-year-old students across 66 countries ( N = 427,584) drawn from the 2018 Programme for International Student Assessment (PISA) data. As expected, we found that higher self-efficacy was related to better life satisfaction among adolescents in every country studied. Notably, the beneficial role of self-efficacy varies based on cultural context: The relationship between self-efficacy and adolescent life satisfaction is stronger in individualistic than collectivistic countries. These findings suggest that cultural environment is important for understanding adolescent social-psychological well-being and potentially offer broader implications for global mental health initiatives.

School composition and academic achievement: The case of Russian-medium and Polish-medium schools in Lithuania

After Lithuania re-gained its independence, linguistic and cultural minority groups residing in the country had the right to be educated in non-Lithuanian-medium schools. Yet, the education of linguistic minority children has been a contentious topic in Lithuania for many years, and there is no conclusive evidence on how minority-language-medium schools in Lithuania perform academically, with national assessment suggesting that they perform worse than Lithuanian-speaking schools, and international assessment showing mixed results. Our research focuses on Russian- and Polish-medium schools and analyses the difference in academic performance among children attending Lithuanian-medium school and minority-language schools, as well as its reasons and social implications. Using the OECD Program for International Student Assessment (PISA) data, we find that minority-language children perform better when attending a school in which the language of instruction is the same as their home language.

Individualism and School Performance: Evidence from PISA Test Scores

Individualism and School Performance: Evidence from PISA Test Scores

What teachers can do to enhance students’ well-being: Discussion

BackgroundThis special issue sheds light on how students’ well-being can be enhanced through teaching practices, emphasizing the significance of well-being in the educational context. My discussion was rooted in extensive well-being literature, which allowed me to outline key findings about the importance, effects, and conceptualizations of well-being, and factors that may influence well-being. MethodThe issue comprises seven empirical studies, each exploring the relationship between teacher-related variables (e.g., emotions, growth mindset) or teaching-related variables (e.g., intercultural education, need-supportive teaching) and student well-being. It utilizes studies with methodologies ranging from large-scale assessments like the Program for International Student Assessment (PISA) to detailed longitudinal designs across various educational levels and contexts. ResultsFindings underscore the pivotal role of teaching in influencing student well-being. That is, specific teacher behaviors and pedagogical approaches were identified that correlate with student well-being. Key aspects included intercultural education, need-supportive teaching, and positive teacher–student relationships. The methodological strengths of the included studies were their use of large datasets, longitudinal designs, and multi-informant data. Based on these findings, I presented some suggestions for possible methodological improvements. ConclusionThis SI contained impressive articles that can be used as excellent models for research in this area. The findings can also expand our current knowledge by providing a multifaceted, multinational, and multi-informant understanding of the topic. Practical applications are suggested how teachers can support their students’ well-being and highlighted the importance of integrating well-being into educational discourse to provide a foundation for future investigations of teaching and learning processes.