Programme For International Student Assessment Research Articles

Elite career expectations of adolescents: Popularity, gender differences, and social divides.

The supply of elite professionals is crucial for economic development, yet little is understood about the appeal and influencing factors of these careers among young people across different economies. It remains unclear whether adolescents in academically high-performing economies growingly expect emerging technological jobs in response to evolving workforce demands. This research used the Programme for International Student Assessment 2000-2018 data in 24 high-performing educational systems to examine the two-decade trends in adolescents' expectations for Science, Technology, Engineering, and Math (STEM), medicine, law, business, and teaching careers. The popularity trend of these careers and the major impacts of gender, socioeconomic status, and academic ability were examined with multilevel logistic regression models. The findings indicated that developed economies such as Singapore, Canada, the United States of America, and the United Kingdom have successfully attracted a greater proportion of students to elite careers. In contrast, many high-performing Asian economies, such as Korea, Japan, and Taipei, have not. STEM and medical fields primarily drew students with high math abilities, whereas legal professions attracted those with superior reading skills. Although girls generally expected teaching and legal careers and boys expected STEM fields, social and gender differences have narrowed over the past decades. Many Western developed economies effectively attract a larger share of adolescents to STEM careers than their Asian counterparts. Although gender and social disparities persist, their impact has diminished. Effective human resource planning should be based on each country's unique trends and influencing factors to promote greater equality and inclusion in the workforce.

Emphasizing symbolic capital: its superior influence on the association between family socioeconomic status and adolescent subjective well-being uncovered by a large-scale multivariate network analysis.

Family socioeconomic status (FSES) serves as a significant determinant for subjective well-being. However, extant research has provided conflicting evidence on the correlation between FSES and adolescent students' subjective well-being (SSWB). Data were collected from 12,058 adolescent students (16 years of age) by the Programme for International Student Assessment (PISA) 2018. Multivariate canonical correlation and Mantel test were utilized to investigate the specific connection between FSES and SSWB. Furthermore, a Gaussian EBICglasso graph-theoretical model was used to capture the topological properties of the FSES-SSWB network and reveal the interplay among multifarious components of FSES and SSWB. FSES was positively correlated with SSWB. In the FSES-SSWB network, parental educational attainment and occupation status demonstrated the highest centrality values, thereby contributing significantly to the relationship between FSES and SSWB. However, family wealth, along with educational and cultural resources, displayed lower centrality values, signifying their weaker roles in this relationship. Our findings suggest that symbolic capital, rather than family affluence, exerts a dominant influence on adolescent SSWB. In other words, SSWB may not be detrimentally influenced by a deficiency in monetary resources. However, it is more susceptible to being unfavorably impacted by inferior parental educational attainment and occupational standing.

Identifying the Careless Responders: A Cross-Country Comparison on PISA 2018 Dataset

Careless responding (CR) is one of the major concerns in self-report measures since it affects the quality of collected data. In this research, it was aimed to examine the dataset in terms of CR, to investigate the effects of CR on the psychometric properties of data, and to examine the effectiveness of CR detection methods in a cross-cultural context. Specifically; response time, outlier and long-string indices were used to flag CR and efficiency of using these were compared on PISA 2018 reading related attitude scales for Singapore and Turkey. As a result, it was indicated that the amount of CR was higher for Singapore than Turkey at long-string and response time methods. Furthermore, after removal of CR from dataset, enjoyment level of reading, perception of competence and perception of difficulty of PISA tests scores increased. Another result was that long-string and response time analyses provided significant improvement in model fit and long-string had the highest improvement level. Moreover, removing respondents according to outlier analyses didn’t provide any improvement on model-data fit. In general, results illustrated that careless responding behavior may have important effects on psychometric properties of self-report and screening the data for CR is strongly suggested in future studies.

Where nonresponse is at its loudest: Cross-country and individual differences in item nonresponse across the PISA 2018 student questionnaire

The Programme for International Student Assessment (PISA) student questionnaire, despite being designed for low cognitive demand, may induce test burden due to its 306-item length, resulting in increased item nonresponse toward the questionnaire’s end. Using the PISA 2018 response data from 80 countries and a cross-classified mixed effects model, this study explored the average, cross-country, and individual nonresponse trends throughout the questionnaire. The average nonresponse probability increased from 2% at the questionnaire’s start to 11% at its end. East Asian countries displayed the flattest nonresponse trends, while South American countries showed the steepest. Individual differences were also prominent: students with a lower initial nonresponse propensity experienced a steeper increase in their nonresponse propensity as they progressed through the questionnaire, whereas those with a higher initial propensity remained at a relatively constant level. Our findings highlight the importance of the questionnaire design, the context in which it is administered, and its interaction with individual and cultural factors in shaping student response behavior. We discuss the implications for questionnaire design and researchers using PISA student questionnaire data for secondary analysis.

An Analysis of DIF and Sources of DIF in Achievement Motivation Items Using Anchoring Vignettes

ABSTRACT This study evaluated differential item functioning (DIF) in achievement motivation items before and after using anchoring vignettes as a statistical tool to account for group differences in response styles across gender and ethnicity. We applied the nonparametric scoring of the vignettes to motivation items from the 2015 Programme for International Student Assessment (PISA) and examined changes to DIF characteristics between the raw self-report item scores and the vignette-adjusted item scores. Overall, applying the anchoring vignettes changed DIF classification, magnitude, direction, and form for some items. Group-specific response patterns by gender and by ethnicity group were also seen. Our findings contribute to the research literature on observed response styles in PISA motivation items, DIF in PISA items, and changes to DIF characteristics after adjusting self-report item scores with anchoring vignettes.

Science career expectation and science-related motivation: a latent profile analysis using PISA 2015 data

ABSTRACT In response to growing concerns about students’ low aspiration for science-related careers, more understanding on how student science motivation is related to science career aspiration is needed. This study utilised a person-centred approach to investigating the association between students’ science-related motivation profiles and their science career expectation using the 2015 Programme for International Student Assessment (PISA) data set. Analysing responses from 9841 15-year-old students in China through Latent Profile Analysis (LPA), the study found that: (1) five distinct students’ motivation profiles were identified based on science-related motivation factors; (2) significant variations were observed in science career expectations across students’ motivation profiles, and different profiles predicted science career expectations differently; and (3) the perceived instructional practices and science learning engagement, social and economic status, and students’ gender significantly predicted students’ motivation profiles differently. These research findings contribute to new insight on the non-linear relationships between science-related motivation and science career expectation, and how student motivation profiles are impacted by demographic and context factors such as instructional practices, learning engagement, students’ social and economic status, and gender.

Socioeconomic factors and performance on the PISA test

Socioeconomic factors and performance on the PISA test

TIGA-Based English Learning Platform for Language Learner Competency Development in a Digital Environment

This study aimed to develop and implement a digital learning platform (DLP) for English language learners at the junior high school level in Thailand. The study emphasizes the necessity of customizing digital tools to individual needs and behaviors for effective implementation. The successive approximation model (SAM) (Herrholtz, 2020) was used for instructional design, focusing on rapid prototyping and continuous feedback. The DLP was meticulously designed to align with the Basic Education Core Curriculum 2008, the Common European Framework of Reference for Languages (CEFR), and reading literacy under the Programme for International Student Assessment (PISA). The architecture of the platform prioritized open standards and web services to ensure seamless integration and data exchange, although integration challenges were noted. Content and activities were acquired from the TIGA English teaching model, fostering various language skills through tasks (T), language input (I), genre (G), and authentic assessments (A). The TIGA-based DLP was implemented in 8 schools involving 7 teachers and 212 students. Questionnaires and group discussions were used to obtain participant feedback. The questionnaire data were analyzed quantitatively using means and standard deviations. The discussion data were analyzed qualitatively to identify themes. Positive feedback and challenges were revealed. Identified limitations include the need for additional learning activities, clearer teacher guidelines, and enhancements to the graphical user interface (GUI). These findings highlight the potential of the DLP to enrich English language learning and guide future enhancements in digital learning environments.

GAME EDUKASI AUGMENTED REALITY BERBASIS RME (REALISTIC MATHEMATICS EDUCATION) UNTUK MENINGKATKAN COMPUTATIONAL THINKING SISWA

This research addresses the common problem in mathematics education in Indonesia, which is the students' difficulties, as reflected in Indonesia's declining ranking in the Program for International Student Assessment (PISA). The research aims to create an innovative Augmented Reality (AR) Educational Game based on the Realistic Mathematics Education (RME) approach to improve students' understanding of the spatial geometry mathematics subject at the elementary school level. AR technology allows students to interact with mathematical objects in real-world contexts using Android devices. This game is designed to make mathematics more engaging and relevant to elementary school students. The research was conducted at Purwantoro 1 Public Elementary School in Malang City over four months, following the Game Development Lifecycle (GDLC). The test results showed an n-gain improvement of 0.71, indicating a positive impact of the game's development on students' learning outcomes.

The development of PISA-like mathematics problem using Lombok Island contexts

The results of the PISA test for mathematical literacy in 2000–2018 and several previous studies show that students' critical thinking abilities in Indonesia are still relatively low. One of the reasons why students' critical thinking skills in Indonesia are low is because PISA-like problems have not been integrated evenly into mathematics learning. The aim of this research is to obtain a prototype of PISA-like mathematics problems in the context of Lombok Island that are oriented towards critical thinking skills that are valid, according to experts. This research is design research, which consists of 3 stages, namely analysis, design, and evaluation. The PISA-like problem that has been developed consists of twelve questions about space and shape, uncertainty, and data. Each set of questions consists of levels 1 to 6. Lombok Island contexts contained in the questions are Mount Rinjani, woven cloth with Subahnale and Alang motifs, the game of Dengkleng (Engklek), Gendang Beleq, and the Limbungan traditional house. At the evaluation stage, the validity of the development product is assessed by experts. The instrument used is a PISA-like problem validation sheet. The results of this research obtained a prototype of a PISA-like mathematics problem with a valid category, according to experts.

MENGKAJI KEUNGGULAN KURIKULUM IPA FINLANDIA UNTUK MENINGKATKAN PENDIDIKAN SAINS DI INDONESIA

<p><em>The issues in education in Indonesia, particularly in the field of Natural Sciences, are very concerning. According to the Programme for International Student Assessment (PISA) survey by the OECD, Indonesian students' abilities in science rank low globally, with a ranking of 71 out of 77 countries in 2018 and 68 out of 81 countries in 2023. The science scores also declined from 396 to 383, far below the RPJMN target of 402. Data from the Trends in International Mathematics and Science Study (TIMSS) shows that Indonesian 4th and 8th-grade students also rank low internationally. The research methodology uses a qualitative approach through a literature study. Data analysis indicates that the content and competency-based science curriculum in Finland provides a good model by integrating theoretical knowledge and practical skills. Science education in Finland, which is interactive and student-centered, can serve as a reference for improving the curriculum in Indonesia. A comprehensive and context-based science learning approach is needed to enhance the quality of science education in Indonesia. The PISA and TIMSS survey results emphasize the need for improvements in science learning in Indonesia. Science, as systematic knowledge and a scientific process, requires a holistic and integrated learning approach. An integrated science approach that connects science to everyday life is crucial for developing scientific processes in students. Science teachers are expected to have interdisciplinary abilities and knowledge integration. However, often the curriculum and the quality of teachers do not support this goal. This study compares the 2013 Curriculum in Indonesia with the Finnish Curriculum to see the similarities, differences, and obstacles in science learning</em>.</p>

Collaborative Learning Approach (CLA) in Improving Reading Proficiency

Studies show that Filipino learners are not demonstrating proficiency in reading. In the recent Program for International Student Assessment (PISA) 2023 results, Filipino students were found to be poor at reading proficiency. One potential strategy is collaborative learning. Thus, this study aimed to determine the effect of the Collaborative Learning Approach (CLA) on the reading proficiency of seventh-grade students. This study used a quasi-experimental design with two experimental and control groups. Using purposive sampling, two grade 7 sections of the research site were included in the study. There were 26 students in one section who were chosen to be part of the experimental group and 24 in the other section, constituting the control group, for 50 students. Each group was given a pretest and a post-test. Students in the experimental group employed the collaborative reading method, while the control group used the traditional method. The questionnaire was validated by experts and deemed reliable, scoring .70 on the Kuder-Richardson test. Mean scores and independent t-tests were used to analyze the data. Results revealed a significant difference between the experimental and control groups regarding reading proficiency after the intervention of CLA with a p-value of 0.000. It was concluded that CLA significantly influences reading proficiency and can be recommended for use in the classroom. However, based on the PHIL-IRI criteria for reading proficiency, the students in the experimental group did not meet the minimum instructional or proficiency level requirements. It is therefore suggested that further research must be done to make conclusive evidence that CLA significantly influences reading proficiency and evaluate PHIL-IRI criteria for reading proficiency, considering that it is inconsistent with the independent t-test results showing a significant difference in the reading proficiency of the experimental group.

The influence of age at first regular digital device access on digital reading performance: the mediating effect of cognitive flexibility

Younger generations are receiving their first regular digital device access at progressively earlier ages than previous generations on average, whereas disparities in their early digital device access are still being observed globally. The inequality of first regular digital device access may have profound long-term effects on one’s information and communication technology (ICT) skills, but no evidence has suggested how unequal first regular digital device access impacts one’s digital reading, which enables one to process and comprehend information in this digitized world. Thus, the current study investigated the relationship between the age at which children first regularly access digital devices and their digital reading performance at age 15 while considering the potential mediating effect of cognitive flexibility. A multilevel mediation model was proposed and tested using data from a cross-national sample of 156,277 15-year-old students from 18 OECD countries from the Programme for International Student Assessment (PISA). The results suggested that first regular digital device access after age 9 exerted a negative influence on digital reading scores in countries that generally had high levels of early digital access, and after age 13 in countries that had lower levels of early digital access. Additionally, cognitive flexibility mediated the association between the age at first regularly accessing digital devices and digital reading performance. Starting using digital devices regularly only after age 6, more evident at ages 9 to 12, was associated with lower levels of cognitive flexibility, which in turn led to worse digital reading performance. These findings demonstrated the importance of early first regular access to digital devices before school age (age 6) and highlighted the need for the cultivation of cognitive flexibility in family and school uses of digital devices.

Overview of the STEM Education in Israel

In this study, we provide an overview of the STEM education in Israel, the formal and the informal, the strengths and challenges, the problems, and the national plans to solve them. STEM education in Israel is an interesting phenomenon, as the State of Israel is considered a start-up nation, even though students have scored mediocre on the PISA tests over the years compared to the OECD average. The paper is based on literature review of position papers written for decision makers in the Israeli education system and for the purpose of comparison with other advanced education systems in the world.

Factors Associated with STEM Career Expectations of Greek 15-Year-Old Students

AbstractScience, technology engineering, and mathematics (STEM) professions offer competitive earnings, tend to be compatible with technological advances, and are predicted to remain in high demand in the future. Many countries, including Greece, have prioritised STEM education on their educational agendas, aiming to prepare students for careers in these fields. In this study, we investigated factors that may predict Greek students’ STEM career expectations through analysing data from the 2015 cycle of the Programme for International Student Assessment (PISA). In doing so, we categorised the responses of 4910 15-year-old students to the question “What kind of job do you expect to have when you are about 30 years old?” into STEM and non-STEM professions, following a wide approach that included health and medical professions in the STEM category and a narrow approach that excluded these professions from the STEM category. Binary logistic regression results showed that student enjoyment of science, instrumental motivation, interest towards broad science topics, and their science performance were associated with their career expectations across both approaches; family economic, social, and cultural status was associated with students’ career expectations within the wide approach only, while gender and achievement motivation were associated with students’ career expectations within the narrow approach only. The differences in the factors predicting students’ career expectations between the two approaches indicate that transparency in the definitions and operationalisations of STEM is needed within the relevant research.

Beyond group comparisons: Accounting for intersectional sources of bias in international survey measures

International surveys are increasingly being used to understand nonacademic outcomes like math and science motivation, and to inform education policy changes within countries. Such instruments assume that the measure works consistently across countries, ethnicities, and languages—that is, they assume measurement invariance. While studies have already demonstrated that some items in international survey measures are noninvariant using basic group comparisons, they do not investigate complex, intersectional sources of bias that go beyond group membership. In this study, we use an emergent method to examine the sensitivity of item parameters from the Programme for International Student Assessment (PISA) survey instruments to intersectional sources of bias. Results indicate that non-invariance exists for most of the items examined, which can change individual scores after accounting for the moderators. Although country-level ranking did not change substantively after accounting for bias, policymaking is likely to be influenced negatively if such sources of non-invariance are not addressed.

The healthy context paradox: a cross-country analysis of the association between bullying victimisation and adolescent mental health.

Bullying victimisation is an increasing global health problem among adolescents and is associated with short- and long-term adverse mental health outcomes. Investigating whether associations with mental health vary across national contexts and why, can provide insights into mechanisms underlying those associations and inform policy. We used data from 479,685 adolescents participating in the 2018 Program for International Student Assessment (PISA) cross-sectional survey and examined whether the associations between bullying victimisation, psychological distress and life satisfaction vary across 63 countries. We further tested the modifying role of country-level factors - bullying prevalence, income inequality and national wealth, by implementing multilevel cross-country analyses. We found significant associations between bullying victimisation, increased psychological distress (β = 0.181; 95%CI: 0.178, 0.184) and decreased life satisfaction (β = -0.158; 95%CI: -0.162, -0.155). Associations between bullying victimisation, psychological distress and life satisfaction among adolescents were consistent across countries in terms of direction but effect sizes varied substantially. The effects ranged from β = 0.08 in the Philippines to β = 0.40 in South Korea for psychological distress and from β = -0.05 in the Philippines to β = -0.36 in the United Kingdom for life satisfaction. In addition, consistent with the "healthy context paradox" effect, associations between bullying and mental health were larger in countries where the prevalence of bullying was lower, as well as in higher-income countries. Interventions aiming to reduce bullying victimisation should aim to provide additional targeted support for those who still experience bullying after the intervention.

Hubungan Keterampilan Literasi Awal dengan Kemampuan Bahasa Ekspresif Tingkat Kata pada Anak Usia Prasekolah di RA Sunan Ampel Al-Jauhar Ngawi

Abstract Background: The education of children of preschool age is closely related to the development of children's literacy skills. The lack of readiness of early literacy skills will affect the child's success in communicating and speaking with their environment. Based on a survey conducted by the Program for International Student Assessment (PISA) released by the Organization for Economic Co-operation and Development (OECD) in 2019. Indonesia ranks 62nd out of 70 countries related to literacy rate, or is in the bottom 10 countries that have low literacy levels. Objectives: analyze the Corellation of Early Literacy Skills with Word-Level Expressive Language Skills in Preschool-Age Children at RA Sunan Ampel Al-Jauhar Ngawi. Methods: This study was a quantitative study using an observational analytic research design with a cross sectional approach. The sample size of this study was 32 respondents aged 3–6 years (preschool). The statistical test used in this study was the Kendall Tau test. Results: The Kendall's tau test results revealed that a significance value of 0.000 was obtained with corellation coefficient or a r value of 0.603 which means that the correlation value is strong and positive, so Alternative hypotheses is accepted. This means that there is a strong relationship between early literacy skills and word-level expressive language skills. The better the early literacy skills, the better the expressive language skills of children is, and vice versa. Conclusion: There is a strong and positive correlation between early literacy skills and expressive language skills in preschool children. Keywords: Early literacy skills, expressive language skills, preschoolers

Optimizing Large-Scale Educational Assessment with a "Divide-and-Conquer" Strategy: Fast and Efficient Distributed Bayesian Inference in IRT Models.

With the growing attention on large-scale educational testing and assessment, the ability to process substantial volumes of response data becomes crucial. Current estimation methods within item response theory (IRT), despite their high precision, often pose considerable computational burdens with large-scale data, leading to reduced computational speed. This study introduces a novel "divide- and-conquer" parallel algorithm built on the Wasserstein posterior approximation concept, aiming to enhance computational speed while maintaining accurate parameter estimation. This algorithm enables drawing parameters from segmented data subsets in parallel, followed by an amalgamation of these parameters via Wasserstein posterior approximation. Theoretical support for the algorithm is established through asymptotic optimality under certain regularity assumptions. Practical validation is demonstrated using real-world data from the Programme for International Student Assessment. Ultimately, this research proposes a transformative approach to managing educational big data, offering a scalable, efficient, and precise alternative that promises to redefine traditional practices in educational assessments.

Low PISA Performance Students: Factors, Perceptions, and Improvement Strategies

The PISA assessment serves as a critical tool for evaluating and improving educational systems worldwide. The ethical implications of low performance in PISA are significant, requiring a comprehensive approach to address both immediate educational needs and broader systemic issues. By leveraging insights from PISA and implementing evidence-based interventions, policymakers and educators can work towards creating a more equitable and effective educational environment for all students. This study investigates the underlying factors contributing to the low achievement of Jordanian students in the Program for International Student Assessment (PISA) tests and proposes effective strategies to enhance their performance. PISA, developed by the Organization for Economic Co-operation and Development (OECD), evaluates 15-year-old students' skills in reading, mathematics, and science, focusing on their ability to apply knowledge in real-world contexts. Despite its global significance, Jordanian students have consistently underperformed in these assessments. The study employs a mixed-methods approach, including a comprehensive literature review, analysis of the Jordanian educational context, and surveys with educators and students. The findings reveal several key factors impacting performance: curriculum misalignment with PISA competencies, socio-economic disparities, high teacher turnover, and varying levels of teaching quality. Additionally, perceptions of PISA among educators and students highlight challenges related to unfamiliar test formats and applied knowledge requirements. Based on these insights, the study recommends a series of interventions to improve performance. These include aligning the national curriculum with PISA competencies, enhancing teacher training programs, integrating technology into learning processes, promoting extracurricular activities, and fostering active parental involvement. By addressing these areas, the study aims to provide a roadmap for policymakers, educators, and stakeholders to enhance Jordanian students' performance in future PISA assessments and overall educational outcomes.