Trends In International Mathematics And Science Study Research Articles

Writing PISA-Like Mathematics Items: The Case of Tertiary Mathematics Instructors from a State University in the Philippines

<p style="text-align:justify">Mathematics test items in International Large-Scale Assessments (ILSAs) such as the Programme of International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS) are nested in contexts defined in their assessment framework (e.g., the Personal, Occupational, Societal, and Scientific contexts in PISA). This study followed the item-writing activities of four tertiary mathematics instructors in the Philippines as they constructed context-based mathematics items. They were tasked to write four items each, following a set of specifications for PISA content and context categories. The data consisted of transcripts from the focus-group discussion which was conducted days after the task. The transcripts were then analyzed using thematic analysis. The results of this study showed that the phenomenon of item-writing in the context of writing PISA-like mathematics items had two themes: the phases of item-writing and the dimensions of item-writing. Findings showed that the respondents struggled to find realistic contexts and that they engaged in a problem-solving task likened to solving a puzzle as they attempted to satisfy the content, context, and process categories in the table of specifications (TOS). This study contributes to filling in the research gap on item-writing activities, particularly those of mathematics teachers in the Philippines- a country whose recent mathematical performance in the PISA 2018, TIMSS 2019, and PISA 2022 was nothing short of dismal.</p>

Pre-service Teachers’ Preparedness for In-service Science Teaching in Primary Education – A Case Study in Croatia

An experimental science curriculum for primary school was introduced in Croatia for the 1st time in the 2023/24 school year. This is taking place against the backdrop of a shortage of teachers from various science subjects, especially physics. They are increasingly being replaced by non-professional teachers, often those with a Master of Primary Education, which gives access to the profession of primary junior grade teacher. In this study, the subject matter knowledge (SMK) in science of 44 pre-service teachers aiming for the above-mentioned degree is examined using the SMK test. The test contains 40 trends in international mathematics and science study (TIMSS) items from the 4th-grade science content domain Physical Science and allowed us to compare the achievement of our respondents with the achievement of pupils who participated in the TIMSS assessments. The results show that the percentage correctness of the pre-service teachers on the SMK test was statistically significantly higher than that of the 4th-grade pupils. However, the correlation coefficients showed that the pre-service teachers had greater difficulties with the items with which the pupils had greater difficulties, and vice versa. This applies in particular to the items on forces and motion and to the items from the cognitive domain of reasoning. The reason for the unsatisfactory science SMK of our respondents could be the incompatibility of the university science curriculum with the demands placed on pupils in the relevant school subjects and International science assessments such as TIMSS. The results of this study confirm and extend the contribution of existing research and should serve as a basis for subsequent research as well as for the design of a further course of science education reform in Croatia and in the countries of the region.

Investigating Factors that Predict Japanese Science Teachers’ Job Satisfaction: Evidence from TIMSS 2019

Trends in International Mathematics and Science Study (TIMSS) 2019 data revealed that Grade 8 science teachers in several Asian countries (Chinese Taipei, Hong Kong SAR, Japan, and South Korea) reported lower job satisfaction levels than the international average. For example, while Japanese Grade 8 students ranked among the highest in science achievement on the TIMSS 2019, their science teachers reported the lowest job satisfaction composite scale score compared to the other 63 participating countries. Using teacher questionnaire data from TIMSS 2019, we utilized correlation analysis and multiple linear regression to uncover the answers to this research question: Which factors contribute to Japanese Grade 8 science teachers’ job satisfaction? The results from the analysis indicate that working in a safe and orderly school environment was the strongest predictor of Japanese Grade 8 science teacher job satisfaction. The number of working hours, amount of perceived pressure exerted by their students’ parents, participation in professional development focused on science curriculum, and adopting a constructivist orientation to science teaching were the other significant factors that predicted job satisfaction. Findings from this study have implications for Japanese educational policy stakeholders, school administrators, mentor teachers, science department leadership, and those invested in science teachers’ professional development.

Trends in Malaysian Adolescents’ Mathematics Performance Across Two Decades: What Factors Matter?

The relationship between individual and school factors plays a crucial role in shaping adolescents' mathematics achievement. Individual factors, such as a student's motivation and educational expectation, directly influence their engagement and persistence in mathematical learning. Concurrently, school factors, including teacher quality and the availability of resources, create an environment that either fosters or hinders students' academic success in mathematics. Together, these factors underscore the need for a holistic approach to education that recognizes the interconnectedness of personal and institutional influences on student performance. Thus, this study investigates what individual and school factors have been associated with adolescents’ mathematics achievement over the last two decades. We examined six-wave data of Malaysia from the Trends in International Mathematics and Science Study (TIMSS). Multilevel modeling technique was employed for data composed of 5557, 5314, 4466, 5733, 9726, and 7065 eighth-grade students participating in 1999, 2003, 2007, 2011, 2015, and 2019, respectively. The results showed that 65.17%, 57.44%, 65.89%, 73.00%, 43.16%, and 61.69% of the total variance in mathematics achievement was due to school differences in these years. The findings also indicated that students with higher mathematics motivational beliefs scored higher. Students who more frequently spoke the language of the test at home achieved lower scores in the four waves of TIMSS; however, this trend changed in 2015 and 2019. Student educational aspiration emerged as a positive predictor of mathematics achievement in the last two assessments of TIMSS. Home educational resources and book ownership also significantly correlated with mathematics achievement. At the school level, teacher gender was negatively associated with achievement in all TIMSS waves except 2019. Students whom female teachers taught outperformed those whom male teachers taught. The longitudinal study of Malaysian students' mathematics performance reveals critical insights for educational stakeholders. The findings indicate that 43.16% to 73.00% of the variance in mathematical achievement is linked to school differences, underscoring the importance of equitable resource distribution. Moreover, students' motivational beliefs significantly influence their mathematics scores, suggesting a need for interventions that foster positive attitudes. The evolving correlation between home language proficiency and achievement emphasizes the necessity of integrating language support into educational strategies. Additionally, promoting educational aspirations is vital for enhancing performance, while the role of gender dynamics in teacher effectiveness warrants further exploration.

Worldwide predictors of science literacy in lower-secondary students: a TIMSS 2019 analysis

ABSTRACT In today's technologically advanced and environmentally challenging landscape, enhancing global scientific literacy is increasingly imperative. A solid grasp of scientific principles is vital for fostering innovation and achieving sustainable development worldwide. Utilizing data from the International Association for the Evaluation of Educational Achievement (IEA) Trends in International Mathematics and Science Study (TIMSS), this study inquiries into various factors impacting grade eight students' science literacy in the TIMSS 2019 assessment. Exploring student socioeconomic status, cultural background, attitudes towards science, school organisation, teacher experiences, and country-level variables, we aimed to identify predictive associations with science literacy. Additionally, we correlated TIMSS 2019 results with the United Nations Development Program's Human Development Index (HDI). Employing three-level Hierarchical Linear Modeling, we assessed which student, teacher, and country-level variables were linked to grade 8 science literacy. Results indicate that the students' socioeconomic resources and motivation/confidence in science positively influence science literacy. Conversely, a higher percentage of science items covered in national curricula was weakly associated with lower science literacy scores. Teachers reporting greater teaching limitations due to student needs showed negative correlations with science scores. At the country level, the HDI emerged as the strongest correlate with grade 8 science literacy. These findings underscore the multifaceted nature of science literacy and its interplay with various socio-cultural and educational factors on a global scale.

Teacher-centered analysis with TIMSS and PIRLS data: weighting approaches, accuracy, and precision

This paper extends existing work on teacher weighting in student-centered surveys by looking into aspects of practical implementation of deriving and using weights for teacher-centered analysis in the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS). The formal conditions to compute teacher-centered weights are detailed, including mathematical equations. We provide a proposal on how to define the targeted populations as well as how to collect data that is needed to derive teacher-centered weights, yet currently unavailable. We also tackle the issue of teacher nonresponse by proposing a respective adjustment factor, as well as mentioning the challenge of multiple selection probabilities when teachers teach in multiple schools. The core part of the paper focuses on studying the level of accuracy that can be expected when estimating teacher population characteristics. We use TIMSS 2019 data and simulate likely scenarios regarding the variance in weights. The results show that (i) the different weighting scenarios lead to relatively similar estimates; however, the differences between the scenarios are sufficient to justify the recommendation to use correctly derived teacher weights; (ii) differences between estimated standard errors based on complex sampling and corresponding estimates based on simple random sampling are sufficiently consistent to support use of a procedure to estimate standard errors that accounts for both sample weights and the complex sampling design; (iii) sample sizes and variance in weights significantly limit estimate precision, so that total population estimates with sufficient precision are available in the majority of countries but subpopulation features are generally not sufficiently precise. To provide a critical evaluation of our results, we recommend implementation of the proposed method in one or more countries. This recommended study will permit examination of logistical considerations in implementation of required changes in data acquisition and will provide data to replicate the analysis with teacher-centered weights.

The cloak of equality in STEM education: institutionalised masking of gender differences in STEM expectations, task values, and participation

ABSTRACT Do gender parity and egalitarian values in STEM education mask institutionalised gender inequalities in STEM? This study uses student background questionnaire data from the 2015 Trends in International Mathematics and Science Study (TIMSS) supplemented by national level data on gender parity in educational enrolment and female participation in the STEM workforce from the United Nations and International Labor Organization to empirically examine the evidence. Using two-level hierarchical linear models, we analyse the effects of students’ STEM expectancies and task values on their anticipated STEM education and career participation. Results confirm that gender equity and egalitarian values dominate the cross-national expectancies and task values for STEM education, both across genders as well as for female and male students separately, even though there is wide variation and significant gendered inequality that persists in both educational access and STEM workforce participation. Thus, the ‘cloak of equality’ hypothesis is supported by the evidence presented.

The impact of early childhood education and mathematical abilities on student achievement: Analysis of TIMSS 2019

This paper is based on the analysis of the relationship between early education, mathematical skills, and student achievement in Bosnia & Herzegovina, using data from the trends in international mathematics and science study (TIMSS) 2019. The study involves 5,628 fourth-grade students whose average age was 10.1 years. The research specifically focuses on factors such as the age of students at the beginning of schooling, attendance at preschool institutions, and their mathematical skills before starting school. Through the application of SPSS analysis, insights into the statistical relationships and trends between these factors are provided, emphasizing the importance of early educational experiences to improve the quality of education in Bosnia & Herzegovina. The analysis results show that the length of stay in preschool institutions, attendance in preschool programs, and the early acquisition of mathematical skills (counting skills, recognition of numeral representations, numeral writing, simple addition, and subtraction) impact later mathematical achievements of students.

MENGKAJI KEUNGGULAN KURIKULUM IPA FINLANDIA UNTUK MENINGKATKAN PENDIDIKAN SAINS DI INDONESIA

<p><em>The issues in education in Indonesia, particularly in the field of Natural Sciences, are very concerning. According to the Programme for International Student Assessment (PISA) survey by the OECD, Indonesian students' abilities in science rank low globally, with a ranking of 71 out of 77 countries in 2018 and 68 out of 81 countries in 2023. The science scores also declined from 396 to 383, far below the RPJMN target of 402. Data from the Trends in International Mathematics and Science Study (TIMSS) shows that Indonesian 4th and 8th-grade students also rank low internationally. The research methodology uses a qualitative approach through a literature study. Data analysis indicates that the content and competency-based science curriculum in Finland provides a good model by integrating theoretical knowledge and practical skills. Science education in Finland, which is interactive and student-centered, can serve as a reference for improving the curriculum in Indonesia. A comprehensive and context-based science learning approach is needed to enhance the quality of science education in Indonesia. The PISA and TIMSS survey results emphasize the need for improvements in science learning in Indonesia. Science, as systematic knowledge and a scientific process, requires a holistic and integrated learning approach. An integrated science approach that connects science to everyday life is crucial for developing scientific processes in students. Science teachers are expected to have interdisciplinary abilities and knowledge integration. However, often the curriculum and the quality of teachers do not support this goal. This study compares the 2013 Curriculum in Indonesia with the Finnish Curriculum to see the similarities, differences, and obstacles in science learning</em>.</p>

Teachers’ and principals’ perceptions of school emphasis on academic success: measurement invariance, agreement, and relations to student achievement

A school’s emphasis on academic success (SEAS) is a crucial aspect of school climate that can be rated by different informants, such as teachers or principals, and with respect to their perception of different groups’ emphasis on academic success, such as parents or students. We know little about whether there is agreement between these ratings. Utilizing Israeli Trends in International Mathematics and Science Study (TIMSS) 2019 data and multilevel structural equation modeling, we examined how teachers and principals rate teachers’, students’, and parents’ emphasis on academic success and how these ratings are related to student achievement. We found that (a) ratings are invariant across informants; (b) teachers and principals agree on parents’ and students’ yet not teachers’ emphasis; (c) teacher ratings explain more variation in student achievement than principal ratings. Since SEAS ratings diverge by informants and relations to student achievement, researchers need to craft validity arguments in their studies.

Using machine learning to predict student science achievement based on science curriculum type in TIMSS 2019

ABSTRACT Most educational systems use either an integrated or a separated science curriculum. However, it is unclear which of these science curricula benefits students more author and existing research provides insufficient information about the implementation details of the curriculum employed. Therefore, this study compares the effects of two science curricula on students’ science literacy, drawing on socio-ecological theory and employing educational data mining techniques. Results from Grade 8 Science students in 44 countries sampled in the Trends in International Mathematics and Science Study (TIMSS) 2019 showed that (1) the integrated curricula benefitted students marginally more than the separated curricula; (2) curriculum type was not essential in directly predicting students’ academic performance; and (3) random forest outperformed linear regression, lasso regression, decision trees, and neural networks in predicting student science achievement. This study advances our understanding of the predictors of student science performance, demonstrates that machine learning techniques can be applied successfully to examine curriculum effects, and provides directions for implementing integrated science curricula.

Exploring The Relationship Between Private Tutoring And Mathematics Achievement Among Qatari Primary School Students

This study aims to shed light on the association between private tutoring and academic achievement in mathematics among primary school students in Qatar. Utilizing data from the Trends in International Mathematics and Science Study (TIMSS) 2019, we employ a propensity score technique to elucidate the disparities in mathematics achievement levels between two distinct groups: students who receive private tutoring and those who do not. Our findings indicate that students who engage in private tutoring have lower odds of attaining higher achievement scores in mathematics. .

Bayesian causal forests for multivariate outcomes: application to Irish data from an international large scale education assessment

Abstract Bayesian Causal Forests (BCF) is a causal inference machine learning model based on the flexible non-parametric regression and classification tool, Bayesian Additive Regression Trees (BART). Motivated by data from the Trends in International Mathematics and Science Study (TIMSS), which includes data on student achievement in both mathematics and science, we present a multivariate extension of the BCF algorithm. With the help of simulation studies, we show that our approach can accurately estimate causal effects for multiple outcomes subject to the same treatment. We apply our model to Irish data from TIMSS 2019. Our findings reveal the positive effects of having access to a study desk at home (Mathematics ATE 95% CI: [−0.50, 10.14]) while also highlighting the negative consequences of students often feeling hungry at school (Mathematics ATE 95% CI: [−8.86, −1.56] , Science ATE 95% CI: [−10.35, −0.94]) or often being absent (Mathematics ATE 95% CI: [−11.88, −2.27]). Code for replicating the results can be found at https://github.com/Nathan-McJames/MVBCF-Paper.

Year Five Pupils’ Perception Towards Higher Order Thinking Skills in Learning Mathematics

Higher-order thinking skills (HOTS) are the cognitive skills that mathematics should apply to fulfil current needs. Unfortunately, Mathematics achievement in Trends in International Mathematics and Science Study (TIMSS) has shown that the thinking ability of Malaysian pupils still needs improvement. Identifying pupils' perceptions of implementing HOTS in mathematics is essential. Thus, this study was conducted to determine Year Five pupils' perception towards implementing HOTS in terms of attitude, readiness and belief in Mathematics learning, as well as the differences in these three aspects based on gender. The survey study involved 142 Year Five pupils at a Sekolah Jenis Kebangsaan Cina (SJKC) in Johor Bahru district. The instrument used in this study is a questionnaire. The data was analysed using Statistical Package for Social Sciences (SPSS) software version 27. The descriptive statistical analysis results found that pupils’ perception towards HOTS in Mathematics in terms of Attitude and readiness are moderate. At the same time, pupils' perception towards HOTS in mathematics in terms of belief is high. The Mann-Whitney U test results found a significant difference in pupils’ attitudes towards HOTS in Mathematics based on gender. However, this study showed no significant difference in student’s readiness and belief towards HOTS in Mathematics based on gender. This study brings implications for educators in schools to realise the importance of modifying HOTS teaching practices by considering pupils’ perceptions.

A Psychometric Network Analysis Approach for Detecting Item Wording Effects in Self-report Measures across Subgroups

In this study, we explored psychometric network analysis (PNA) as an alternative method for identifying item wording effects in self-report instruments. We examined the functioning of negatively worded items in the network structures of two math-related scales from the 2019 Trends in International Mathematics and Science Study (TIMSS); Students Like Learning in Mathematics (SLLM); and Students Confident in Mathematics (SCM). We also explored how the negatively worded items functioned in network structures across demographic subgroups. Data were drawn from eight countries that represented diverse levels of math performance and cultural attitudes toward school ( n = 75,972). We found that negatively worded items were distinct from the positively worded items in the SLLM and SCM item networks, and that this effect was consistent across all age- and country-level subgroups. Based on these findings, we recommend PNA as a data-driven approach for detecting wording effects effectively.

Year Five Pupils’ Perception Towards Higher Order Thinking Skills in Learning Mathematics

Higher-order thinking skills (HOTS) are the cognitive skills that mathematics should apply to fulfil current needs. Unfortunately, Mathematics achievement in Trends in International Mathematics and Science Study (TIMSS) has shown that the thinking ability of Malaysian pupils still needs improvement. Identifying pupils' perceptions of implementing HOTS in mathematics is essential. Thus, this study was conducted to determine Year Five pupils' perception towards implementing HOTS in terms of attitude, readiness and belief in Mathematics learning, as well as the differences in these three aspects based on gender. The survey study involved 142 Year Five pupils at a Sekolah Jenis Kebangsaan Cina (SJKC) in Johor Bahru district. The instrument used in this study is a questionnaire. The data was analysed using Statistical Package for Social Sciences (SPSS) software version 27. The descriptive statistical analysis results found that pupils’ perception towards HOTS in Mathematics in terms of Attitude and readiness are moderate. At the same time, pupils' perception towards HOTS in mathematics in terms of belief is high. The Mann-Whitney U test results found a significant difference in pupils’ attitudes towards HOTS in Mathematics based on gender. However, this study showed no significant difference in student’s readiness and belief towards HOTS in Mathematics based on gender. This study brings implications for educators in schools to realise the importance of modifying HOTS teaching practices by considering pupils’ perceptions.

Factors predicting teachers' implementation of inquiry‐based teaching practices: Analysis of South African TIMSS 2019 data from an ecological perspective

AbstractInquiry‐Based Teaching Practice (IBTP) is an essential component of science education, and promoting its implementation is at the heart of various reform efforts. Even though science teachers regard IBTP as an essential pedagogical method, they rarely use it for various reasons. This study utilizes Bronfenbrenner's ecological framework to examine potential factors at various levels of the educational ecosystem that predict the implementation of inquiry‐based teaching practices among Grade 9 science teachers in South Africa. To this end, quantitative data from 537 educators who participated in the Trends in International Mathematics and Science Study (TIMSS) 2019 national assessment were utilized. Data were analyzed using descriptive statistics, Pearson correlation, and hierarchical multiple regression analysis. Results revealed that four variables at different socioeconomic levels were perceived to strongly predict teachers' implementation of inquiry teaching practices. These factors include teachers' job satisfaction, instructional resource shortage, and teachers' perception of the significance of various assessment strategies at the mesosystem level as well as teachers' participation in professional development at the exosystem level. Significant correlations exist between some of the predictive variables. Implications for policy are discussed.

Examining Position Effects on Students’ Ability and Test-Taking Speed in the TIMSS 2019 Problem-Solving and Inquiry Tasks: A Structural Equation Modeling Approach

Position effects occur when changes in item positions on a test impact the test outcomes (e.g., item parameters or test scores). Previous studies found that position effects may vary by the testing context and conditions and thus affect each test-taker differently. With the increasing adoption of digital assessments involving innovative item types that are focused on problem-solving skills, it is also essential to study position effects within this context. This study incorporates item-level scores and screen-level response time data from the Trends in International Mathematics and Science Study (TIMSS) 2019 problem-solving and inquiry tasks for Grade 4 students to examine position effects on students’ ability and test-taking speed. This dataset included responses from 27,682 students from 36 countries. A structural equation modeling approach was employed to model ability and test-taking speed within the same model. The results indicated a small but statistically significant booklet effect on students’ overall mathematics and science ability. The impact of block position changes within the booklets seemed to be greater than the impact of a reordering of subjects tested in the two sessions. The results also showed that when an item block was placed earlier in a test session, students spent more time on the items and performed better. The implications of these findings are discussed.

Re‐examining the relationship between patience, risk‐taking, and human capital investment across countries*

SummaryHanushek et al. (2022) show that students in countries in which people are more patient and less risk‐taking perform better in the Programme for International Student Assessment (PISA) test. In this paper, we probe the robustness of this study. Our narrow replication shows that most of the results are robust to alternative model specifications. Our broad replication shows that the main results are robust to measuring student performance with data from the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) instead of PISA.

Investigating Variables Predicting Environmental Awareness in the Context of Education for Sustainable Development: An Analysis Based on TIMSS 2019 Data

Trends in International Mathematics and Science Study (TIMSS) provides unique and detailed information to explore the competencies and attitudes of fourth and eighth-grade students worldwide. The "Environmental Awareness Scale" was developed for the first time in the TIMSS 2019 cycle to measure students' environmental awareness. This cognitive scale gives essential information on students' scientific understanding of some environmental problems at the national and international levels. Using this scale to assess students' environmental awareness will improve education for sustainable development (ESD) in this setting since the obtained data provide reliable information that education policymakers use to integrate the most important environmental education topics and concepts into their curricula and identify the factors that should be considered to raise students' environmental awareness. This study aims to investigate to what extent the science self-concept variable from science-related factors, as well as the gender and socioeconomic status (SES) variables from demographic characteristics, predict the environmental awareness of eighth-grade students in Türkiye participating in the TIMSS 2019. For this purpose, a theoretical model was developed and analyzed with Structural Equation Modeling (SEM) to investigate the association between students' environmental awareness and science self-concept, gender, and SES. The analysis revealed that science self-concept, SES, and gender variables together explained 36% of students' environmental awareness. Furthermore, while science self-concept appears to be the strongest predictor of students' environmental awareness, SES level was also a significant variable explaining students' environmental awareness. On the other hand, there was no significant difference in environmental awareness between male and female students. The study's findings are discussed based on the United Nations Educational, Scientific and Cultural Organization (UNESCO)'s emphasis on the periodic monitoring and evaluation of progress in implementing ESD based on data.