International Mathematics Study Research Articles

Perspectives on educational effectiveness in science and mathematics: The role of non-cognitive measures in TIMSS. Introduction to a special issue

This special issue focuses on perspectives on educational effectiveness in science and mathematics and addresses the role of non-cognitive measures in Trends in International Mathematics and Science Study (TIMSS). Moving beyond a mere ranking of countries based on their average achievement by making use of the large amount of collected non-cognitive measures, brings many challenges. This special issue aims to address some of these challenges by including contributions with a more conceptual or methodological focus that encompass several non-cognitive scales, international comparative studies or multiple data points over time. The conceptual contributions are clustered around aspects of engagement and motivation, the experience of school bullying and its effects, and quality and equality. The methodological contributions study topics such as measurement invariance, sampling and teacher-centric inferences, and trend scaling. We hope that the papers in this special issue may stimulate and guide future work in the area of TIMSS.

An Examination of Science Achievement and School Compositional Effects in Ireland Using TIMSS Data

<p style="text-align:justify">Recent educational policy initiatives in Ireland have focused on improving outcomes in reading and mathematics among students, particularly those experiencing educational disadvantage. However, science achievement in Irish primary schools has received much less research attention, especially in the context of educational disadvantage. This article examines science achievement and its relationship to school compositional effects in primary schools at the national level, including school-average indicators of the school context, as well as examining factors associated with science achievement in three distinct categories of schools (those with high, moderate, or minor levels of educational disadvantage). The data are drawn from the Fourth grade Trends in International Mathematics and Science Study (TIMSS) 2015 database for Ireland. Multilevel analyses were implemented in a stepwise manner. Findings suggest the relevance of school contexts with regard to science achievement. Before including school-level contextual variables, students from ‘minor disadvantaged’ schools achieved significantly higher science scores than students from schools with ‘moderate’ or ‘high’ levels of disadvantaged. However, this difference disappears after controlling for predictors at the school level. The findings highlight the importance of the home environment, including early numeracy activities and skills before children start school. Results are discussed with regard to educational policy and educational practice in Ireland.</p>

Trend analyses of TIMSS 2015 and 2019: school factors related to declining performance in mathematics

Gaining knowledge of what contextual factors may contribute to changes in student achievement across cycles of international large-scale assessments (ILSA), is important for educational policy and practice. Addressing this necessitates advanced methodology that utilizes the trend design of the ILSAs. The Trends in International Mathematics and Science Study (TIMSS) data is suitable for such analyses as it measures students’ competence based on the participating countries curricula. In Norway, students’ performance in mathematics decreased from 2015 to 2019 as evidenced by TIMSS. During this time-period, there are indications that also school climate, student motivation and self-concept decreased. This study investigates whether school climate (including bullying, a safe environment, school emphasis on academic success, and students’ sense of school belonging and well-being), student motivation (including interest-enjoyment and utility value), and self-concept declined from 2015 to 2019, and whether this possible decline is related to the decline in mathematics performance in Norway. The present study utilized a trend approach with mediation structural equation modelling. The results showed that school climate, the utility value of the subject and students’ self-concept declined during this period, and that a safe environment and student self-concept mediated the changes in achievement from TIMSS 2015 to 2019. Hence, declines in a safe school climate and student self-concept were associated with declining achievements. While the study cannot prove a causal relation, it is discussed whether this could indicate that the decline in these predictors may explain the decline in mathematics achievement. The usefulness of the methodological approach for other countries, as well as the implications of the results for policy, practice, and research are discussed.

An Investigation of the Effect of Missing Data on Differential Item Functioning in Mixed Type Tests

In this research, the aim was to examine the effects of Markov Chain Monte Carlo (MCMC), multiple imputation (MI), and expectation maximization (EM), all methods of coping with missing data in mixed type tests including dichotomous and polytomous items, on the differential item functioning (DIF). The study was carried out on a complete data set consisting of the scores of 1160 students who took booklet number 9 in the science test in Trends in International Mathematics and Science Study (TIMSS) 2019 and answered it in full. The conditions to be examined for the effectiveness of the methods were missing data mechanism (MCAR and MAR), DIF level (A, B, and C), and missing data rate (10% and 20%). Data were assigned to the missing data sets created by deleting data at different rates under the missing completely at random (MCAR) and missing at random (MAR) mechanisms over the aforementioned data set. DIF analysis was performed on all the data sets obtained with the poly-SIBTEST method using the MCMC, MI, and EM methods. The results obtained from the complete data set were then compared with the result implications from other data sets for reference. The study showed that the EM and MCMC methods performed better for the C-level DIF than the A and B levels in terms of all conditions examined. MI was observed to be the most successful method in determining DIF in items showing DIF in 10% and 20% MCAR mechanisms. When compared with the complete data set, the three methods showed similar results in the 10% MAR mechanism while MCMC gave the closest results in the 20% MAR mechanism.

Associations between sleep deficit and academic achievement - triangulation across time and subject domains among students and teachers in TIMSS in Norway

BackgroundSufficient sleep is important to an individual’s health and well-being, but also for school achievement among adolescents. This study investigates the associations between sleepiness, sleep deficits, and school achievements among adolescents.MethodsThis trend study involved a representative sample of Norwegian adolescents based on the “Trends in International Mathematics and Science Study” (TIMSS), N = 4499 (2015) and N = 4685 (2019) and their teachers. The students were 9th graders from a Norwegian compulsory secondary school. The survey included questions on students’ sleepiness as students reported in 2019 and sleep deficits among students that limited teaching in class as their teachers reported in 2015 and 2019. Regression, triangulation, and mediation analyses were used. Mplus was used to perform the statistical analyses.ResultsThe results revealed significant negative associations between sleep deficits and school achievements, adjusted for gender, socioeconomic status (SES), and minority status among Norwegian 9th graders. These results were found for both mathematics and science achievements in 2015 and 2019. Sleepiness that the students reported was negatively associated with school achievements in 2019. Trend and mediation analyses showed that sleep deficits explained 18 and 11% of the decrease in mathematics and science achievements, respectively, from 2015 to 2019.ConclusionsSleep deficits were associated with school achievements in mathematics and science among Norwegian 9th graders. Mediation analyses revealed that sleep deficits explained a significant part of the decline in academic achievements. Insufficient sleep may have negative public health implications and influence adolescents’ academic achievements and competences, and should therefore be discussed in both the educational and health systems.

Factors Influencing Students’ Achievements in the Content and Cognitive Domains in TIMSS 4th Grade Science and Mathematics in the United Arab Emirates

Trends in International Mathematics and Science Study (TIMSS) is a comparative international assessment study conducted by the International Association for the Evaluation of Educational Achievement (IEA). TIMSS aims to study how educational opportunities are provided for students and what factors are associated with these opportunities. The purpose of this study was to examine the student factors in the United Arab Emirates that have an association with grade 4 students’ TIMSS 2015 results in the content and cognitive domains in the subjects of mathematics and science. The study adopted the quantitative research approach through the data analysis of TIMSS 2015 for grade 4 students in these subjects. The study sample consisted of 21,177 students enrolled in 372 UAE private schools and 186 public schools. The percentage of grade 4 girls who participated in the study was 48%, while the percentage of boys was 52%. A multiple linear regression analysis was conducted to examine the most influential student factors that impact on science and maths achievement. Structural equation modeling (SEM) was implemented to examine the relationships between student factors and the content and cognitive domains of mathematics and science in the TIMSS 2015 results. The findings showed that the student factors with a positive association with student achievement were having breakfast on school days, engaging teaching in mathematics lessons, liking learning science, and confidence in mathematics and science. There was a non-significant correlation between gender and mathematics and science achievement. A surprising finding was that “liking learning mathematics” had a negative association with student performance in that subject. There was a positive association between student engagement and mathematics achievement, while the association between the engagement in science lessons and student performance was found to be insignificant.

Kız ve Erkek Öğrencilerin Fen ve Matematik ile İlgili Kariyer İsteklerindeki Eğilimler

The aim of this study is to examine secondary school students' desire to pursue a career in science and mathematics in 2011 and 2015, taking into account gender differences, and to examine how variables related to students' socioeconomic status, motivation and performance predict the desire of female and male students to pursue a career in science and mathematics. The number of books, study environment at home and parents’ educational levels were analyzed as SES variables. While motivational variables were students’ self-efficacy and task value beliefs, performance related variables were their achievement and engagement. The data of the eighth-grade students from Trends in International Mathematics and Science Study (TIMSS) 2011 and 2015 were analyzed. To investigate the year differences of girls’ and boys’ career aspirations in science and mathematics, Chi-Square test was performed. In addition, logistic regression analysis was conducted to investigate the relation between students’ SES, motivation, performance and career aspirations for 2011 and 2015 data separately. The results have indicated that students’ motivation was the strongest and the most common predictor of career aspirations in science and mathematics for boys and girls.

Identifying the Predictors of Mathematics Anxiety AND Performance in Canada: An Educational Data Mining Approach

Over the last decade, Canadian students have exhibited insubstantial improvements in mathematical scores compared to other countries as indicated by large-scale educational assessments such as the Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS). In relation to students’ mathematical performance, math anxiety - the feeling of fear or nervousness when performing math-related tasks - was found as an associated factor. However, no previous study has explored math performance and math anxiety, specifically among Albertan students. We present a work-in-progress that identifies significant predictors of math performance and math anxiety among Canadian and Albertan students, using the PISA 2018 and TIMSS 2019 datasets. This study has three phases: first, a list of predictors will be selected from the data set based on existing theories regarding students’ math performance and math anxiety. The initial list of predictors will be presented to domain experts (i.e., math teachers) for refinement based on their practical experience. A predictive model for math performance and math anxiety will be developed with Educational Data Mining techniques in the second phase. Results from the model will be presented to the domain experts for their inputs as the qualitative component, and variable importance metrics of the model will be consulted for the quantitative component. Findings from both components will be integrated consulted with the domain experts to derive actionable recommendations that would inform various stakeholders (e.g., educators, school districts, and Alberta Education) of ways to improve math performance in Alberta students.

CRITICAL THINKING EFFECTIVE EDUCATION RESOURCES MANAGEMENT DURING THE CRISIS OF THE 21st CENTURY

In general, there is a wide gap in the quality of education between education in our country and the quality of education in developed countries, although in some individual achievements at the international level, Indonesian children are able to excel. The wide gap in the quality of education in Indonesia is reflected, among other things, from the results of a study on reading ability for elementary school (SD) conducted by the International Educational Achievement (IEA) organization which conducted a research called Trend in International Mathematics and Sciences Study (TIMSS) in in 1999 which indicated that elementary school students in Indonesia were in 38th place out of 39 countries studied. Meanwhile, for junior high school (SMP) level, the study for mathematics ability of junior high school students in Indonesia was only 39 out of 42 countries studied, and natural science (IPA) skills only ranked 40 out of 42 countries studied. In 2003, the same research was conducted. To catch up with education, the government launched a superior school policy. This policy is expected to be able to improve the quality of education, especially schools. This policy is stated in the Strategic Plan of the Ministry of National Education 2010-2014, namely by doing: 1. Development of local excellence-based schools in each district/city. 2. Construction of quality schools commensurate with international education management in each province/district/city. Before the policy of superior schools proclaimed by the government was set, actually schools that could be categorized as superior schools had been scattered in this country, unfortunately most of them were scattered in big cities. Until 2020, the survey results prove that the State of Indonesia is in 10th place from the bottom with other countries in terms of literacy and science. This article uses library research methods from books and journals, aiming to find out how effective HR quality is in 21st century education unit management.

Estimating Heterogeneous Treatment Effects Within Latent Class Multilevel Models: A Bayesian Approach

This article presents a latent class model for multilevel data to identify latent subgroups and estimate heterogeneous treatment effects. Unlike sequential approaches that partition data first and then estimate average treatment effects (ATEs) within classes, we employ a Bayesian procedure to jointly estimate mixing probability, selection, and outcome models so that misclassification does not obstruct estimation of treatment effects. Simulation demonstrates that the proposed method finds the correct number of latent classes, estimates class-specific treatment effects well, and provides proper posterior standard deviations and credible intervals of ATEs. We apply this method to Trends in International Mathematics and Science Study data to investigate the effects of private science lessons on achievement scores and then find two latent classes, one with zero ATE and the other with positive ATE.

A systematic review of studies investigating science teaching and learning: over two decades of TIMSS and PISA

ABSTRACT A great number of studies have investigated science teaching and learning (STL) using data from the Trends in International Mathematics and Science Study (TIMSS) and Programme for International Student Assessment (PISA). Nevertheless, there is little effort to synthesise these studies. Reviewing the status of research on STL, including the trends, approaches, and findings is crucial for identifying research gaps that require greater attention. Consequently, this review synthesises empirical studies investigating STL using TIMSS and PISA. First, we mapped their characteristics based on the aims, data, STL measures, and research approaches. Second, patterns of the findings were summarized by focusing on the (1) relationships between STL and student outcomes; (2) factors that explain STL variation; and (3) patterns of STL. Most studies examined STL related to inquiry activities and their relationships to student outcomes. Their findings were largely inconsistent and could be traced back to different ways STL was conceptualized and measured. This review calls for the studies examining TIMSS and PISA data to enhance the quality of research approaches and clarity in reporting them. It also encourages researchers to reflect upon the knowledge gained from harnessing these data to examine STL and discusses the challenges and opportunities that lie ahead.

Measuring students' learning progressions in energy using cognitive diagnostic models.

This study applied cognitive diagnostic models to assess students’ learning progressions in energy. A Q-matrix (i.e., an item attribute alignment table) was proposed based on existing literature about learning progressions of energy in the physical science domain and the Trends in International Mathematics and Science Study (TIMSS) assessment framework. The Q-matrix was validated by expert review and real data analysis. Then, the deterministic inputs, noisy ‘and’ gate (DINA) model with hierarchical relations was applied to data from three jurisdictions that had stable, defined science curricula (i.e., Australia, Hong Kong, and Ontario). The results suggested that the hypothesized learning progression was consistent with the observed progression in understanding the energy concept. We also found similarities in students’ attribute mastery across the three jurisdictions. In addition, we examined the instructional sensitivity of the selected item. We discuss several curriculum-related issues and student misconceptions that may affect students’ learning progressions and mastery patterns in different regions of the world.

KEMAMPUAN NUMERIK SISWA SEKOLAH MENENGAH ATAS DALAM PEMECAHAN MASALAH MATEMATIS

The importance of problem solving skills is based on the results of the Trends in International Mathematics and Science Study (TIMSS) survey which means that students' mathematical problem solving abilities in Indonesia are still in a very low category. Where one factor in the low problem solving ability of students is the low numerical ability of students. Based on these results, researchers are interested in revealing deeper results regarding students' numerical abilities in solving mathematical problems. Thus, this study was conducted with the aim of knowing students' numerical abilities in solving mathematical problems. The method used in this research is descriptive method. With the form of descriptive quantitative research, the subjects in this study were students of class XII MIA MA Khulafaur Rasyidin and the object of this research was numerical ability in solving mathematical problems. The data collection technique used a test technique with the data analysis technique used, namely the average percentage. The results of data analysis showed that the numerical ability of students in solving mathematical problems was included in the medium category with the percentage obtained at 58.428%. Which means that overall there are still students who are not fast and careful in doing arithmetic calculations when solving problem solving problems properly.

Is inequitable teacher sorting on the rise? Cross-national evidence from 20 years of TIMSS

Unequal access to qualified teachers for children of different socioeconomic status—also known as inequitable teacher sorting—has been increasingly put forth as one potential factor contributing to the socioeconomic achievement gap. Despite this, few studies have investigated cross-national differences in teacher sorting, and none have examined it within-countries over time. International large-scale assessments in education are uniquely positioned to answer such questions due to their longitudinal nature at the system level. This study uses six waves of data from the Trends in International Mathematics and Science Study (TIMSS) from 1999 to 2019 for 32 education systems. We compare differences in grade 8 mathematics teacher qualifications for each country at each time point, across top and bottom groups on the student socioeconomic spectrum. Results show that on the whole many countries display negligible gaps in access to teacher quality, with some key exceptions. With respect to inequity in novice teacher sorting, the problem is most prevalent in low- and middle- income education systems (i.e. in Turkey, Morocco, Tunisia and Indonesia). Inequity in sorting based on mathematics education is less common, with no clear pattern in regards to level of economic development (i.e. in Chile, Australia, New Zealand, and Chinese Taipei). Socio-economic inequality in teacher sorting has also remained broadly stable over time. Based on experience and mathematics education, less than a handful of systems show systematic upward trends in teacher sorting inequity (i.e. in Chile, Morocco, Singapore, and New Zealand). Given the increasing focus on inequity in access to teacher competence, these results have economic and policy implications for tackling the socioeconomic achievement gap.

The Implementation of Cognitively Challenging Tasks: The Role of Science Teachers’ Professional Development and Teaching Experience

This study deals with the implementation of cognitively challenging tasks for students in science education following professional development interventions for teachers based on the theory of change. These tasks require non-routine behaviors and encourage students to work beyond instructions and accept a high level of responsibility. This study explores how the content of professional development, the duration of professional development, and the extent of the teaching experience of science teachers are associated with the implementation of cognitively challenging science tasks in science education. A secondary data analysis of Trends in International Mathematics and Science Study (TIMSS) 2019 data from three countries (Singapore, Lithuania, and South Africa) was conducted. Ordinal logistic regression (OLR) was used to predict an ordinal dependent variable (the implementation of cognitively challenging science tasks) according to the independent variables (the content of professional development, the duration of professional development, and the extent of the teaching experience of science teachers). The results of the OLR confirmed that the implementation of cognitively challenging tasks is associated with professional development content (improving students’ critical thinking or inquiry skills and science content) and the extent of the teaching experience of science teachers from the selected countries.

Textbooks and Students’ Knowledge

In Slovenia, textbooks are an integral part of the curriculum. Nationally certified textbooks guarantee both teachers and students that they provide all of the necessary knowledge in each subject. There are many available certified textbooks for each subject and teachers must decide which will be the source of instruction for their students. Our research question is whether groups of students who use different textbooks as their mandatory learning resource differ in their knowledge and their attitudes towards learning. We linked existing data from several sources and explored the scope of the use of different textbooks for mathematics and science subjects in primary schools. Data on student knowledge measured independently by National Assessments (and the Trends in International Mathematics and Science Study were used, and differences in knowledge and attitudes to learning between students who are taught using different textbooks were explored. Although the study has considerable limitations due to missing data, the results of the analyses indicate some profound differences in knowledge and attitudes between groups of students using different textbooks. These findings could serve as a guide for teachers when choosing the optimal available textbook for their students and, even more so, as support for improving the criteria in the national system of validation of textbooks in the future. The link between the use of textbooks and student learning outcomes also highlights the need to systematically collect information on the use of textbooks among students and follow the effects on achievement in order to improve the quality of future textbooks.

The Influence of School Factors on Students’ Mathematics Achievements in Trends in International Mathematics and Science Study (TIMSS) in Abu Dhabi Emirate Schools

This study aimed to explore school factors that influence students’ achievements in Trends in International Mathematics and Science Study (TIMSS) in schools in the Emirate of Abu Dhabi, in the United Arab Emirates. The study sample for TIMSS 2015 consisted of 4838 students in eighth grade; 2172 girls, 2666 boys, and 156 schools from Abu Dhabi attended and participated in TIMSS 2015. Principal component analysis (PCA) was run on 77 items of school questionnaires administered to school principals to provide information about the school contexts for teaching and learning. The five factors from the school questionnaire were general school resources, school discipline and safety, parental support, principal experience and education, and library and instruction resources. Multiple regression models were implemented to examine the impact of school factors on student achievement in TIMSS 2015. The models are statistically significant, indicating that they fit the data well. This also demonstrates a significant linear relationship between students’ achievement in TIMSS 2015 and the variables related to school factors.

An evaluation of the impact of flipped‐classroom teaching on mathematics proficiency and self‐efficacy in Saudi Arabia

AbstractThe flipped classroom (FC) is becoming an increasingly popular teaching method in mathematics education. However, few studies have rigorously evaluated its effectiveness, and less so in countries where students' level of mathematics achievement is low. In this study, we evaluated the impact of an FC intervention in Saudi Arabia, the country with the lowest level of maths achievement in the last iteration of the Trends International Mathematics and Science Study TIMSS . A total of 281 secondary school students received eight weeks of algebra training using either FC or traditional instruction, and were tested on algebra problems taken from past national standardised tests, as well as on a measure of self‐efficacy. Students who received the intervention showed higher self‐efficacy but no significant difference in maths achievement was observed. Students' and teachers' perceptions of the intervention were positive.Practitioner notesWhat is already known about this topic The flipped classroom is an increasingly popular teaching method in mathematics education. There is evidence that the method is beneficial for students. However, there have been very few rigorous evaluation studies of flipped classrooms and most were conducted in the US, Taiwan and China. Whether the beneficial impact of flipped classrooms can be generalised to other countries, particularly countries with low proficiency in mathematics, remains unclear. What this paper adds We evaluated the impact of the flipped classroom in Saudi Arabia, a country with low proficiency in mathematics and limited technological resources. We used a rigorous design (with a control group) and educationally relevant outcome measures. We found that a flipped classroom can have a positive impact on students' self‐efficacy, and that the method was perceived positively by both teachers and students. Based on interview data, we also documented the perceptions and concerns of the participating teachers. Implications for practice and/or policy Flipped classrooms showed promising results in a context which differs markedly from previous evaluations, suggesting that the method can be beneficial in a range of contexts. This positive impact was observed despite the intervention being relatively short (six weeks) and implemented by teachers previously unfamiliar with the flipped classroom method.

A Cultural-historical Pedagogical Model as a Potential Developmental Tool in Schools

South Africa consistently lags behind other nations on international benchmarking tests of literacy, mathematics and, especially, science as illustrated in the low achievement outcomes on the Progress in International Reading Literacy Study (PIRLS) and Trends in International Mathematics and Science Study (TIMSS) tests. Research in the 21st century has shown that learning science at an early age is a predictor of scientific literacy later in life. The low science achievement in South Africa is impacted by several variables, however, one important aspect of students’ failure to acquire science reasoning skills lies in how they are taught science. Drawing on the work of Vygotsky, Hedegaard, Freire, and Feuerstein, this paper reports on an intervention using a novel pedagogical model to teach science in grade two classrooms. A comparative case study is presented where one teacher is trained in the novel pedagogical model and one is not trained to use this model. Both teachers teach in the same school and the demographics of the children in the two separate classes studied are similar. Findings indicate that where the novel pedagogical model is used, the teacher uses more scientific, abstract concepts in her lesson; she links the abstract to the children’s everyday concepts and, perhaps most significantly, she illustrates to students why they must learn the content she is teaching them.

Are instructional practices different between East and West? An analysis of Grade 8 TIMSS 2019 data

After each round of International Large-scale Assessments such as the Trends in International Mathematics and Science Study (TIMSS) and the Programme for International Student Assessment (PISA), excellent achievements by countries in East Asia are explained in terms of their instructional practices. However, instructional practices are not easily captured in self-reported scales. In addition, speculations on what approaches lead to the highest mathematics achievement are often combined and conflated with discussions on how mathematical content needs to be taught. This article reports on a secondary analysis of TIMSS 2019 Grade 8 data from 46 jurisdictions of teacher-reported answers on instructional practices. Using a data-driven exploratory factor analysis, three instructional scales are compiled and supplemented with variables on barriers to instruction and instructional time. Multilevel models of all countries are created for students and teachers, in which the relationship between these instructional practices and mathematics achievement is explored. Across countries, the results show that the three new scales for instructional practice work in limited and different ways, but that barriers toward instruction, and to a lesser extent, instructional time, predict mathematics achievement. However, there is no consistent pattern of instructional practices across all countries, and also not among East Asian countries.