Trends In International Mathematics And Science Study Research Articles

TIMSS 2015: Illustrating Advancements in Large-Scale International Assessments

International large-scale assessments of student achievement such as International Association for the Evaluation of Educational Achievement’s Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study and Organization for Economic Cooperation and Development’s Program for International Student Assessment that have come to prominence over the past 25 years owe a great deal in methodological terms to pioneering work by National Assessment of Educational Progress (NAEP). Using TIMSS as an example, this article describes how a number of core techniques, such as matrix sampling, student population sampling, item response theory scaling with population modeling, and resampling methods for variance estimation, have been adapted and implemented in an international context and are fundamental to the international assessment effort. In addition to the methodological contributions of NAEP, this article illustrates how the large-scale international assessments go beyond measuring student achievement by representing important aspects of community, home, school, and classroom contexts in ways that can be used to address issues of importance to researchers and policymakers.

Girls Do Better

abstractIn this article we analyse gender gaps in educational outcomes in South Africa using four nationally-representative datasets - Progress in International Reading and Literacy Study (PIRLS), Trends in International Mathematics and Science Study (TIMSS), Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ) and Matriculation – over the period 1995-2018. We show that girls outperform boys at the mean in all subjects and all grades, including in Mathematics and Physical Science, as well as in the school-leaving exam, Matric. Pro-girl gaps at the primary-school level are large and statistically significant with Grade 4 girls an entire year ahead of Grade 4 boys in reading outcomes. Similarly, large differences can be found in the most recent (2018) Matric microdata. We show that the received wisdom in South Africa, that males outperform females in Mathematics and Physical Science in Matric, is partly a function of higher rates of male dropout in high school, leaving a stronger cohort of males to write matric. In 2018, for every 100 females in matric there were only 80 males. If one compares an equal number of males and females in matric, girls do unequivocally better in all 13 subjects when looking at average performance. When looking at higher levels of achievement (60%+) girls do better in nine subjects and boys do better in two subjects (Mathematics and Physical Science). However, girls are still less likely to fail these two subjects when compared to their male counterparts. Given these results and the theory that human capital contributes to employment and earnings, we see the need for further research in the South African context for why the female advantage in education does not translate into a female advantage in the labour market. What are the general, and specific, constraints for why women experience inferior labour outcomes in the world of work?

Negative Keying Effects in the Factor Structure of TIMSS 2011 Motivation Scales and Associations with Reading Achievement

ABSTRACTThe Student Background survey administered along with achievement tests in studies of the International Association for the Evaluation of Educational Achievement includes scales of student motivation, competence, and attitudes toward mathematics and science. The scales consist of positively- and negatively keyed items. The current research examined the factorial structure of the 18-item motivational scales in fourth-grade mathematics in the 2011 Trends in International Mathematics and Science Study (TIMSS). Survey data from six European countries were analyzed. In comparisons of alternative models, the fit was adequate when three correlated factors were specified and negative keying was taken into account as a latent factor, or with correlated uniquenesses among negatively keyed items. Participants reading achievement scores correlated systematically to negative keying with coefficients ranging from .254 to .395 in the six samples. Unlike their higher-scoring peers, fourth-graders with lower reading achievement responded differentially to similar items depending on the direction of item keying, in such a way that their motivation scores were biased downward. Implications about the use of reverse keying in surveys for young students are discussed.

The contribution of South African teachers to students’ sense of belonging and mathematics achievement: Students’ perspective from the 2015 Trends in International Mathematics and Science Study

Background: The role of teachers in nurturing students’ sense of belonging cannot be over-emphasised. Students who do not feel accepted by their teachers are at risk of withdrawing from school life and feeling disaffected. This study contributes to theories on school belonging by investigating the contribution of teachers to students’ sense of school belonging, the association of students’ attitudes towards teachers, and their sense of school belonging with students’ mathematics achievement.Aim: To provide empirical evidence of how students’ attitudes towards teachers contributed to their sense of school belonging, as well as their mathematics achievement.Setting: A representative sample of 10932 grade 5 students at 297 schools in South Africa completed a contextual questionnaire and a mathematics assessment during the 2015 Trends in International Mathematics and Science Study (TIMSS).Methods: The TIMSS 2015 data were used to develop indicators of students’ attitudes towards teachers, sense of school belonging and home socio-economic status. Absenteeism and the extent of bullying were also considered. Descriptive and inferential statistical analyses were performed.Results: A high positive correlation between students’ attitudes towards teachers and their sense of school belonging was found. Students’ attitudes towards teachers and their sense of belonging contributed significantly to mathematics achievement.Conclusion: The study confirms the crucial role that a sound student–teacher relationship plays in a healthy sense of school belonging and in terms of academic performance. The school environment should be managed in a manner that allows for mentoring relationships between students and teachers to be strengthened.

The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters

In large-scale assessments like Programme for International Students Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS), plausible values are often used as students’ ability estimations. In those studies, stratified sampling method is employed in order to draw participants, and hence, the data gathered has a hierarchical structure. In the context of large-scale assessments, plausible values refer to randomly drawn values from posterior ability distribution. It is reported that using one of plausible values or mean of those values as independent variable in linear models may lead to some estimation errors. Moreover, it is observed that sampling weights sometimes are not used during analysis of large-scale assessment data. This study aims to investigate the influence of three approaches on the parameters of linear and hierarchical linear regression models: 1) using only one plausible value, 2) using all plausible values, 3) incorporating sampling weights or not. Data used in the present study is obtained from school and student questionnaires in PISA (2015) Turkey database. Results revealed that the use of sampling weights and number of plausible values has significant effects on regression coefficients, standard errors and explained variance for both regression models. Findings of the study were discussed in details and some conclusions were drawn for practice and further research.

Trends in Qualification of South African Mathematics Teachers: Findings from TIMSS 2003, 2011, 2015

The chain of progression from teacher education to qualification to classroom instruction and student engagement, and then to learning and achievement, is not as straightforward as often presented. Societal, political and economic pressures create either a positive climate or a stressful one for both teachers and students. Nevertheless it is safe to say that in a subject like mathematics, the importance of subject matter knowledge cannot be underestimated. Research in this area supports the presumption that formal qualifications in the subject, in this case mathematics, are necessary, as this focused engagement with mathematics provides the foundation for mathematics teaching and learning. The allocation of fully qualified teachers to schools has in the past been uneven, with the supposition that students at higher socio-economic levels are being taught by better qualified teachers. In South Africa the improvement of teacher qualifications has been identified as a core area of action with the relevant policy on teacher education renewed in 2007 and again in 2014. The focus of this paper is to understand the change in patterns of teacher qualifications, and the interrelationship with school contextual variables. The analysis is based on the Trends in International Mathematics and Science Study (TIMSS) data from 2003, 2011 and 2015, where teachers from the TIMSS sample provided contextual information. Descriptive statistics were calculated using the IEA IDB Analyzer. The results show that South Africa has made substantial progress in uplifting teachers’ formal qualification levels and in reducing structural inequality within its education system. Out-of-field teaching of mathematics is still shown to be a common phenomenon and is unevenly distributed.

Development of Test Instrument TIMSS Model to Measure Mathematical Ability VIII Grade Students

The teachers are still lack found practice tests that have characteristics such as the Trend in International Mathematics and Science Study (TIMSS) in Junior High School mathematics textbooks. This study aims to develop a TIMSS test instrument model that refers to the cognitive domain and the content domain for class VIII students. The respondents of this study were 5 people in the expert test, 10 people in the one to one test and 53 people in the small-scale test. This research is development research conducted following the development model of Borg & Gall. This step is only taken until the ninth step with a few modifications. Modifications are made by adding a one to one test copy from Tessmer. In content validity, as many as 5 items out of 35 items developed are not yet valid with a coefficient range of 0.6-0.79. Items were then revised to be tested in the small-scale test stage. One to one test results showed the instrument readability level was 81%. The validity test of the criteria shows a coefficient of 0.84 which means that the product developed is comparable to the original product. Interrater reliability was 0.661 and Alpha Chronbach reliability testing was a small scale at 0.854. The results showed that the instrument developed was valid in terms of content and criteria as well as reliable for use. The benefit of this research is that the developed test instrument can be used as a tool to measure the mathematics abilities of eighth-grade students.

A COMPARATIVE STUDY OF QUADRILATERALS TOPIC CONTENT IN MATHEMATICS TEXTBOOKS BETWEEN MALAYSIA AND SOUTH KOREA

This study compares Malaysian and Korean geometry content in mathematics textbooks to help explain the differences that have been found consistently between the achievement levels of Malaysian and South Korean students in the Trends in International Mathematics and Science Study (TIMSS). Studies have shown that the use of textbooks can affect students’ mathematics achievements, especially in the field of geometry. Furthermore, to date, there has been no comparison of geometry content in Malaysian and Korean textbooks. Two textbooks used in the lower secondary education system in Malaysia and South Korea were referred. The topic of quadrilaterals was chosen for comparison, and the topic’s chapter in the South Korean textbook has been translated into English. The findings show four main aspects that distinguish how quadrilaterals are taught between the two countries. These aspects include the composition of quadrilaterals topics, the depth of concept exploration activities, the integration of deductive reasoning in the learning content and the difficulty level of mathematics problems given at the end of the chapter. In this regard, we recommend the Division of Curriculum Development of the Malaysian Ministry of Education reviews the geometry content of mathematics textbook used today to suit the curriculum proven to produce students who excel in international assessments.

Analysis of the Science Course Curriculum Objectives and High School Entrance Examination Questions According to TIMSS Framework

<p style="text-align:justify">The aim of this study is to compare 2018 Science Course Curriculum (SCC), 2015 Trends in International Mathematics and Science Study (TIMSS) and 2018 High School Entrance Examination (HSE) in terms of content domains, cognitive domains and learning objectives. Qualitative research method, was used in this study. Data were analyzed using document review matrices to determine the similarities and differences between the objectives of SCC, TIMSS and HSE. SCC outcomes and HSE science questions were also classified according to TIMSS cognitive domains. Results show that the learning objectives of the fields of Physics, Biology and Earth Sciences of TIMSS are compatible with those of all grade levels of SCC and that the objectives of Chemistry are compatible with those of the seventh and eighth grades. Most of HSE questions are compatible with the objectives of SCC, however, the latest revision in the curriculum has introduced some eighth grade objectives to other grade levels. HSE science questions measure higher-level skills than TIMSS science questions. The subject domain of the “Organisms and Life” of SCC has the most learning objectives in the levels of “knowing” and “reasoning” while the subject domain of the “Physical Events” has the most learning objectives in the levels of “applying.” Besides, the seventh-, fifth- and eighth-graders have the most objectives in the levels of “knowing,” “applying,” and “reasoning,” respectively. It is hoped that the results will contribute the literature in improvement of science curricula and interpretation of national and international exams.</p>

Looking through Philippine’s K to 12 Curriculum in Mathematics and Science vis-a-vis TIMSS 2015 Assessment Framework

The Philippines implemented its educational reform commonly known as K to 12 Program in 2013 for Filipino learners’ holistic development and global competitiveness. One indicator of the reform’s effectiveness is the country’s performance in international large-scale assessments like the Trends in International Mathematics and Science Study (TIMSS), if it participates again in 2019. To gauge readiness and inform decisions on its participation, this study examines the alignment of TIMSS 2015 Assessment Framework vis-à-vis the 2016 Philippine’s K to 12 Curriculum in Mathematics and Science. Through curriculum mapping of experts on the competencies in the two documents, the study reveals that K to 12 Mathematics Grade 4 is more aligned with TIMSS 2015 Assessment Framework than that of Math Grade 8, Science Grade 4, and Science Grade 8. The study recommends continuous curriculum improvement of the Philippine’s K to 12 Curriculum and implementation of relevant intervention programs and support system from different education stakeholders, including country-based publishing and testing companies to address gaps on competencies found for the country to meet international benchmarks.

Analysis of Student Errors in Completing PISA Content Space and Shape Problems Viewed from Vak Learning Style

Trends in International Mathematics and Science Study (TIMSS) study and the Program for International Student Assessment (PISA) is one of the principles in curriculum development in Indonesia. Judging from the results of the PISA study, the achievements obtained by Indonesian children especially in the aspects of mathematical literacy have not been satisfactory. The low achievement is inseparable from the learning process in schools, one of which students are not accustomed to solving questions with characteristics such as the PISA context. In addition, the way students absorb information also determines how learning achievements will be obtained by students. The way students learn is often referred to as a learning style. This indicates that the mathematics literacy of junior high school students in Indonesia is so weak that they still have difficulty in solving PISA questions. As a result, many mistakes were made by junior high school students when faced with PISA questions. The purpose of this study was to describe the errors of students with visual, auditory, and kinesthetic learning styles in solving space and shape content PISA questions. Students' errors were analyzed from the adaptation of Newman's error analysis model namely comprehension, transformation, proccess skills, and encoding. This type of research is qualitative descriptive. Data collection techniques that be used are tests and interviews. The results showed that students with visual learning styles tend to make mistakes in the transformation step. Whereas students with auditory learning styles tend to make mistakes in understanding and transformation steps, and students with kinesthetic learning styles make mistakes in the steps of understanding, transformation, and process skills. This shows that students who have kinesthetic learning styles do not have a tendency in one type of error. Keywords: Error analysis, PISA, VAK Learning Style

Relationships between self-efficacy and achievement moderated by teacher interaction: an international comparison study

This research aims to scrutinize the moderating effect of teacher interaction on the relationships between students’ self-efficacy and achievement in mathematics in three countries, Korea, Singapore, and the USA. We analyzed the Trends in International Mathematics and Science Study (TIMSS) 2015 data with hierarchical linear modeling. The findings show that in each country, different types of teacher interaction moderate the relationships between students’ self-efficacy and achievement in mathematics. Also, when moderated relationships are found, Korean students with lower self-efficacy are likely to have higher achievement when teachers interact with each other very often. Singaporean students across all self-efficacy levels are expected to have advantages of high achievement from teachers interacting with other teachers. However, US students have similar achievement although teachers have frequent interaction with other teachers. We suggest several implications: To improve educational systems, it is necessary to consider which type of teacher interaction is common in that specific context as well as to examine how teacher interaction moderates the relationships between students’ self-efficacy and achievement in mathematics.

The effect of instructional quality on student achievement: Evidence from Japan

This paper investigates the heterogeneous effects of changes in instructional time on student achievement due to differences in instructional quality, using student test scores and data on teachers and schools from the Trends in International Mathematics and Science Study (TIMSS), as well as a natural experiment caused by a 2002 Japanese curriculum standards revision that reduced instruction time for math but not for science for second-year junior high school (grade 8) students. The random matching of teachers and students in Japan and our application of the time dimension of panel data to two subjects, mathematics and science, allow us to control for unobserved heterogeneities such as individual fixed effects and school fixed effects. We confirm that instructional time is more effective in combination with higher-quality teachers, as measured by common indicators such as experience, schooling, and academic major. We also find that these effects are larger for students whose socioeconomic status (SES) is lower.

Examination of the Extreme Response Style of Students using IRTree: The Case of TIMSS 2015

In the literature, response style is one of the factors causing an achievement-attitude paradox and threatens the validity of the results obtained from studies. In this regard, the aim of this study is two-fold. Firstly, it attempts to determine which item response tree (IRTree) models based on the generalized linear mixed model (GLMM) approach (random intercept, random intercept with the fixed effect of extreme response and random intercept-slope model) best fit the Trends in International Mathematics and Science Study (TIMSS) 2015 data. Secondly, it purports to explore how the extreme response style affects students’ attitudes toward mathematics of students. This study is both basic research and descriptive research in terms of seeking for answers for two different research questions. For the sample of this research, 15 countries were randomly selected among countries participated in TIMSS 2015. The students’ responses to items measuring attitude in the student questionnaire were analyzed with the packages “lme4” and “irtrees” in R software. When the model fit indices were evaluated, the random intercept-slope model was found to be the best fit to the data. According to this model, the extreme response style explains a significant amount of variances in the students’ attitude toward mathematics. Additionally, students with a negative attitude toward mathematics were found to have an extreme response style. It was concluded that an extreme response style had an effect on students’ attitude.

Does instructional leadership mediate effects of student home resources on opportunity to learn and math reasoning skills? A cross-national comparison of 4th grade students

ABSTRACT For many countries, inequitable access to rigorous content and instruction at the early grades leads to the low academic performance for students with fewer resources at home. One way to assess these inequities, and identify possible mediators, is to consider the role of instructional leadership as a means for schools to set educational goals for teachers and students which might direct how learning is accessed. This study examines the extent that instructional leadership influences the mediated relationship between students’ home resources and opportunity to learn on math reasoning skills across a set of 53 participating countries. We applied a two-level structural equation model to data from the 2011 Trends in International Mathematics and Science Study (TIMSS). Using a nationally representative sample of schools and nested students for each participating country in TIMSS 2011, we demonstrate the variation in the extent and direction of these relationships by country context. For a few countries, instructional leadership mediated the relationship between home resources and opportunity to learn on math reasoning skills.

From the Confucian Tradition to the Digital Era: The Case of Mathematics Teaching in Hong Kong

Recent years have witnessed an increase in research focused on studying on perspectives of Chinese mathematics instructions. The sustained interest is partly due to the outstanding performances of Chinese students in international studies such as the Trends in International Mathematics and Science Study (TIMSS) (Mullis, et al., 2012) and the Programme for International Student Assessment (PISA) (Organization for Economic Co-operation and Development (OECD), 2010); and partly due to the shared interest in comparative studies of instructional practice across different cultural systems. What have we learned about in mathematics classrooms from international studies? Findings of international studies suggest that how the teacher used the tasks so that the cognitive demand of the learning tasks could be sustained is very important. Despite the good performances of Hong Kong students in international studies, there is a gap between traditional classroom practice and the long established goals for promoting generic capacity in mathematics learning; most of the traditional learning tasks in Hong Kong classrooms are apparently routine and serving a demonstrative purpose as a result of the highly competitive systems. Based on triangulation of the findings of the analysis of the mathematics lessons at different levels, the study shown some robust features in the traditional mathematics teaching practice in Hong Kong classrooms in contrast to some innovative scenarios in a special mathematics lesson. Finally, the author re-examines the robust features in the context of curriculum reforms and the cultural context of Confucian tradition.

Multimodal literacy and large-scale literacy tests: Curriculum relevance and responsibility

An educational chasm divides the Australian National Curriculum, which is permeated with detailed requirements for students to develop multimodal literacy, and the very substantially mono-modal literacy of the reading tests of the Australian National Assessment Program in Literacy and Numeracy (NAPLAN). This paper proposes that mandated centralised large-scale testing approaches to literacy assessment, such as NAPLAN, need to be re-thought and reformed to be consistent with curriculum requirements in relation to multimodal literacy and with the multimodal nature of international tests such as Trends in International Mathematics and Science Study (TIMSS) and the Program for International Student Assessment (PISA) and, most importantly, to influence and support teachers in ensuring that students acquire essential multimodal literacy competencies in the twenty-first century. This paper firstly demonstrates that the interaction of language and image in multimodal texts are integral to the nature of the multimodal literacy that is increasingly needed to negotiate the vast majority of texts we encounter in all domains of our lives, and are reflected in national curriculum documents. Secondly, analyses of international tests such as TIMSS and PISA show the significant proportion of assessment items that specifically address images and image-language relations for correct responses to test items. Thirdly, we provide analyses of the NAPLAN reading tests from 2008-2016, showing the minimal proportion of test items that deal with images and image-language relations. NAPLAN results contrast markedly with the former NSW Basic Skills Tests (BST), which show a significantly higher proportion of test items that dealt with image-language interaction. Fourthly, we reiterate that the paucity of attention to images in NAPLAN demonstrates that this mandated national literacy assessment does not assess multimodal literacy. This signals a strong need for re-thinking and reforming of NAPLAN, and similar large-scale literacy tests so that assessment approaches will more effectively contribute to enhancing the students' multimodal literacy development. Implications of the paper for supportive action by administrators, teachers and researchers towards such reform are briefly noted.

ALIGNMENT OF 6TH GRADE LARGE-SCALE ASSESSMENT CONSTRUCTS WITH THE REVISED CURRICULUM FRAMEWORK

One of the goals of the National Development Plan 2014-2020 is to reduce the proportion of students with low cognitive skills, and at the same time increase the proportion of students with higher level cognitive skills. In line with those goals, the National Centre for Education is implementing the project “Competency-based approach to curriculum”, funded by the European Social Fund. The purpose of the research described in this article is to find out to what extent the current large-scale national assessments for 6th Grade are coherent with the new curriculum and what improvements are needed for aligning the national assessments with the national curriculum. The theoretical framework of the research is developed by analysing the frameworks of the programme for international student assessment (PISA), trends in international mathematics and science study (TIMSS), progress in international reading literacy study (PIRLS), as well as the framework of the revised national curriculum in Latvia. National 6th Grade assessments of the year 2018 are analysed by using Classical test theory and Rasch model. The indicators of the test items are mapped according to the developed theoretical framework. Authors conclude that the national 6th Grade tests assess the elements of literacy, numeracy and scientific literacy. Students have a high level of performance in test items with low cognitive depths, but there is an insufficient number of test items with high cognitive depths, allowing pupils to demonstrate skills in new contexts, which is an essential goal of the new national curriculum. Further research is required on the use of data from the large-scale assessment in supporting and guiding student instruction and learning.

Predicting Mathematics Achievement: The Role of Socioeconomic Status, Parental Involvement, and Self-Confidence

This study examines the direct and indirect effects of family socioeconomic status (SES), through parental involvement and self-confidence, on students' mathematics learning. The multilevel analyses are conducted within the context of Trends in International Mathematics and Science Study (TIMSS) 2011 in Turkey. Results indicate that two SES components - home educational resources and school composition - positively predict students' mathematics achievement. Furthermore, results indicate that parental involvement at home, and parental involvement in school activities, both have a mediating role between home educational resources, and school composition and mathematics achievement, respectively. Self-confidence appears as the strongest predictor of mathematics achievement, which also mediates the impact of both home educational resources, and parental involvement at home on achievement. Finally, the findings from this current study highlight the importance of multilevel analyses in examining the role of parental involvement in mediating the relationship between SES and students' mathematics achievement.

Mathematics Self-Efficacy Scale Based on Computer

The results of the PISA (Program for International Student Assessment) survey in 2012-2015 on the achievement of mathematical competencies of Indonesian students showed a significant increase, but the overall achievement was still below the average of the countries belonging to the Organization for Economic Co-operation and Development (OECD). Furthermore, the results of the Trends in International Mathematics and Science Study (TIMSS) report showed that many students like and feel good about mathematics, but their confidence in their mathematical abilities was quite low. Many studies reveal the close association of Mathematics Self-Efficacy (MSE) with the performance/achievement of students' mathematical competencies. In 2015, the PISA survey was done using computerization except in 15 countries, one of them was in Indonesia. Therefore, the results of this study are to produce the first computer-based MSE scale developed in Indonesia.