Abstract
In our paper, we compared some characteristics of TIMSS 2015 and the National Assessment of Basic Competencies (NABC) 2015. The NABC assesses all students’ reading and mathematics performance in Grades 6, 8 and 10. Both studies assessed Hungarian Grade 8 students’ mathematics abilities in the spring of 2015. We linked data of the two studies on the student level using Student Measurement IDs. We compared TIMSS and NABC mathematics scales based on the Assessment Framework of the two studies along with the results of students in the two assessments. The comparison of the Frameworks revealed that although the two tests use similar content and cognitive categorizations, there are crucial differences between the two constructs. While the basis of TIMSS’s mathematics construct is the common part of mathematics curricula of participating countries, NABC intends to measure mathematical literacy, the ability of students to use their mathematical knowledge and competencies in real life situations. The correlation between the TIMSS and NABC mathematics test results (0.79) also confirms that the two tests measure related, but not identical abilities. To evaluate the representativeness of the TIMSS sample we used school- and class-level weight factors of TIMSS and the student-level weights of NABC combined. The mean performances of the TIMSS sample are only slightly lower than the full NABC cohort’s, the effect size of the difference is 0.042 and 0.046 in mathematics and reading respectively. The differences in the standard deviations are somewhat but not considerably larger. The SES-index shows a very good match with no statistically significant differences in the mean and standard deviation of the sample and the full cohort. Our analysis confirms that estimations of population parameters based on TIMSS samples are of a good quality.
Highlights
We compared TIMSS and National Assessment of Basic Competencies (NABC) mathematics scales based on the Assessment Framework of the two studies along with the results of students in the two assessments
TIMSS testing time is divided so 30% of testing time used to solve Number items, 30% used for Algebra items, 20% used for Geometry and 20% used for Data and Chance
In NABC, 35−40% of items belong to Quantities, numbers, operations, 25−30% belong to Assignments, relationships, 20−25% belong to Shapes, orientation, and 12−15% belong to Statistical characteristics, probability
Summary
We chose to link and analyze TIMSS 2015 and NABC 2015 Grade 8 databases, since these assessed the same student population, and have mathematics as one of their cognitive domain in common. TIMSS item parameters are calculated based on data from every participating country using equal weights for every country (Foy & Yin, 2016) These differences might affect Hungarian students’ TIMSS 2015 performance scores and the correlation between the two tests’ results. In order to exclude these effects from our comparison, we have compared NABC scores with a performance score of the TIMSS mathematics test calculated from item response level data of the Hungarian students with methods and software used in NABC. The index consist variables related to highest education of parents, number of books at home and educational and economical resources possessed by the students’ family It was anchored so the average SES is 0 and the standard deviation is 1 for the overall student population of the three grades involved in the assessment (Aux-Bánfi et al, 2015). We have calculated the effect sizes of differences, the difference divided by the standard deviation using the estimation method described in Hedges (2007) for nested data with unequal school sizes
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
