STATISTICAL TECHNIQUES UTILIZED IN ANALYZING PISA AND TIMSS DATA IN SCIENCE EDUCATION FROM 1996 TO 2013: A METHODOLOGICAL REVIEW

Abstract

We conducted a methodological review of articles using the Programme for International Student Assessment (PISA) or Trends in International Mathematics and Science Study (TIMSS) data published by the SSCI-indexed science education journals, such as the International Journal of Science and Mathematics Education, the International Journal of Science Education, the Journal of Research in Science Teaching, and Science Education, from 1996 to 2013. A total of 51 empirical articles from 8 journals were identified as the sample for this study. These articles were analyzed in terms of the 2 essential statistical techniques, sampling weights and design effects, used to analyze the international large-scale assessment (ILSA) data. The study also summarized the most commonly used quantitative methods for analyzing PISA and TIMSS data in these articles. The results indicate that the weights and design effects, essential adjustments for analyzing large-scale data, were reported in less than half of the studies. Suggestions regarding the use of appropriate techniques and reporting as well as data analysis methods are made for science education researchers who use ILSA data in their research.