Abstract
International large-scale assessments such as PISA (The Programme for International Student Assessment), PIAAC (The Programme for the International Assessment of Adult Competencies) and TIMSS (Trends in International Mathematics Science Study), play a key role in determining educational policies besides their primary objectives of measuring, evaluating and monitoring the educational process. Therefore, it is critical to analyze the data gathered from the large scale assessments using scientifically accurate statistical methods as the results have the potential to influence millions of stakeholders through major policy changes. Analysis of these data that consists of hundreds of different genuine variables requires expertise and using specific methods. This study illustrates issues to be considered while analyzing PISA, PIAAC and TIMSS data by presenting relevant syntax and exemplifying the possible incorrect results that might be encountered. In Turkey, there are very limited courses that focus on large scale data analysis. Workshops are also very limited to reach major groups. The aim of this study is to raise awareness related to sample weights and plausible values. Comparative findings of the study showed that without using sample weights and plausible values there is a high probability to get incorrect results. In this study, t-test and multiple regression analyses conducted by IDB Analyzer and multilevel regression analysis by Mplus were exemplified.
Highlights
International large-scale assessments such as Programme for international student assessment (PISA) (The Programme for International Student Assessment), PIAAC (The Programme for the International Assessment of Adult Competencies) and Trends in international mathematics science study (TIMSS) (Trends in International Mathematics Science Study), play a key role in determining educational policies besides their primary objectives of measuring, evaluating and monitoring the educational process (Bialecki, Jakubowski, & Wisniewski, 2017; Figazzolo 2009; Novoa & YarivMashal, 2003; Steiner-Khamsi & Waldow, 2018)
This study aims to compare large scale assessment (LSA) data analysis with and without taking into account sample weights and plausible values
The first one is “Is there a statistically significant difference between mean TIMSS 2015 mathematics scores of boys and girls in Turkey?”. ttest was conducted as the grouping variable, gender, contained two categories, boys and girls
Summary
International large-scale assessments such as PISA (The Programme for International Student Assessment), PIAAC (The Programme for the International Assessment of Adult Competencies) and TIMSS (Trends in International Mathematics Science Study), play a key role in determining educational policies besides their primary objectives of measuring, evaluating and monitoring the educational process (Bialecki, Jakubowski, & Wisniewski, 2017; Figazzolo 2009; Novoa & YarivMashal, 2003; Steiner-Khamsi & Waldow, 2018). In the following periods, cross-country comparisons raised the interest of both local and international media, which led the test results to be used as for indicators of economic growth and rationales for policy reforms. Addey, Sellar, Steiner-Khamsi, Lingard and Verger (2017) explained the reasons for participation of the countries to these tests as follows: to provide data-based information for policies, technical capacityinfrastructure building, to provide financial support and assistance, prominence in international relations, decision making in domestic politics, economic reasons, reforms to curriculum and teaching. To date, large-scale assessment data provide crucial information for the efficiency of countries’ educational system elements and comparable data about the current student, teacher, and administrator profiles
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.