COMPREHENSIVE ANALYSIS OF THE FORT INSTRUMENT: USING DISTRACTOR ANALYSIS TO EXPLORE STUDENTS’ SCIENTIFIC REASONING BASED ON ACADEMIC LEVEL AND GENDER DIFFERENCE

Abstract

Scientific reasoning ability is essential to get developed in the current digital age, particularly in the process of judgement and decision-making in complex problems. Few studies have conducted an in-depth exploration of scientific reasoning ability, especially in relation to the confidence level and gender. The scientific reasoning ability of Indonesian upper-secondary school and university students were examined and compared with previous recorded data of US students. In this study, the data were collected from 372 university and 528 upper-secondary education students in Indonesia. Students’ scientific reasoning ability was measured using a scientific formal reasoning test (FORT). In addition, confidence level and metacognitive data was collected through self-reported measures. Two-way ANOVA was performed to compare mean differences between groups based on academic level and gender and to observe interaction between the variables. Students’ confidence level in selecting the correct answer and distractor answer was analyzed using an independent t-test. The results reveal that many Indonesian students selected specific distractors with relatively high confidence. Moreover, upper-secondary school students and female students selected more distractors than the groups’ counterparts. Finally, the factors related to Indonesian students’ responses to the scientific formal reasoning were discussed. Keywords: confidence level, distractor analysis, gender differences, scientific (formal) reasoning test, scientific reasoning ability, Indonesian student