Development of scientific reasoning test measuring control of variables strategy in physics for high school students: evidence of validity and latent predictors of item difficulty

Abstract

ABSTRACT Control of variables strategy (CVS) is a core scientific reasoning skill related to domain-general experimentation for evaluating an experimental system and deducing valid conclusions. This study aims to develop, validate, and explore latent factors predicting item difficulty in the test, measuring CVS in physics for high school students. The original test contains 24 items related to mechanics, thermodynamics, and electricity. It measures three CVS subskills: identifying controlled experiments, interpreting the controlled experiment outcome, and understanding confounded experiments’ determinacy. The study assessed 470 10th, 11th, and 12th-grade students from 9 public high schools in the southern province of An Giang (Vietnam). All test items fit the data well according to the Rasch model measurement, in which the three-dimensional model fit better than did the unidimensional one. Differential item functioning analysis indicated no statistically significant differences between genders in any items. Furthermore, we found that item difficulty depended on the subskills but was not affected by physics-related content or the number of independent variables. The test showed the potential to serve as a measurement tool for assessing CVS in basic physics at the upper secondary education level.