Gathering validity evidence in the development of a new version of the Attitude toward the Subject of Chemistry Inventory (ASCI-UE)

Abstract

Organic chemistry is one of the most feared and failed courses due to its complex and fast-paced nature. Investigating affective metrics that relate to achievement in these courses can be worthwhile, particularly when these metrics show predictive relationships to achievement. Attitude toward chemistry has been investigated utilizing a variety of instruments. We present herein an instrument related to the well-established Attitude toward the Subject of Chemistry Inventory (ASCIv2). This new instrument was developed utilizing The Standards of Psychological and Educational Measurement, which describe five aspects of validity evidence that should be gathered when using instruments in research. Content validity was gathered through consultation with experts. Response process validity was gathered through student interviews. Internal structural validity was collected through confirmatory factor analysis. Relations to other variables were investigated through correlation analysis and structural equation modeling. Consequential validity was studied through measurement invariance testing before comparing scores for subgroups (high- and low-achieving students). Additionally, reliability evidence was tested with Omega coefficients for each of the factors. We showed that all tests and analyses were done with high rigor, and that this new instrument, ASCI-UE, measuring utility and e motional satisfaction, can provide interesting results and implications for research and practice.