Examining the Psychometric Properties of the Redox Concept Inventory: A Rasch Approach

Abstract

The current study examined the psychometric properties of scores from the Redox Concept Inventory (ROXCI) scores using both confirmatory factor analysis (CFA) and Rasch analysis. CFA analysis provides evidence for construct validity, and Rasch analysis demonstrates the performance of ROXCI items in terms of their accuracy in estimating person ability as well as providing person ability estimates for the distractor analysis. Our work provides some insights into ROXCI items that were not presented in the original development paper. Results showed that ROXCI is an appropriate instrument to measure student understanding of redox reactions at the introductory college level based on the sample data, because ROXCI items were especially useful for discriminating between students with latent abilities below +0.5 logits but not as much for students with higher latent abilities. Based on the trace plots, distractor analysis found improved scores for 12 out of 18 items after the instruction on the redox reaction with small to medium effect size, little change in scores for 2 items, and no improved scores for 4 items, which were the items with nonsignificant factor loadings in the CFA analysis. In addition, item feature analysis was performed for the first time for items in chemistry concept inventories to evaluate the effects of item features on item performance, which provided some useful information for test developers in the future. Results showed that the item type had significant effects on the item performance after controlling for the person feature effect.