Addressing Uncodable Behaviors: A Bayesian Ordinal Mixture Model Applied to a Mathematics Learning Trajectory Teaching Experiment

Abstract

When researchers code behavior that is undetectable or falls outside of the validated ordinal scale, the resultant outcomes often suffer from informative missingness. Incorrect analysis of such data can lead to biased arguments around efficacy and effectiveness in the context of experimental and intervention research. Here, we detail a new Bayesian mixture approach that analyzes ordinal responses with undetectable/uncodable behaviors in two stages: (1) estimate a likelihood of response detection and (2) estimate an Explanatory Item Response Model for the ordinal variable conditional on detection. We present an independent random effects and correlated random effects variant of the new model and demonstrate evidence of model functionality using two simulation studies. To illustrate the utility of our proposed approach, we describe an extended application to data collected during a length measurement teaching experiment (N = 186, 56% girls, 5-6 years at preassessment). Results indicate that students assigned to a learning trajectories instructional condition were more likely to use detectable, mathematically relevant problem-solving strategies than their peers in two comparison conditions and that their problem-solving strategies were also more sophisticated.