Least Squares Distance Method of Cognitive Validation and Analysis for Binary Items Using Their Item Response Theory Parameters

Abstract

The validation of cognitive attributes required for correct answers on binary test items or tasks has been addressed in previous research through the integration of cognitive psychology and psychometric models using parametric or nonparametric item response theory, latent class modeling, and Bayesian modeling. All previous models, each with their advantages and disadvantages, require item score information and do not focus on conditional validation of cognitive attributes across ability levels and individual test items. This study proposes a method of estimating the probability of correct performance on cognitive attributes across fixed ability levels. The proposed method, referred to here as the least squares distance method (LSDM), is based on the minimization of matrix norms using the Euclidean least squares distance. The LSDM does not require raw or trait scores of examinees as long as IRT estimates of the item parameters are available.