Psychometric Properties of Finite-State Scores Versus Number-Correct and Formula Scores: A Simulation Study

Abstract

As developed by García-Pérez (1987), finite-state scores are nonlinear transformations of the proportions of conventional multiple-choice responses that are cor rect, incorrect, and omitted. They estimate the propor tions of item alternatives which the examinees had the knowledge needed to classify (as correct or incorrect) before seeing them together in the items. The present study used simulation techniques to generate conven tional test responses and to track the proportions of al ternatives the examinees could classify independently before taking the test and the proportions they could classify after taking the test. Then the finite-state scores were computed and compared with these actual values and with number-correct and formula scores based on the conventional responses. Highly favorable results were obtained leading to recommendations for the use of finite-state scores. These results were al most the same when the simulation proceeded accord ing to the model and when it was based on a natural istic process completely independent of the model. Hence the scoring procedures on which finite-state scores are based are both accurate and robust. Index terms: applied measurement models, examinee behav ior, finite-state scores, guessing, multiple-choice tests, test scoring.