Multimodal Likelihoods in Educational Assessment

Abstract

It has been known for some time that item response theory (IRT) models may exhibit a likelihood function of a respondent’s ability which may have multiple modes, flat modes, or both. These conditions, often associated with guessing of multiple-choice (MC) questions, can introduce uncertainty and bias to ability estimation by maximum likelihood (ML) when standard Newton solutions are used. This article evaluates the performance of several maximization methods, including initial (grid) searches probing the function slopes, simulated annealing, exhaustive likelihood evaluation, and the standard Newton algorithm. In extensive studies, involving several million records of both generated and real data, the algorithms were evaluated with respect to precision and speed. Two methods, exhaustive search and grid search, followed by Newton steps, all yielded ML estimates at the required precision. At today’s computer speeds, either of these algorithms may be considered for high-volume response pattern scoring.