Testing Finite State Models of Performance in Objective Tests Using Items with ‘None of the Above’ as an Option

Abstract

Finite state test theory provides equations for the probability of every response outcome that occurs under any format of administration of an objective test. By changing the assumptions about item characteristics and examinee response behavior, different models realizing the theory can be set each of which prescribes a different scoring formula. Since the assumptions that apply best to a real testing situation are not always known in advance, the appropriateness of each of a set of models needs to be determined, in order to decide on the way the test should be scored. This paper presents a method on which to base this decision. Responses to items which had ‘none of the above’ as an option are used to illustrate it. Several models are put forth which make different assumptions as to item characteristics and examinee response behavior. The fit of data to the predicted theoretical relationships among response outcomes is the criterion for deciding on the most suitable finite state model. This procedure for determining the fit of data to each of a set of finite state models forms the basis for choosing item characteristic curves for use with finite state poiynomic IRT models.