13 - A Comparison of IRT-Based Adaptive Mastery Testing and a Sequential Mastery Testing Procedure

Abstract

This chapter presents a study for the comparison of item response theory-based adaptive mastery testing (AMT) and a sequential mastery testing procedure. Monte Carlo simulation was used to delineate circumstances in which one of the mastery testing procedures might have an advantage over the other. The method used to compare the two variable-length mastery testing procedures, which were AMT and sequential probability ratio test (SPRT), to one another, as well as to a conventional testing procedure, consisted of the following steps. (1) Four item pools were generated in which the items differed from one another to different degrees. (2) The desired mastery level on the proportion-correct metric was converted to the θ metric by means of the test response function from each item pool, as required by the AMT procedure. (3) Item responses were generated for 500 simulated subjects for each of the items in the four item pools. (4) Conventional tests of three different lengths were drawn from the larger item pools; these conventional tests served as item pools from which the SPRT and AMT procedures drew items. (5) The AMT and SPRT procedures were simulated for each of the four different item pool types and the three conventional test lengths. (6) Comparisons were made among the three types of tests, AMT, SPRT, and conventional concerning the degree of correspondence between the decisions made by the three test types and the true mastery status. Further comparisons were made based on the average test length that each test type required to reach its decisions.