How Adaptive Is an Adaptive Test: Are All Adaptive Tests Adaptive?

Abstract

The primary distinguishing feature of computerized adaptive testing (CAT) is that sets of items that are administered to examinees are specifically selected for each examinee during the test process. This implies that examinees who respond differently to items on a test will get different sets of test items. However, if an item bank is small, if there are strict content constraints, or if there is strong exposure control, there might not be much adaptation during the test. Examinees who perform differently might get many of the same test items. The research reported here recommends some statistical indicators of how much adaptation is taking place and shows how these indicators vary for different kinds of adaptive test designs. Guidelines are provided for the values of the statistics indicating that a CAT is strongly adaptive.