Individual Differences and Test Administration Procedures: A Comparison of Fixed-Item, Computerized-Adaptive, and Self-Adapted Testing

Abstract

In this study, we compared vocabulary fixed-item, computerized-adaptive, and self-adapted tests (referred to hereafter as FITs, CATs, and SATs, respectively). In particular, we evaluated three aspects of the tests: (a) their measurement precision and efficiency, (b) the effects of several individual difference variables. (test anxiety, verbal self-concept, computer usage, and computer anxiety) on ability estimates alone and in interaction with the test administration procedures, and (c) examinees' attitudes about various features of the tests they took. We randomly assigned volunteer college students to take a vocabulary test using one of the three test administration procedures. All the tests used the same large, well-calibrated item bank. Results indicate that the CAT was more precise and efficient than the SAT, which was more precise and efficient than the FIT. The SAT yielded higher ability estimates than the other tests for individuals with lower verbal self-concepts. Examinees indicated by their evaluations of elements of the testing procedures that they prefer tests in which they can have as much control and as much information as possible. Taken collectively, these results indicate that no single test administration procedure simultaneously maximizes precision, efficiency, validity, and examinee satisfaction.