Conspiracies and Test Compromise: An Evaluation of the Resistance of Test Systems to Small-scale Cheating

Abstract

Test compromise is a concern in cognitive ability testing because such tests are widely used in employee selection and administered on a continuous basis. In this study, the resistance of cognitive tests, deployed in different test systems, to small-scale cheating conspiracies, was evaluated regarding the accuracy of ability estimation. Conventional tests with different numbers of test forms (one, two, or four) and different lengths (30 or 60 items) were simulated and evaluated. Computerized adaptive testing (CAT) systems employing the same test lengths were simulated using six different item selection methods. Results showed that 60-item tests were more resistant than 30-item tests given the same number of compromised items; when the frequency of test administration was held constant, CAT systems were better at resisting small-scale cheating than conventional test systems. For the conventional tests, lengthier tests and more test forms enhanced test security. Item selection methods had a strong effect on the resistance of CAT to compromise under some conditions. The results of our simulations have practical implications for the types of test systems used for delivering cognitive tests in employee selection.