Retest Performance on an Experimental Computer-Based Pilot Aptitude Test Battery

Abstract

Abstract : Basic Attributes Test (BAT) is a computer based battery that contributes to a US Air Force pilot selection composite known as the Pilot Candidate Selection Method (PCSM). When the BAT was operationally implemented in 1993, 110 retests were allowed because its retest characteristics had not been adequately examined at that point. Results from a recent study of BAT retest characteristics (mean score change, reliability) concluded that a BAT retest could be permitted no less than six months after initial testing. The current research effort examined retest performance on several experimental computer based tests that are being considered as candidate replacement tests for the BAT. Participants were 340 paid volunteers who completed the test battery and were retested after two weeks, one month, three months, or six months. Results were consistent with those observed earlier for the BAT. First, the experimental computer based tests demonstrated acceptable reliability. Second, most students showed improvement on retest, regardless of the length of the retest interval. Third, in general, practice effects diminished as the length of the retest interval increased. However, the size and duration of the practice effect varied by test content. A few tests showed performance improvements for the shorter retest intervals (two weeks or one month), but little or no improvements for longer retest intervals (three or six months). Other tests showed moderate to large improvements in performance on retest, even for the six month retest interval. These results suggest that as with the BAT, retests could be performed no sooner than six months after initial testing.