Air Force Officer Qualifying Test Research Articles

Development of a situational judgment test to supplement current US air force measures of officership

ABSTRACT Aptitude requirements for US Air Force officer commissioning include completion of a college degree and minimum scores on the Air Force Officer Qualifying Test (AFOQT) Verbal and Quantitative composites. Although the AFOQT has demonstrated predictive validity for officer training, the Air Force has striven to improve predictive validity and diversity. To this end, a Situational judgment Test (SJT) was added to the AFOQT in 2015. SJT development was consistent with recommendations to broaden the competencies assessed by the AFOQT with the goal of providing incremental validity, while reducing adverse impact for historically underrepresented groups. To ensure content validity and realism, SJT development was based on competencies identified in a large-scale analysis of officership and input from junior officers in scenario and response generation and scoring. Psychometric evaluations have affirmed its potential benefits for inclusion on the AFOQT. An initial study showed the SJT to be perceived as highly face valid regardless of whether it was presented as a paper-and-pencil test (with narrative or scripted scenarios) or in a video-based format. Preliminary studies demonstrated criterion-related validity within small USAF samples, and a larger Army cadet sample. Additionally, operational administration of the SJT since 2015 has demonstrated its potential for improving diversity (i.e., reduced adverse impact relative to the AFOQT Verbal and Quantitative composites). Predictive validation studies with larger Air Force officer accession samples are ongoing to assess the incremental validity of the SJT beyond current AFOQT composites for predicting important outcomes across accession sources.

Factor Structure and Criterion-Related Validity of the Air Force Officer Qualifying Test Form T for Pilot Applicants.

The Air Force Officer Qualifying Test (AFOQT) is the primary selection tool for officer applicants in the U.S. Air Force (USAF) for nearly seven decades. The AFOQT is revised and modified periodically, with rigorous equating and linking effort to ensure comparability and connectivity across forms. The most recent version of AFOQT is Form T that includes 10 cognitive ability and knowledge subtests. Despite the continuing validation effort of the AFOQT across forms, it was mostly directed to the general population of officer applicants, but not to any specific subpopulation. The current investigation reported three studies in an attempt to provide evidence for factor structure and criterion-related validity of AFOQT Form T for pilot applicants via four analytical approaches: meta-analysis, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and structural equation modeling (SEM). The results suggested that AFOQT Form T data are best represented by a bifactor model with a general ability and four specific abilities, and that each latent construct has a distinct predictive utility for pilot performance criteria.

Predictive Validity of the Air Force Officer Qualifying Test (AFOQT) for Pilot Performance

Abstract. The Air Force Qualifying Test (AFOQT) has been the primary selection test battery for officer candidates in the US Air Force since 1953. Despite a wealth of literature on the validity of the AFOQT in predicting pilot performance, there is less evidence on its validity generalization. This study investigated the predictive validity of 16 AFOQT subtests and its Pilot composite via psychometric meta-analytic procedures. Based on 32 independent samples from 26 studies, results indicated that pilot performance is best predicted by subtests indicative of perceptual speed, aviation-related aptitude and knowledge, and quantitative ability constructs, and least predicted by subtests indicative of verbal ability construct. Evidence for validity generalization of AFOQT subtests is presented, and implications for practical use are discussed.

Seek and Ye Shall Find

Being familiar with their earlier work investigating the factor structures of the Armed Services Vocational Aptitude Battery and the Air Force Officer Qualifying Test, we read with interest Ree, Carretta, and Teachout's (2015) proposal to extend the idea of a dominant general factor (DGF) beyond the realm of cognitive abilities to other areas of research and practice in industrial–organizational (I-O) psychology. We found their ideas intriguing and their arguments compelling, but we stumbled on a reference to an article of one of the present authors (Lance, Teachout, & Donnelly, 1992) and Ree et al.’s claim that Lance et al. (1992) had found a DGF that accounted for 59% of the variance in job performance ratings in a military job. They did not. Rather, Lance et al. reported a hierarchical confirmatory factor analysis (CFA) model that explained parsimoniously the correlations among 15 job performance first-order factors in terms of four Job Proficiency and four other Measurement Source second-order factors. We reasoned that Ree et al. must have conducted some secondary analysis on the results presented by Lance et al., and indeed we replicated their claim by finding that the first unrotated principal component accounted for 59% of the variance in correlations among the four Proficiency second-order factors reported in Lance et al.’s Table 6.

Predictive Validity of Knowledge Tests for Pilot Training Outcome

Tests of subject-specific knowledge (mathematics, physics, and English as a foreign language for non-English-speaking countries) are commonly used in test batteries for the selection of ab initio pilots – for example, the Air Force Officer Qualifying Test (AFOQT) or the German Aerospace Center (DLR) assessment. However, in validity research, knowledge tests are often underrepresented. This study evaluated the predictive validity of knowledge tests compared with cognitive ability tests and school grades. The validity criterion was the outcome of pilot training (pass/fail) of a preselected group of applicants (N = 402) who completed a 2-year flight training program. The predictive validity of the entire test battery was r = .55. Cognitive ability tests, knowledge tests, and school grades emerged as comparably valid predictors. These findings are discussed in the framework of Cattell’s theory of fluid and crystallized intelligence, suggesting that knowledge tests are predictively valid because they are indicators of motivation and of being a good learner.

Predictive Validity of Pilot Selection Instruments for Remotely Piloted Aircraft Training Outcome

Demand for remotely-piloted aircraft (RPA) support has increased dramatically over the last decade. Initial efforts to meet the demand focused on cross-training experienced manned aircraft pilots and funneling recent Specialized Undergraduate Pilot Training (SUPT) graduates to RPA pilot training. This approach reduced the number of personnel available for manned airframes and is no longer sustainable. In 2009, the USAF established an RPA career field and the Undergraduate RPA Training (URT) course to train officers with no prior flying experience to be RPA pilots. URT selection methods are very similar to those for SUPT. Some important factors for URT applicants are medical flight screening and aptitude tests [Air Force Officer Qualifying Test (AFOQT) and Pilot Candidate Selection Method (PCSM)]. The current study examined the predictive validity of the AFOQT pilot and PCSM composites for URT completion. Subjects were 139 URT students with AFOQT and PCSM scores. The training criterion was URT pass/fail and the pass rate was 74.8%. Both the AFOQT pilot (r = 0.378) and PCSM (r = 0.480) composites demonstrated good predictive validity. No minimum qualifying PCSM score exists for URT. Had a minimum PCSM score of 25 been used, the pass rate would have been 80.2%; 12 more eliminees would have been screened out compared with the current AFOQT pilot minimum qualifying score of 25. Although current selection methods are effective, based on results of several RPA job/task analyses, the Air Force is examining the utility of other measures to supplement current methods.

Predictive Validity of the Air Force Officer Qualifying Test for Non-Rated Officer Specialties

The Air Force Officer Qualifying Test (AFOQT) is used to qualify applicants for officer commissioning and aircrew training programs. The current study examined its predictive validity for 14 officer technical training courses for which there are no additional AFOQT minimum qualification requirements beyond those for officer commissioning. Sample sizes ranged from 16 to 2,190 with a mean and median size of 753 and 319 officers. Ninety percent (63 of 70) of the observed correlations between the AFOQT composites and average technical training grades were statistically significant. Meta-analyses were conducted to determine whether the AFOQT validities were generalizable across training specialties. Analyses were done on the observed data, after correction for range restriction, and after correction for both range restriction and criterion unreliability. The pattern of validities was similar for all three meta-analyses. The lower bound of the 95% confidence interval and the 95% credibility interval around the weighted mean validities were greater than zero for all five AFOQT composites supporting its utility for making personnel selection decisions for these jobs. The Verbal composite had the lowest and the Academic Aptitude composite had the highest weighted mean validity. The weighted mean of the validity coefficients across training specialties ranged from .261 to .326 for the observed data, .322 to .387 for the range-restriction corrected data, and .360 to .433 for the fully corrected data. Additional efforts are required to examine the generalizability of the results for a broader range of occupational specialties and to set minimum qualifying scores.

Factor Structure of the Air Force Officer Qualifying Test Form S: Analysis and Comparison with Previous Forms

Due to its importance for assignment and classification in the U.S. Air Force, the Air Force Officer Qualifying Test (AFOQT) has received a substantial amount of research. Recently, the AFOQT was revised to reduce administrative burden and test-taker fatigue. However, the new version, the AFOQT Form S, was implemented without explicitly examining the latent structure of the exam. The current study examined the factor structure of Form S as well as its measurement equivalence across race- and sex-based groups. Results indicated that a bifactor model with a general intelligence factor and five content-specific factors fit the best. The measurement equivalence of the AFOQT across gender and racial/ethnic groups was also supported.

Gender and Racial Equity of the Air Force Officer Qualifying Test in Officer Training School Selection Decisions

Using Cleary's (1968) model of test bias, relations between aptitude scores and training performance were evaluated for race and gender subgroups of military officer candidates. Regression analyses revealed level bias with minority performance being overpredicted by a small and constant amount at all aptitude levels, suggesting that test usage results in higher selection rates for female and Black cadets. These results are consistent with the literature in education, industry, and prior studies conducted in the military.

Factor Structure of the Air Force Officer Qualifying Test: Analysis and Comparison

The Air Force Officer Qualifying Test (AFOQT) is used to qualify men and women for commissions in the Air Force, classify them for pilot and navigator jobs, and award Reserve Officer Training Corps (ROTC) scholarships. Despite more than three decades of use, little published literature exists outside of Air Force technical reports, which do not receive wide distribution. One of the most important details about a test battery is which factors it measures. To determine this, several factor models were tested with structural equations. Most of the models were hierarchical with general cognitive ability (g) as the highest factor. A model with hierarchical g and the five lower order factors of verbal, math, spatial, aircrew, and perceptual speed fit the data best. The factor structure of the AFOQT was compared to the factor structure of the Armed Services Vocational Aptitude Battery (ASVAB), the enlistment qualification test battery. The AFOQT was found to contain a greater number of factors than did the ASVAB...

Role of ability and prior knowledge in complex training performance.

A causal model of the role of general cognitive ability and prior job knowledge in subsequent job-knowledge acquisition and work-sample performance during training was developed. Participants were 3,428 U.S. Air Force officers in pilot training. The measures of ability and prior job knowledge came from the Air Force Officer Qualifying Test. The measures of job knowledge acquired during training were derived from classroom grades. Work-sample measures came from check flight ratings. The causal model showed that ability directly influenced the acquisition of job knowledge. General cognitive ability influenced work samples through job knowledge. Prior job knowledge had almost no influence on subsequent job knowledge but directly influenced the early work sample. Early training job knowledge influenced subsequent job knowledge and work-sample performance. Finally, early work-sample performance strongly influenced subsequent work-sample performance.

Armed Services Vocational Aptitude Battery and Air Officer Qualifying Test: Analyses of Common Attributes

The United States Air Force has separate enlisted and officer selection and classification systems. The Armed Services Vocational Aptitude Battery (ASVAB) is used for selecting and classifying occupational specialties for enlisted personnel. The Air Force Officer Qualifying Test (AFOQT) assists in the selection and classification of officers. This study evaluated the 10 ASVAB subtests and the verbal and quantitative subtests of the AFOQT for common properties. Airmen (N = 516) in basic military training (BMT), who had been administered the ASVAB prior to enlistment, were administered the subtests of the Verbal, Quantitative, and Academic Aptitude composites of the AFOQT Form 0. Analyses were conducted to describe and evaluate the relations between the two tests. A correlation matrix containing ASVAB subtests and AFOQT verbal and quantitative subtests was computed and corrected for restriction in range. Using this corrected matrix, ordinary and canonical correlation analyses were conducted. Results of the analyses indicate that both tests measure similar attributes. The findings have implications for operational test use.

Development of a Causal Model of Processes Determining Job Performance

6. M. Morales, The function of general and spe cific abilities in the validity of the Air Force Officer Qualifying Test, unpublished master's thesis, St. Mary's University of Texas, San Antonio (1991). 7. M.J. Ree, J.A. Earles, and M. Teachout, Gen era/ Cognitive Ability Predicts Job Performance, AL TR-1991-0057 (Armstrong Laboratory, Brooks AFB, TX, 1991). 8. N. Stewart, A.G.C.T. scores of Army person nel grouped by occupation, Occupations, 26, 1-37 (1947). It is interesting to note that job hierarchies are not new and that Deuteronomy 29:9 presents a list with managers and overseers at the top and hew ers of wood at the bottom. A.R. Jensen noted that even relatively mundane tasks such as cooking rely on intelligence: Army cooks who could successfully make jelly rolls had higher scores on g-loaded tests than those who could not; Straight Talk About Men tal Tests (Free Press, New York, 1981). 9. R.J. Sternberg, Death, taxes, and bad intelli gence tests, Intelligence, 15, 257-269 (1991). There are a few people who want competency tests, which are even more content specific than typical aptitude tests. The rationale for competency tests has been dispelled by G.V. Barrett and R.L. Depinet, A re consideration of testing for competence rather than for intelligence, American Psychologist, 46, 1012? 1024(1991). 10. R. Cattell, Intelligence: Its Structure, Growth and Action (Elsevier Science Publishing, New York, 1987). This is a revised version of Cattell's Abilities: Their Structure, Growth and Action, published in 1971. In this revision, he elucidates the investment theory and his concepts of fluid and crystallized in telligence. Note the significant change in the title from the original.

Cross Validation of Experimental USAF Pilot Training Performance Models

Abstract : A series of studies have indicated that individual differences in hand-eye coordination, information processing ability, personality and attitudes are related to USAF pilot training performance. The current investigation was designed to cross-validate these results. Eight hundred eighty-five (885) USAF Undergraduate Pilot Training (UPT) students were divided randomly into two groups. Pilot selection models that used a combination of Air Force Officer Qualifying Test (AFOQT) and Basic Attributes Test (BAT) battery scores were developed independently for each group and then cross-validated with the other group. In the model development phase, subjects with good hand-eye coordination who made quick decisions were more likely to complete UPT successfully in both groups. Although there was some reduction in the validity coefficients in the cross-validation phase, the selection models were related significantly to UPT final outcome in both groups. These results suggest that the AFOQT/BAT pilot selection models are sufficiently robust to be used as adjuncts to operational USAF pilot trainee selection procedures. Keywords: Aptitude testing, Basic attributes test, Pilot selection.

The Basic Attributes Tests (BAT) System: A Preliminary Evaluation of Three Cognitive Subtasks

In 1955, the U.S Air Force discontinued apparatus-based testing as a component of its aircrew selection and classification system due to administrative problems. Since then, the Air Force has relied on paper and pencil test batteries such as the Air Force Officer Qualifying Test to select pilot and navigator trainees. Unfortunately, the aircrew selection system without apparatus testing failed to produce acceptible attrition rates in light of escalating training costs. As a result, a computer based testing system, the Basic Attributes Tests (BAT) system, was developed to assess psychomotor skills as well as a variety of psychological and cognitive attributes that are believed to be related to flight training performance. This paper evaluated three subtests used to assess cognitive abilities: Perceptual Speed (information input efficiency), Decision Making Speed (low level cognitive and high level sensory-perceptual motor involvement) and Item Recognition (short-term memory storage, search and comparison operations). Each of the subtests was evaluated in terms of its internal consistency and ability to predict flight training performance. An integrated model, based on results from the three cognitive subtests, was evaluated against flight training performance criteria.