Does Choice of a Criterion Matter?

Abstract

For reasons of objectivity and job relevance, a hands-on performance test (HOPT) has been advocated as the best measure to use as a validation criterion. However, the cost of HOPTs has inspired a search for surrogate measures such as job knowledge tests (JKTs). This study questions whether choice of criteria makes a difference for the purpose of developing an accurate selection composite. Hanser's (1990) research using Army Project A data found that use of surrogates in place of a HOPT made little difference for the selection accuracies of predictor composites of the Armed Services Vocational Aptitude Battery (ASVAB). This study uses Marine Corps Job Performance Measurement (JPM) data to replicate Hanser's results. It also questions whether a criterion is needed at all for developing an accurate selection composite. It demonstrates that predictor composites such as Ree and Earles's (1990a) psychometric g and unit-weighted regression coefficients can be as useful as those developed using an outcome criteri...