A meta-analysis and validity generalization study of a personality test and a general cognitive ability test for measuring managerial aptitude

Abstract

A meta-analysis was conducted based on 24 studies reporting the validity of apersonality test and a general cognitive ability test, both of which are subtests ofa managerial aptitude test, NMAT (New Managerial Aptitude Test). Criteriaused in the studies were typically based on performance appraisals by superiors.Regarding the personality test, the estimated correlation coefficients of Compliance- Leadership (similar to Potency claimed to be a subdimension ofthe Big5 Extraversion) and of Conservative -Innovative (similar to Achievementclaimed to be a subdimension of the Big5 Conscientiousness) were r=0.311 andr=0.259 respectively, after correction of range restriction and imperfectreliabilities. As to the cognitive ability test, the estimated validity coefficient of Cognitive Ability was r=0.257 after correction of range restriction and imperfectreliabilities. In general, the validity of the personality test is higher and thevalidity of the cognitive ability test is lower than those generally reported in the U.S.The possibility of validity generalization was also explored. For the personality test, 45 to 100% of total variance was accounted for by artificialfactors, whereas only 25% was so for the general cognitive ability test. Thus, itwas concluded that the validity of the personality test is more stable than that ofthe cognitive ability test over various factors, such as differences in organizationalculture or in job content.The results were discussed in relation to the nature of job environment inJapan, where behaviors regarded as Organizational Citizenship Behavior (OCB) are more explicitly expected and appraised than in the U.S. Further, thepossibility was considered that the relatively high validity of the personality testmay be partly due to the fact that the predictors had been developed specifically for evaluation of managerial competenciesThe article concludes by pointing out the limitations of this study and futuredirections of the meta-analysis.