Power and Accuracy of the Schmidt and Hunter Meta-Analytic Procedures

Abstract

Many Monte Carlo studies have evaluated the Schmidt and Hunter (S&H) (1977) meta-analytic procedures for combining independent correlation coefficients and their 75% rule for testing the homogeneity of variance. This study is different in using simulated data that are more typical of those data sets included in meta-analyses (Cormwell, 1989) such that sample size, effect size, degree of range restriction, measurement error, and number of correlations were simultaneously manipulated in evaluating the S&H procedures' reliability, Type I and II error, bias, and standard error. Results supported other studies which revealed that estimates of mean rho tend to be accurate even in the presence of small sample sizes, few correlations, and substantial measurement error. The power to detect variance among the rhos, however, is low in most situations; thus, the S&H procedures do not find variance among rhos when it exists (Type II error) and also have a high Type I error rate under all conditions simulated. These results are summarized in terms of the reliability of the S&H procedure in estimating mean rho and the variance of rhos.