Computing Multivariate Effect Sizes and Their Sampling Covariance Matrices With Structural Equation Modeling: Theory, Examples, and Computer Simulations.

Mike W.-L Cheung

doi:10.3389/fpsyg.2018.01387

Abstract

In the social and behavioral sciences, it is recommended that effect sizes and their sampling variances be reported. Formulas for common effect sizes such as standardized and raw mean differences, correlation coefficients, and odds ratios are well known and have been well studied. However, the statistical properties of multivariate effect sizes have received less attention in the literature. This study shows how structural equation modeling (SEM) can be used to compute multivariate effect sizes and their sampling covariance matrices. We focus on the standardized mean difference (multiple-treatment and multiple-endpoint studies) with or without the assumption of the homogeneity of variances (or covariance matrices) in this study. Empirical examples were used to illustrate the procedures in R. Two computer simulation studies were used to evaluate the empirical performance of the SEM approach. The findings suggest that in multiple-treatment and multiple-endpoint studies, when the assumption of the homogeneity of variances (or covariance matrices) is questionable, it is preferable not to impose this assumption when estimating the effect sizes. Implications and further directions are discussed.

Highlights

This study shows how structural equation modeling (SEM) can be used to compute multivariate effect sizes and their sampling covariance matrices
Since there were two effect sizes for two treatment groups, we reported the average of their absolute biases B(θ) = B(θ)T1 + B(θ)T2 /2, where B(θ)T1 and B(θ)T2 are the absolute biases for treatments 1 and 2, for ease of presentation
The results were summarized in the heat maps, which provide an easy way to visualize the performance of the statistics

Summary

A SEM APPROACH TO ESTIMATING EFFECT SIZE

Cheung (2015a, Chapter 3) presents a SEM approach to estimating various effect sizes, including those in multipletreatment and multiple-endpoint studies. When the assumption of the homogeneity of variances is questionable, it may not be appropriate to use σCommon in the denominator This is because σCommon is not estimating any of the population SDs. A better alternative is to use the control group σ(C) as the standardizer in calculating the effect sizes. The metaSEM package provides smdMTS() and smdMES() to calculate the effect sizes for a multiple-treatment study and a multiple-endpoint study with or without the assumptions of homogeneity. When the data are generated from populations with equal variances, the effect sizes both with and without the homogeneity assumption should be correct. When the data are generated from unequal population variances, the effect sizes without the homogeneity assumption should still be correct. For the multiple-treatment studies, multivariate normal data were generated from the known data structures with or without the assumption of the homogeneity of variances

Methods

Results

CONCLUSION