Retrospective Psychometrics and Effect Heterogeneity in Integrated Data Analysis: Commentary on the Special Issue.

Abstract

The current special issue of Prevention Science indicates that momentum in using individual participant data (IPD) and integrative data analysis (IDA) to combine and synthesize findings in prevention science has accelerated over the past decade. In this commentary, we focus on two general themes involving methods for harmonizing measures and findings of effect heterogeneity. We describe methods for harmonization as retrospective psychometrics, requiring that we attend to the assumptions necessary for accurate measurement, but adjust our methods given the constraints of working with existing datasets that often involve different measures in different studies. We point to novel approaches for increasing confidence that semantic matching and empirical modeling used in these studies will yield accurate and valid measurements that can be combined in IDA. We also review findings about effect heterogeneity, emphasizing the importance of using etiologic and action theories to identify and evaluate sources of such effects. We note that all of the papers in this issue deserve careful attention, as they illustrate how prevention scientists are approaching the complexities of IDA and exploring novel methods for overcoming its challenges.