Comparing Alternate Approaches to Calculating Reliability for Dichotomous Data: The Sample Case of Adolescent Selection, Optimization, and Compensation

Abstract

Estimating reliability for scales or factors is an essential data analysis step in much of the research in developmental science. In this article, we demonstrate the importance of using the appropriate statistical method and underlying correlation matrix to estimate reliability for dichotomous data that represent a normally-distributed latent factor. We used an example case of three waves of adolescent data collected from responses to the Selection, Optimization, and Compensation questionnaire (SOC; Freund & Baltes, 2002) of intentional self-regulation to illustrate how calculating composite reliability (or ω) using tetrachoric correlations provides a more accurate estimate of reliability when compared to both raw covariance-based ω, as well as raw covariance-based and tetrachoric correlation-based Cronbach's α approaches. In addition, we describe methods for calculating each of these approaches to reliability estimation, and we offer suggestions for future researchers for estimating reliability for such dichotomous data.