Bias in the study of prediction of change: a Monte Carlo simulation study of the effects of selective attrition and inappropriate modeling of regression toward the mean.

Kristin Gustavson,Ingrid Borren

doi:10.1186/1471-2288-14-133

Kristin Gustavson, Ingrid Borren

Open Access

https://doi.org/10.1186/1471-2288-14-133

Copy DOI

Abstract

BackgroundMedical researchers often use longitudinal observational studies to examine how risk factors predict change in health over time. Selective attrition and inappropriate modeling of regression toward the mean (RTM) are two potential sources of bias in such studies.MethodThe current study used Monte Carlo simulations to examine bias related to selective attrition and inappropriate modeling of RTM in the study of prediction of change. This was done for multiple regression (MR) and change score analysis.ResultsMR provided biased results when attrition was dependent on follow-up and baseline variables to quite substantial degrees, while results from change score analysis were biased when attrition was more strongly dependent on variables at one time point than the other. A positive association between the predictor and change in the health variable was underestimated in MR and overestimated in change score analysis due to selective attrition. Inappropriate modeling of RTM, on the other hand, lead to overestimation of this association in MR and underestimation in change score analysis. Hence, selective attrition and inappropriate modeling of RTM biased the results in opposite directions.ConclusionMR and change score analysis are both quite robust against selective attrition. The interplay between selective attrition and inappropriate modeling of RTM emphasizes that it is not an easy task to assess the degree to which obtained results from empirical studies are over- versus underestimated due to attrition or RTM. Researchers should therefore use modern techniques for handling missing data and be careful to model RTM appropriately.Electronic supplementary materialThe online version of this article (doi:10.1186/1471-2288-14-133) contains supplementary material, which is available to authorized users.

Highlights

Medical researchers often use longitudinal observational studies to examine how risk factors predict change in health over time
Selective attrition and inappropriate modeling of regression toward the mean (RTM) biased the results in opposite directions
Analyses were run with a 50% attrition rate and with inappropriate modeling of RTM, in the same way as the analyses reported in Tables 3 and 4

Summary

Introduction

Medical researchers often use longitudinal observational studies to examine how risk factors predict change in health over time. Selective attrition and inappropriate modeling of regression toward the mean (RTM) are two potential sources of bias in such studies. Medical researchers often use longitudinal studies to examine how one variable predicts change in a health measure over time. A much discussed source of bias in longitudinal studies is selective attrition [1] If those who stay and those who drop out of a study differ regarding the phenomena of interest, results may be biased. An additional source of bias in the study of prediction of change is regression toward the mean (RTM). The current study will use Monte Carlo computer simulations to examine bias related to selective attrition and to RTM in the study of prediction of change. For simplicity of the presentation, all examined variables will be standardized with a standard deviation (SD) of 1

Objectives

Methods

Results

Discussion

Conclusion