Quantifying potential selection bias in observational research: simulations and Analyses exploring religion and depression using a prospective UK cohort study (ALSPAC)

Abstract

ABSTRACT A major problem in research is non-random participation, which can occur either at recruitment into, or loss to follow-up during, the study. In both cases the analytic sample differs from the target population, potentially resulting in selection bias and erroneous inferences. We perform simulations and analyses using data from ALSPAC (Avon Longitudinal Study of Parents and Children) to quantify the potential impact of selection bias. Our motivating example focuses on religion (the exposure) and depression (the outcome). ALSPAC was broadly representative of the target population of pregnant mothers during recruitment, but continued participation after 30 years is non-random, including by factors such as religion and mental health. Despite non-random participation by the exposure and outcome (i.e., collider stratification selection bias), given plausible selection scenarios our simulations found little bias across all models with broadly nominal (i.e., 95%) confidence interval coverage. Analyses comparing the full cohort (i.e., the target population) against a highly-selected sample of participants still participating after 30 years also suggested minimal bias. Given the strengths and patterns of selection explored in this study, the resulting bias could be negligible. However, this cannot be assumed to hold for all studies and must be explored on a case-by-case basis.