Logistic regression frequently outperformed propensity score methods, especially for large datasets: a simulation study

Abstract

ObjectivesIn observational studies, researchers must select a method to control for confounding. Options include propensity score (PS) methods and regression. It remains unclear how dataset characteristics (size, overlap in PSs, and exposure prevalence) influence the relative performance of the methods. Study Design and SettingA simulation study to evaluate the role of dataset characteristics on the performance of PS methods, compared to logistic regression, for estimating a marginal odds ratio was conducted. Dataset size, overlap in PSs, and exposure prevalence were varied. ResultsRegression showed poor coverage for small sample sizes, but with large sample sizes was relatively robust to imbalance in PSs and low exposure prevalence. PS methods displayed suboptimal coverage as overlap in PSs decreased, which was exacerbated at larger sample sizes. Power of matching methods was particularly affected by a lack of overlap, low exposure prevalence, and small sample size. The advantage of regression for large data size was reduced in sensitivity analysis with a complementary log–log outcome generation mechanism and unmeasured confounding, with superior bias and error but inferior coverage to matching methods. ConclusionDataset characteristics influence performance of methods for confounder adjustment. In many scenarios, regression may be the preferable option.