Propensity Score Analysis With Unreliable Covariates: A Comparison of Five Reliability-Adjustment Methods

Abstract

Propensity score analysis (PSA) is a crucial tool for researchers in mitigating selection bias arising from multiple covariates in quasi-experimental studies. Nevertheless, the impact of low-reliability covariates on PSA necessitates careful consideration. This study employs Monte Carlo simulation to assess five methods to adjust propensity scores for unreliability of covariates. The findings reveal that the latent variable model incorporating inclusive factor scores (PSIF) results in the smallest relative bias of treatment effect estimates. Notably, only PSIF consistently provides unbiased treatment effect estimates across all conditions. Furthermore, the study underscores the potential for a misleading covariate balance evaluation when dealing with unreliable covariates, given that treatment effect estimates may be biased even when the covariate balance is perceived as adequate.