PRM113 - Comparing propensity score, propensity score with covariates and genetic algorithm methods for covariate matching in observational studies

Abstract

As the population ages an increasing number of individuals are providing informal (unpaid) care for an aging relative. We compare three different methods of covariate matching to determine the effect of caregiving on the mental health states of informal caregivers. Covariate matching methods pair observations from different treatment groups by matching the members of each pair on a set or vector of covariates that would be randomly distributed across the groups in a randomized trial. Multiple waves of an online survey conducted among a representative sample of U.S. adults yielded 740 informal caregivers and 2260 non-caregivers. We applied three different methods for covariate matching to determine the “average effect of treatment on the treated” (ATT) of caregiving on mental health states (MH): 1. Propensity score within calipers; 2. Propensity score and covariates within calipers; and 3. Genetic algorithm matching. All three methods provide adequate balance on the covariates used for matching. Methods 2 and 3 produce the best covariate balance, with absolute mean covariate differences less 0.0008 on all covariates and less than 0.00001 on the core set of covariates. Because methods that censor observations (i.e. matching within calipers) may artificially improve covariate balance, we take the ATT estimate from genetic matching to be the least biased estimate of the true effect. Using a standard 5-point self-report measure of mental health, caregivers, on average, report a mental health state that is 5.4% worse than non-caregivers (roughly one-fourth “less healthy” within any given scale range (e.g. 2-3, 3-4). As all three methods provide adequate matching, our consideration turns to bias reduction and the fact that the genetic matching does not require that we estimate the propensity score prior to matching. We consider the drivers or caregiver MH and implications for health care policy.