Using propensity scores to adjust for selection bias when assessing the effectiveness of Alcoholics Anonymous in observational studies

Abstract

Background The effectiveness of Alcoholics Anonymous (AA) is difficult to establish. Observational studies consistently find strong dose–response relationships between AA meeting attendance and abstinence, and the only experimental studies favoring AA have been of 12-step facilitation treatment rather than of AA per se. Pending future randomized trials, this paper uses propensity score (PS) method to address the selection bias that potentially confounds the effect of AA in observational studies. Method The study followed a treatment sample for 1 year to assess post-treatment AA attendance and abstinence ( n = 569). Propensity scores were constructed based on known confounders including motivation, problem severity, and prior help-seeking. AA attendance during the 12-month follow-up period was studied as a predictor of alcohol abstinence for 30 days prior to the follow-up interview. PS stratification and PS matching techniques were used to adjust for the self-select bias associated with respondents’ propensity to attend AA. Results The overall advantage in abstinence initially observed narrowed when adjusted. The odds ratio associated with AA attendance reduced from 3.6 to 3.0 after PS stratification and 2.6 after PS matching to AA-attenders. Support for AA effectiveness was strengthened in the quintile with lower propensity scores and when AA-nonattenders were matched as the target group, but was weakened among those in the higher PS quintiles and when matching to AA-attenders. Discussion These results confirm the robustness of AA effectiveness overall, because the results for higher abstinence associated with AA attendance following propensity score adjustment remained significant, and the reduction in the magnitude of AA's effect was moderate. However, the effect modification by propensity scores in both PS stratification and PS matching approaches seems to suggest that AA may be most helpful, or matter more, for those with a lower propensity to attend AA. Conversely, for those with a high propensity to go to AA (operationalized as higher motivation, greater problem severity, more prior AA and treatment exposure, etc.), attending AA may not make as much of a difference. It will be important that future studies replicate our results, as this is the first paper to use propensity score adjustment in this context.