Abstract 171: User Beware: Understanding the Adjusted Estimates of Propensity Score Methods

Abstract

Background: Propensity score methods are used in observational studies to compensate for the lack of random allocation by balancing measured baseline characteristics between treated and untreated patients. Yet default implementations of propensity score methods may lead to contradictory findings. Our goal was to illustrate differences in treatment effect estimates produced using propensity score methods such as matching and inverse probability weighting (IPW). Methods: We used data from a retrospective analysis of the Northern New England Cardiovascular Disease Study Group registry that studied reintervention after single internal mammary artery (SIMA) versus bilateral internal mammary artery (BIMA) conduit use in 41,481 coronary artery bypass grafting (CABG) procedures from 1992-2014. Results: The mean duration of follow-up was 13.2 (IQR: 7.4-17.7) years. In our standard multivariable Cox regression analysis, the adjusted HR for reintervention was 0.83 (95% confidence interval (CI): 0.75-0.92) in patients receiving BIMA compared to SIMA (Table). The 1:1 propensity matched analysis (HR=0.79, 95% CI: 0.69-0.91) and IPW estimate among the treated (HR=0.83, 95% CI: 0.75-0.92) show a similar protective treatment effect of BIMA use on reintervention. However, the IPW approach for the overall population effect unusually showed no difference between BIMA and SIMA on reintervention-free survival (HR=1.08, 95% CI: 0.94-1.24). Conclusions and Relevance: The adjusted HR for reintervention after BIMA versus SIMA use in CABG was remarkably different for the two IPW estimates. This difference is attributed to the population represented by the two IPW approaches- the IPW among the treated estimate represents the effect in the average study population, whereas the IPW in the treated represents the effect in the treated population alone. Determining how the study population is balanced (weighted to look like the overall study population versus treated group) is a large driver of treatment effect, and a key element of propensity score methods. Ultimately, the treatment effect estimate desired should drive the choice of propensity score adjustment method.