P5707An evaluation of the use of propensity scores in cardiovascular literature: a systematic review and recommendations

Abstract

Abstract Background As randomized trials cannot always be conducted, propensity score (PS) methods are frequently used in cardiovascular observational studies. Previous evaluations found insufficient and inaccurate reporting of PS methods in the clinical literature, however; a comprehensive and current evaluation of their use and reporting is thus warranted. Purpose 1) To conduct a systematic review of cardiovascular articles published in high-impact medical and cardiovascular journals to evaluate the use and reporting of PS methods, as well as interpretation of results. 2) To identify key reporting and analysis elements when using PS methods. Methods Cardiovascular articles using PS published between 2010 to 2017 in high-impact medical (5) and cardiovascular (3) journals were reviewed. Included studies used a PS-based method, and focused on cardiovascular diseases, outcomes, interventions, or techniques. Information was extracted on: (1) PS assumptions, (2) variable selection and assessment of model success, (3) specifics of each PS method, and (4) consistency of written interpretation with the causal effect estimated. Each article was evaluated by 2 reviewers. Recommendations for key elements to be included in publications were based on findings and PS methodological literature. Results Of the 296 included articles, most were published in Journal of the American College of Cardiology (83 articles; 28%), followed by Circulation (79 articles; 27%), European Heart Journal (46 articles; 14%), Journal of the American Medical Association (35 articles; 12%), British Medical Journal (30 articles; 10%), New England Journal of Medicine (10 articles; 3%), Annals of Internal Medicine (10 articles; 3%), and Lancet (3 articles; <1%). The most commonly used PS method was matching (53% of studies), followed by a combination of methods (19%), direct adjustment (13%), inverse probability weighting (IPW, 12%), and stratification (3%). Variables in the PS model were predefined in 77% of articles, selected with statistical testing in 17% of articles, or both in 5% of articles (no details for 1% of articles). Balance was not assessed in 16% of papers and 38% of articles that evaluated balance did not report standardized differences, the recommended measure to assess balance. For matching, most studies (85%) conducted a 1:1 match; however, 17% of studies had >50% of unmatched treated (or untreated) patients (based on targeted parameter). Overall, heterogeneity of effects was assessed in 89% of all articles. In 48% of the reviewed articles, however, interpretations of the effect estimates did not correspond to the PS method conducted or described. Conclusions Detailed reporting of PS methods is important to maintain transparency and reproducibility, evaluate the appropriateness of the method, and correctly interpret the results and the population to which they apply. This systematic review describes areas for improvement.