Is It Time for Medicine-Based Evidence?

Abstract

IN THE CURRENT ERA OF EVIDENCE-BASED MEDICINE (EBM), clinicians are expected to use the best available evidence in patient care. Five examples of comparative effectiveness research investigations using observational research methods are included in this theme issue of JAMA. These articles provide new evidence for clinicians to consider, with important messages for each topic under investigation. The studies have in common the potential difficulty of understanding methods of the study designs and analyses—specifically propensity scores and instrumental variables. The concepts, complexity, contributions, and accompanying cautions involving these 2 analytic methods are considerable. Of most relevance to clinicians, the articles also highlight general issues encountered when interpreting the medical literature. Although EBM involves finding as well as evaluating evidence, and even though more recent considerations of EBM have modified prior rigid approaches, a generation of clinicians has been taught that randomized controlled trials (RCTs) can find truth, whereas observational studies are inherently flawed. The 5 articles in this issue of JAMA can be best understood and appreciated when viewed through this prism. Observational studies inherently have a challenge not shared by RCTs: any differences in characteristics of groups being compared, if those characteristics also affect outcomes of interest, may threaten the validity of the treatment-outcome relationship or other associations. Usually referred to by terms such as confounding or susceptibility bias, the main problem is the potential for incorrect inference. In this context, a relevant focus is on the application and interpretation of methods of multivariable analysis. These statistical methods provide adjusted results when examining exposure-outcome associations. For example, multivariable analysis can account for an imbalance in age when comparing groups that received different treatments of interest and also differed in age distribution in terms of longterm survival. Conversely, RCTs can be analyzed on an unadjusted basis by comparing actual outcomes directly if randomization yields compared groups with balanced clinical features (such as age). Many methods of multivariable analysis were developed by statisticians, and only later incorporated into medical research as perceived needs were recognized. Almost a halfcentury ago, analyses from the Framingham Heart Study were a notable early example of multivariable methods in the medical literature. Contemporary commonly used methods include multiple linear regression or analysis of covariance for continuous (dimensional) outcomes, logistic regression for binary (dichotomous) variable outcomes, proportional hazards analysis or Cox regression when a time interval is relevant to a binary outcome (ie, survival analysis), and Poisson regression when outcomes are measured as counts. Whether results are reported as relative risks, odds ratios, hazard ratios, or other formats, the association of one variable with the outcome is calculated mathematically “within the context of other variables, while [the other variables] are held constant.” Comparative effectiveness research, especially when conducted with observational studies, will likely promote increased use of more specialized analytic techniques such as propensity scores and instrumental variables. The propensity score approach involves 2 steps in multivariable analysis: first, using baseline characteristics to determine how likely participants are to receive the treatments of interest (a propensity score between 0 and 1); and second, using the propensity score to adjust for the assignment of treatment, usually by techniques involving the use of scores to match or stratify participants, or inclusion of the scores in a second multivariable model. For example, the study by Jackson et al in this issue of JAMA determined propensity scores for undergoing open vs endovascular abdominal aortic aneurysm repair based on age, sex, emergency presentation, and other baseline factors. These scores were then entered into another multivariable model that found lower mortality among patients undergoing endovascular repair. The instrumental variable approach identifies an instrument (variable) that is thought to be associated with the treatments of interest but not with the outcome. This method, also involving 2 conceptual components, is said to be analo-