The how and why of “negative” research studies

Abstract

In this issue of the Journal, Bryson et al. present a singlecentre study that evaluates the inter-relationship between postoperative delirium, postoperative cognitive dysfunction (POCD), and the apolipoprotein E-e4 (APOE-e4) genetic polymorphism. This prospective cohort study, involving 88 individuals who underwent open abdominal aortic aneurysm repair, has many strong points, including strict inclusion criteria, systematic surveillance for delirium and POCD using valid instruments, and clear documentation of postoperative follow-up. The authors found no statistically significant association of delirium with early POCD (odds ratio [OR] 2.86; 95% confidence interval [CI], 0.99 to 8.27) or with late POCD (OR 2.10; 95% CI, 0.28 to 15.92). They also found no statistically significant association of APOE-e4 with delirium (OR 1.63; CI, 0.59 to 4.56), early POCD (OR 1.33; CI, 0.44 to 4.04), or late POCD (OR 3.64; CI, 0.47 to 28.22). Thus, all of the key study findings were statistically non-significant at the traditional 0.05 level. How then should readers interpret these results? In this article, we present some general approaches both for interpreting these non-statistically significant results and for understanding why the study may have failed to identify associations. For the purpose of our discussion, we focus on the estimated association between APOE-e4 and postoperative delirium (OR 1.63; CI, 0.59 to 4.56). Rather than interpreting this estimate as showing no association (which the study results do not prove), the authors appropriately interpreted this estimate as showing no statistically significant association between APOE-e4 and delirium. However, the key question for most readers is not whether there was a statistically significant association between APOE-e4 and delirium but whether there was a clinically significant association. Man-Son-Hing et al. have proposed an approach for evaluating the clinical significance of study findings. In this approach, readers must consider both the point estimate (OR 1.63) and the confidence interval (0.59 to 4.56). The CI can be interpreted as the plausible range of values for the true odds ratio within the study population. The CI in this study was quite wide, indicating the imprecision of the study results, likely due to the small sample size. Readers can then define a threshold that they will accept as a clinically meaningful result. For example, if the threshold is defined as an OR of C 3.81(the value used in this study for estimating the sample size), the point estimate (1.63) is lower than the threshold, whereas the upper CI (4.56) exceeds the threshold. In this scenario, a clinically significant association is deemed to be ‘‘possible’’. Another approach is to consider primarily the CI for the OR, i.e., 0.59 to 4.56. Based on this CI, readers can confidently rule out population odds ratios \0.6 and [4.6. However, what of all the values in-between? The CI is not ‘‘flat’’, i.e., all of the values within the interval are not equally likely. Values towards the extremes of the interval are less likely than those towards the centre. However, knowing that the values are not equally likely may not be very satisfying to readers D. N. Wijeysundera, MD, PhD (&) Keenan Research Centre, Li Ka Shing Knowledge Institute of St. Michael’s Hospital, 80 Bond Street, Toronto, ON M5B 1W8, Canada e-mail: d.wijeysundera@utoronto.ca