9 Roundtable on philosophy of evidence: ‘dimensions of evidence and criteria for standards improvement’

Abstract

Objectives The ‘reproducibility crisis’ revealed the need for a deep reflection on current approaches to evidence evaluation and their reliability (Begley and Ellis,2012; Ioannidis, 2005, Prinz et al(2011). Hidden moderators (Hanin, 2017), publication bias and low power have been identified as possible sources for such results (Etz and Vandekerckhove, 2016; Marsman et al. 2017). Instead, some understand the failure of replication simply as a result of random error (Stanley et al. 2014; Senn 2002). Finally, others suggested to adopt a more comprehensive view to the analysis of evidence (Andrew Gelman, 2015; Marsman et al. 2017). Philosophy of Science and Epistemology can fruitfully complement Statistical knowledge, Epidemiology, and Medical Methodology in analysing medical evidence from different perspectives. The roundtable intends to offer such an opportunity and to examine the interaction of different dimensions of evidence, such as consistency, reliability, variety with respect to methodology, context of enquiry, and theory. Method Develop a platform for discussing issues arising from the algorithmic approach developed in the EBM and proposals for improvements coming from within EBM and from methodologists and philosophers. These concern how reliability could be broken down in different dimensions, on how context-sensitivity of causation could be exploited for discovery and scientific understanding, and on how theories of causation and knowledge may contribute to improve the concept of evidence developed within the EBM paradigm. These questions will be analysed from the perspective of philosophers of science and epistemologists: Rani Anjum, Vincenzo Crupi, Bennett Holman, Barbara Osimani, Ben Smart as well as scientists: Jeff Aronson, Ralph Edwards, Marie Lindquist, Elena Rocca. The interaction of both sides may greatly contribute to the debate for several reasons; mainly for the complementary expertise and views developed within various philosophical areas and scientific methodology with respect to evidence at different stages of the scientific enquiry. Results The following questions will be examined both in the presentations and in the discussion time: What is the nature of medical evidence? 2) What is the ‘best evidence’ for what aim and under what conditions? This implies a cost-benefit analysis of evidential standards for different purposes both in terms of information value, as well as in terms of (opportunity) costs at the individual and public level (e.g. Evidence for hypothesis generation vs evidence for decision); 3) What is the specific informative contribution of evidence from ‘big-data’, with respect to standard epidemiological and clinical evidence or to simulations? 4) What is the right tradeoff between replication and robustness analysis? Under which circumstances? 5) What is the role of expert opinion in making sense of such an amount of heterogeneous data? 6) Impact of regulatory frames on the development of evidential standards and vice-versa; 7) Is a grand-unified theory of evidence possible? Conclusions Philosophers of science have examined how the choice of scientific methods reveal the implicit philosophical assumptions about the nature of causation. Furthermore, specific domains of causal assessment, such as pharmacovigilance, can be better characterised as contexts of discovery and hypothesis generation, rather than of hypothesis testing and justification, therefore, evidence in these contexts work differently. More generally, the roundtable presents a comprehensive approach to evidence evaluation, which takes also ‘higher-order’ dimensions of evidence into account and discusses the nature of evidence and the role of evidence standards at its roots.