Building a solid foundation: advancing evidence synthesis in agri-food systems science

Abstract

Enhancing the reliability of literature reviews and evidence synthesis is crucial for advancing the transformation of agriculture and food (agri-food) systems as well as for informed decisions and policy making. In this perspective, we argue that evidence syntheses in the field of agri-food systems research often suffer from a suite of methodological limitations that substantially increase the risk of bias, i.e., publication and selection bias, resulting in unreliable and potentially flawed conclusions and, consequently, poor decisions (e.g., policy direction, investment, research foci). We assessed 926 articles from the Collaboration for Environmental Evidence Database of Evidence Reviews (CEEDER) and recent examples from agri-food systems research to support our reasoning. The analysis of articles from CEEDER (n = 926) specifically indicates poor quality (Red) in measures to minimize subjectivity during critical appraisal (98% of all reviews), application of the eligibility criteria (97%), cross-checking of extracted data by more than one reviewer (97%), critical appraisal of studies (88%), establishment of an a priori method/protocol (86%), and transparent reporting of eligibility decisions (65%). Additionally, deficiencies (Amber) were found in most articles (>50%) regarding the investigation and discussion of variability in study findings (89%), comprehensiveness of the search (78%), definition of eligibility criteria (72%), search approach (64%), reporting of extracted data for each study (59%), consideration and discussion of the limitations of the synthesis (56%), documentation of data extraction (54%) and regarding the statistical approach (52%). To enhance the quality of evidence synthesis in agri-food science, review authors should use tried-and-tested methodologies and publish peer-reviewed a priori protocols. Training in evidence synthesis methods should be scaled, with universities playing a crucial role. It is the shared duty of research authors, training providers, supervisors, reviewers, and editors to ensure that rigorous and robust evidence syntheses are made available to decision-makers. We argue that all these actors should be cognizant of these common mistakes to avoid publishing unreliable syntheses. Only by thinking as a community can we ensure that reliable evidence is provided to support appropriate decision-making in agri-food systems science.