Systematic Reviews In Psychology Research Articles

Reporting characteristics of systematic reviews in Psychology: A scoping review.

This article describes the reporting characteristics of systematic reviews (SRs) in Psychology. The inclusion criteria were self-declared SRs in all branches of Psychology, published between 2019 and 2020. The search was performed in the PsycINFO and MEDLINE databases, from which 2487 records were identified, and 305 papers selected. There were many discrepancies in the reporting of Psychology SRs. Some PRISMA items, such as self-identification as an SR, description of the aim and the inclusion criteria, specification of the databases, and the description of the search and selection process using a flow diagram were reported in more than 90% of the SRs. Other items had lower coverage, such as the specification of the PICO framework, presentation of the complete search strategies, mention of the reporting guidelines, description of the exclusion criteria, performance of a risk of bias assessment, and analysis of the quality of the evidence, among others. The study highlights the need to improve the planning, performance and reporting of SR in Psychology.

Doing systematic reviews in Psychology

Systematic reviews by virtue of being a pre-determined, transparent, and comprehensive plan and search strategy are fast gaining popularity in psychology in South Africa. A systematic review allows one to obtain a thorough overview regarding the recent developments and debates on a given topic with the addition of metacommentary. In South Africa, we have noted stark differences in the reporting of systematic reviews. Often studies are identified as systematic reviews but methodologically have failed to meet the rigorous criteria that characterise this method. This article aims to provide a guide for the novice researcher on conducting systematic reviews. We draw on a practical case study by Hassem and Laher where the systematic review method was used to establish the efficacy of online depression screening tools in the South African context as a practical illustration of the systematic review method. In so doing, the affordances and limitations of the systematic review method are discussed.

Advocating for reproducibility

As guest editors, we are excited to publish this special double issue of IASSIST Quarterly. The topics of reproducibility, replicability, and transparency have been addressed in past issues of IASSIST Quarterly and at the IASSIST conference, but this double issue is entirely focused on these issues.
 In recent years, efforts “to improve the credibility of science by advancing transparency, reproducibility, rigor, and ethics in research” have gained momentum in the social sciences (Center for Effective Global Action, 2020). While few question the spirit of the reproducibility and research transparency movement, it faces significant challenges because it goes against the grain of established practice.
 We believe the data services community is in a unique position to help advance this movement given our data and technical expertise, training and consulting work, international scope, and established role in data management and preservation, and more. As evidence of the movement, several initiatives exist to support research reproducibility infrastructure and data preservation efforts:
 
 Center for Open Science (COS) / Open Science Framework (OSF)[i]
 Berkeley Initiative for Transparency in the Social Sciences (BITSS)[ii]
 CUrating for REproducibility (CURE)[iii]
 Project Tier[iv]
 Data Curation Network[v]
 UK Reproducibility Network[vi]
 
 While many new initiatives have launched in recent years, prior to the now commonly used phrase “reproducibility crisis” and Ioannidis publishing the essay, “Why Most Published Research Findings are False,” we know that the data services community was supporting reproducibility in a variety of ways (e.g., data management, data preservation, metadata standards) in wellestablished consortiums such as Inter-university Consortium for Political and Social Research (ICPSR) (Ioannidis, 2005).
 The articles in this issue comprise several very important aspects of reproducible research:
 
 Identification of barriers to reproducibility and solutions to such barriers
 Evidence synthesis as related to transparent reporting and reproducibility
 Reflection on how information professionals, researchers, and librarians perceive the reproducibility crisis and how they can partner to help solve it.
 
 The issue begins with “Reproducibility literature analysis” which looks at existing resources and literature to identify barriers to reproducibility and potential solutions. The authors have compiled a comprehensive list of resources with annotations that include definitions of key concepts pertinent to the reproducibility crisis.
 The next article addresses data reuse from the perspective of a large research university. The authors examine instances of both successful and failed data reuse instances and identify best practices for librarians interested in conducting research involving the common forms of data collected in an academic library.
 Systematic reviews are a research approach that involves the quantitative and/or qualitative synthesis of data collected through a comprehensive literature review. “Methods reporting that supports reader confidence for systematic reviews in psychology” looks at the reproducibility of electronic literature searches reported in psychology systematic reviews.
 A fundamental challenge in reproducing or replicating computational results is the need for researchers to make available the code used in producing these results. But sharing code and having it to run correctly for another user can present significant technical challenges. In “Reproducibility, preservation, and access to research with Reprozip, Reproserver” the authors describe open source software that they are developing to address these challenges. 
 Taking a published article and attempting to reproduce the results, is an exercise that is sometimes used in academic courses to highlight the inherent difficulty of the process. The final article in this issue, “ReprohackNL 2019: How libraries can promote research reproducibility through community engagement” describes an innovative library-based variation to this exercise.
 
 Harrison Dekker, Data Librarian, University of Rhode Island
 Amy Riegelman, Social Sciences Librarian, University of Minnesota
 
 References 
 Center for Effective Global Action (2020), About the Berkeley Initiative for Transparency in the Social Sciences. Available at: https://www.bitss.org/about (accessed 23 June 2020).
 Ioannidis, J.P. (2005) ‘Why most published research findings are false’, PLoS Medicine, 2(8), p. e124. doi: https://doi.org/10.1371/journal.pmed.0020124
 
 [i] https://osf.io
 [ii] https://www.bitss.org/
 [iii] http://cure.web.unc.edu
 [iv] https://www.projecttier.org/
 [v] https://datacurationnetwork.org/
 [vi] https://ukrn.org

Methods reporting that supports reader confidence for systematic reviews in psychology: assessing the reproducibility of electronic searches and first-level screening decisions.

Recent discussions and research in psychology show a significant emphasis on reproducibility. Concerns for reproducibility pertain to methods as well as results. We evaluated the reporting of the electronic search methods used for systematic reviews (SR) published in psychology. Such reports are key for determining the reproducibility of electronic searches. The use of SR has been increasing in psychology, and we report on the status of reporting of electronic searches in recent SR in psychology.
 We used 12 checklist items to evaluate reporting for basic electronic strategies. Kappa results for those items developed from evidence-based recommendations ranged from fair to almost perfect. Additionally, using a set of those items to represent a “PRISMA” type of recommended reporting showed that only one of the 25 randomly selected psychology SR from 2009-2012 reported recommended information for all items in the set, and none of the 25 psychology SR from 2014-2016 did so. Using a second less stringent set of items found that only 36% of the psychology SR reported basic information that supports confidence in the reproducibility of electronic searches. Similar results were found for a set of psychology SR published in 2017.
 An area for improvements in SR in psychology involves fuller and clearer reporting of the steps used for electronic searches in SR. Such improvements will provide a strong basis for confidence in the reproducibility of searches. That confidence, in turn, can strengthen reader confidence more generally in the results and conclusions reached in SR in psychology.

Guidelines for conducting and publishing systematic reviews in Psychology

Abstract The systematic review refers to the literature review guided by scientific methods explicitly intended to reduce bias, resulting in a synthesis of all relevant evidence for a given issue. In Brazil, specifically in Psychology, systematic review is found in the literature; however, the available studies do not always reflect the gold standard or what is expected in terms of typical systematic review procedures. The present study is structured in the form of a didactic guide, organized in topics, which should be typically contemplated in an systematic review in Psychology. The information that must be contained in each of these topics is indicated, including which procedures should be performed in the typical steps of the development of an systematic review. The present publication intends to increase the interest and investment of researchers in systematic review, providing them with information to improve the quality of systematic review in the area of Psychology in Brazil.

Systematic reviews in psychology: Time to review their use?

Researchers, including doctoral students, increasingly believe it is important to conduct systematic reviews – the ‘gold standard’ of reviewing (Boland et al., 2014). Systematic reviews (SRs) can be useful in delineating issues that are addressed by research students, though many areas within psychology may be inappropriate for systematic review as they lack an evidence base that meets SRs’ strict procedures. In medicine, from where the technique derives, one variable is frequently studied using repeated randomised controlled trial (RCT) designs – making SRs relatively easy. We argue that psychologists should use SRs, but with great care and only where appropriate.

Standards on how to develop and report systematic reviews in Psychology and Health

Systematic reviews are secondary studies that summarize the best scientific evidence available by means of explicit and rigorous methods to identify, select, appraise, analyse and summarise the empirical studies that enable responding to specific questions. The aim of this theoretical study is to set out a series of standards and recommendations for the planning, development and reporting of a systematic review in the field of the health sciences. The article describes the systematic reviews in the context of practice based on scientific evidence, their rise, justification, applicability and differences compared to traditional literature reviews. Secondly, the methodology is set out for their development and guidelines are established for their preparation; the stages of the process and preparation of a protocol are described with emphasis on the steps to follow to prepare and report a systematic review. Finally, some additional considerations are set out for their preparation and publication in a scientific journal. This guide is aimed both at authors and reviewers of a systematic review.

Comprehensive computer searches and reporting in systematic reviews.

Systematic reviews (SR) are a strategic resource for many who may assume that comprehensive computer searches are used to identify the studies that are used in SR. The current study assessed the reports of comprehensive computer searching in SR in psychology. Comprehensive computer search methods listed as basic in SR manuals and publications of major SR organizations (e.g., Cochrane Collaboration) were the "recommended methods" that became items on a checklist used to assess computer search reports. A methodology index search in PsycINFO identified SR in psychology that were compared to SR identified in the Cochrane Database of SR. Checklist item frequencies supported descriptive analyses, and Mann-Whitney U-test was used to compare the PsycINFO and Cochrane SR. Two recommended computer search methods were significantly more common in Cochrane SR: truncation (z = -5.64, p < .001), controlled vocabulary (z = -5.08, p < .001 ). A third search method (Cited Reference Searching) was virtually absent (SR in psychology: 0/25; and Cochrane SR: 1/25). Confidence in SR conclusions may be undermined when evidence of recommended or empirically-based search methods is not seen. Results and suggestions might have value for those who use, evaluate, or develop guidelines for SR; research topics are also described. Copyright © 2011 John Wiley & Sons, Ltd.