Transparency And Openness Promotion Research Articles

Investigating the nature of open science practices across complementary, alternative, and integrative medicine journals: An audit.

Open science practices are implemented across many scientific fields to improve transparency and reproducibility in research. Complementary, alternative, and integrative medicine (CAIM) is a growing field that may benefit from adoption of open science practices. The efficacy and safety of CAIM practices, a popular concern with the field, can be validated or refuted through transparent and reliable research. Investigating open science practices across CAIM journals by using the Transparency and Openness Promotion (TOP) guidelines can potentially promote open science practices across CAIM journals. The purpose of this study is to conduct an audit that compares and ranks open science practices adopted by CAIM journals against TOP guidelines laid out by the Center for Open Science (COS). CAIM-specific journals with titles containing the words "complementary", "alternative" and/or "integrative" were included in this audit. Each of the eight TOP criteria were used to extract open science practices from each of the CAIM journals. Data was summarized by the TOP guideline and ranked using the TOP Factor to identify commonalities and differences in practices across the included journals. A total of 19 CAIM journals were included in this audit. Across all journals, the mean TOP Factor was 2.95 with a median score of 2. The findings of this study reveal high variability among the open science practices required by journals in this field. Four journals (21%) had a final TOP score of 0, while the total scores of the remaining 15 (79%) ranged from 1 to 8. While several studies have audited open science practices across discipline-specific journals, none have focused on CAIM journals. The results of this study indicate that CAIM journals provide minimal guidelines to encourage or require authors to adhere to open science practices and there is an opportunity to improve the use of open science practices in the field.

Evaluation of Transparency and Openness Guidelines in Physical Therapist Journals.

The goals of this study were to evaluate the extent that physical therapist journals support open science research practices by adhering to the Transparency and Openness Promotion (TOP) guidelines and to assess the relationship between journal scores and their respective journal impact factor (JIF). Scimago, mapping studies, the National Library of Medicine, and journal author guidelines were searched to identify physical therapist journals for inclusion. Journals were graded on 10 standards (29 available total points) related to transparency with data, code, research materials, study design and analysis, preregistration of studies and statistical analyses, replication, and open science badges. The relationship between journal transparency and openness scores and their JIF was determined. Thirty-five journals' author guidelines were assigned transparency and openness factor scores. The median score (interquartile range) across journals was 3.00 out of 29 (3.00) points (for all journals the scores ranged from 0 to 8). The 2 standards with the highest degree of implementation were design and analysis transparency (reporting guidelines) and study preregistration. No journals reported on code transparency, materials transparency, replication, and open science badges. TOP factor scores were a significant predictor of JIF scores. There is low implementation of the TOP standards by physical therapist journals. TOP factor scores demonstrated predictive abilities for JIF scores. Policies from journals must improve to make open science practices the standard in research. Journals are in an influential position to guide practices that can improve the rigor of publication which, ultimately, enhances the evidence-based information used by physical therapists. Transparent, open, and reproducible research will move the profession forward by improving the quality of research and increasing the confidence in results for implementation in clinical care.

Open Science Standards at Journals that Inform Evidence-Based Policy.

Evidence-based policy uses intervention research to inform consequential decisions about resource allocation. Research findings are often published in peer-reviewed journals. Because detrimental research practices associated with closed science are common, journal articles report more false-positives and exaggerated effect sizes than would be desirable. Journal implementation of standards that promote open science-such as the transparency and openness promotion (TOP) guidelines-could reduce detrimental research practices and improve the trustworthiness of research evidence on intervention effectiveness. We evaluated TOP implementation at 339 peer-reviewed journals that have been used to identify evidence-based interventions for policymaking and programmatic decisions. Each of ten open science standards in TOP was not implemented in most journals' policies (instructions to authors), procedures (manuscript submission systems), or practices (published articles). Journals implementing at least one standard typically encouraged, but did not require, an open science practice. We discuss why and how journals could improve implementation of open science standards to safeguard evidence-based policy.

Exploring enablers and barriers to implementing the Transparency and Openness Promotion Guidelines: a theory-based survey of journal editors.

The Transparency and Openness Promotion (TOP) Guidelines provide a framework to help journals develop open science policies. Theories of behaviour change can guide understanding of why journals do (not) implement open science policies and the development of interventions to improve these policies. In this study, we used the Theoretical Domains Framework to survey 88 journal editors on their capability, opportunity and motivation to implement TOP. Likert-scale questions assessed editor support for TOP, and enablers and barriers to implementing TOP. A qualitative question asked editors to provide reflections on their ratings. Most participating editors supported adopting TOP at their journal (71%) and perceived other editors in their discipline to support adopting TOP (57%). Most editors (93%) agreed their roles include maintaining policies that reflect current best practices. However, most editors (74%) did not see implementing TOP as a high priority compared with other editorial responsibilities. Qualitative responses expressed structural barriers to implementing TOP (e.g. lack of time, resources and authority to implement changes) and varying support for TOP depending on study type, open science standard, and level of implementation. We discuss how these findings could inform the development of theoretically guided interventions to increase open science policies, procedures and practices.

Evaluating Implementation of the Transparency and Openness Promotion Guidelines: Reliability of Instruments to Assess Journal Policies, Procedures, and Practices

The Transparency and Openness Promotion (TOP) Guidelines describe modular standards that journals can adopt to promote open science. The TOP Factor quantifies the extent to which journals adopt TOP in their policies, but there is no validated instrument to assess TOP implementation. Moreover, raters might assess the same policies differently. Instruments with objective questions are needed to assess TOP implementation reliably. In this study, we examined the interrater reliability and agreement of three new instruments for assessing TOP implementation in journal policies (instructions to authors), procedures (manuscript-submission systems), and practices (journal articles). Independent raters used these instruments to assess 339 journals from the behavioral, social, and health sciences. We calculated interrater agreement (IRA) and interrater reliability (IRR) for each of 10 TOP standards and for each question in our instruments (13 policy questions, 26 procedure questions, 14 practice questions). IRA was high for each standard in TOP; however, IRA might have been high by chance because most standards were not implemented by most journals. No standard had “excellent” IRR. Three standards had “good,” one had “moderate,” and six had “poor” IRR. Likewise, IRA was high for most instrument questions, and IRR was moderate or worse for 62%, 54%, and 43% of policy, procedure, and practice questions, respectively. Although results might be explained by limitations in our process, instruments, and team, we are unaware of better methods for assessing TOP implementation. Clarifying distinctions among different levels of implementation for each TOP standard might improve its implementation and assessment (study protocol: https://doi.org/10.1186/s41073-021-00112-8 ).

Evaluating Research Transparency and Openness in Communication Sciences and Disorders Journals.

To improve the credibility, reproducibility, and clinical utility of research findings, many scientific fields are implementing transparent and open research practices. Such open science practices include researchers making their data publicly available and preregistering their hypotheses and analyses. A way to enhance the adoption of open science practices is for journals to encourage or require submitting authors to participate in such practices. Accordingly, the American Speech-Language-Hearing Association's Journals Program has recently announced their intention to promote open science practices. Here, we quantitatively assess the extent to which several journals in communication sciences and disorders (CSD) encourage or require participation in several open science practices by using the Transparency and Openness Promotion (TOP) Factor metric. TOP Factors were assessed for 34 CSD journals, as well as several journals in related fields. TOP Factors measure the level of implementation across 10 open science-related practices (e.g., data transparency, analysis plan preregistration, and replication) for a total possible score of 29 points. Collectively, CSD journals had very low TOP Factors (M = 1.4, range: 0-8). The related fields of Psychology (M = 4.0), Rehabilitation (M = 3.2), Linguistics (M = 1.7), and Education (M = 1.6) also had low scores, though Psychology and Rehabilitation had higher scores than CSD. CSD journals currently have low levels of encouraging or requiring participation in open science practices, which may impede adoption. Open Science Form: https://doi.org/10.23641/asha.21699458.

Adoption of Transparency and Openness Promotion (TOP) Guidelines across Journals

Journal policies continuously evolve to enable knowledge sharing and support reproducible science. However, that change happens within a certain framework. Eight modular standards with three levels of increasing stringency make Transparency and Openness Promotion (TOP) guidelines which can be used to evaluate to what extent and with which stringency journals promote open science. Guidelines define standards for data citation, transparency of data, material, code and design and analysis, replication, plan and study pre-registration, and two effective interventions: “Registered reports” and “Open science badges”, and levels of adoption summed up across standards define journal’s TOP Factor. In this paper, we analysed the status of adoption of TOP guidelines across two thousand journals reported in the TOP Factor metrics. We show that the majority of the journals’ policies align with at least one of the TOP’s standards, most likely “Data citation” (70%) followed by “Data transparency” (19%). Two-thirds of adoptions of TOP standard are of the stringency Level 1 (less stringent), whereas only 9% is of the stringency Level 3. Adoption of TOP standards differs across science disciplines and multidisciplinary journals (N = 1505) and journals from social sciences (N = 1077) show the greatest number of adoptions. Improvement of the measures that journals take to implement open science practices could be done: (1) discipline-specific, (2) journals that have not yet adopted TOP guidelines could do so, (3) the stringency of adoptions could be increased.

Open science and conflict of interest policies of medical and health sciences journals before and during the COVID-19 pandemic: A repeat cross-sectional study: Open science policies of medical journals.

To audit the transparent and open science standards of health and medical sciences journal policies and explore the impact of the COVID-19 pandemic. Repeat cross-sectional study. 19 journals listed in Google Scholar's Top Publications for health and medical sciences. Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences, Science Translational Medicine, The British Medical Journal, The Journal of the American Medical Association, The Lancet, The Lancet Oncology, and The New England Journal of Medicine. We used the Transparency and Openness Promotion (TOP) guideline and the International Committee of Medical Journal Editors (ICMJE) requirements for disclosing conflicts of interest (COIs) to evaluate journals standards. TOP scores slightly improved during the COVID-19 pandemic, from a median of 5 (IQR: 2-12.5) out of a possible 24 points in February 2020 to 7 (IQR: 4-12) in May 2021, but overall, scores were very low at both time points. Journal policies scored highest for their adherence to data transparency and scored lowest for preregistration of study protocols and analysis plans and the submission of replication studies. Most journals fulfilled all ICMJE provisions for reporting COIs before (84%; n = 16) and during (95%; n = 18) the COVID-19 pandemic. The COVID-19 pandemic has highlighted the importance of practising open science. However, requirements for open science practices in audited policies were overall low, which may impede progress in health and medical research. As key stakeholders in disseminating research, journals should promote a research culture of greater transparency and more robust open science practices.

Adoption of Transparency and Openness Promotion (TOP) guidelines across journals

The Transparency and Openness Promotion (TOP) guidelines include eight modular standards, with three tiers of increasing stringency, which journals can adopt to promote open science (Nosek et al. 2015). Standards evaluate journal policies on data citation, data transparency, material transparency, code transparency, design and analysis, study preregistration, analysis preregistration, and replication.To get an idea about whether TOP guidelines are being adopted by journals, we analysed the policies of 1,219 journals available on the website of the Center for Open Science. Journals were categorized by the three stringency tiers of eight TOP standards - were analyzed. 68% of the journals have their policies aligned to at least one of the TOP standards, nearly 46% have their policies aligned to data citation standards, and only 10% have a policy for pre-registration of analysis plans or study preregistration. If a journal policy adopts TOP guidelines, it is most likely of the stringency Level 1 (less stringent) across all eight standards (median=70%). However, policies for the “Analysis code transparency” standard are more “stringent” compared to other standards, and half of the journals require Level 2 and 3 stringency, e.g. storage in an open repository and/or code reproducibility. In the future, we are planning to complement these preliminary results with information about the journal policy requirements stratified across different scientific disciplines and to add a component of time – the evolution of requirements from publishers for adoption of open science practices.Our results already suggest two ways to improve implementation of open science practices:1) adoption of policies that promote open science for journals that still have not adopted them, and2) increasing the stringency of requirements for open science practices for journals that have.

Research practices for a robust psychological science of adult development and aging.

This first issue of 2022 marks the transition of Psychology and Aging in adopting a transparency and openness promotion (TOP) framework. The journal has always had high standards for theoretically meaningful research conducted with methodological and analytic rigor. As the Open Science movement has gathered steam, authors are increasingly submitting papers that fully meet TOP standards at Levels 1 or 2, and those who do not, have generally been quite happy to respond to the gentle nudges of the journal's editors. Thus, in practical terms, the changes at this point are actually quite modest. In what follows, Stine-Morrow addresses questions about the new standards: (a) Why now? and (b) What are they? (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Clearinghouse Standards of Evidence on the Transparency, Openness, and Reproducibility of Intervention Evaluations

Clearinghouses are influential repositories of information on the effectiveness of social interventions. To identify which interventions are “evidence-based,” clearinghouses review intervention evaluations using published standards of evidence that focus primarily on internal validity and causal inferences. Open science practices can improve trust in evidence from evaluations on the effectiveness of social interventions. Including open science practices in clearinghouse standards of evidence is one of many efforts that could increase confidence in designations of interventions as “evidence-based.” In this study, we examined the policies, procedures, and practices of 10 federal evidence clearinghouses that review preventive interventions—an important and influential subset of all evidence clearinghouses. We found that seven consider at least one open science practice when evaluating interventions: replication (6 of 10 clearinghouses), public availability of results (6), investigator conflicts of interest (3), design and analysis transparency (3), study registration (2), and protocol sharing (1). We did not identify any policies, procedures, or practices related to analysis plan registration, data sharing, code sharing, material sharing, and citation standards. We provide a framework with specific recommendations to help federal and other evidence clearinghouses implement the Transparency and Openness Promotion (TOP) Guidelines. Our proposed “TOP Guidelines for Clearinghouses” includes reporting whether evaluations used open science practices, incorporating open science practices in their standards for receiving “evidence-based” designations, and verifying that evaluations used open science practices. Doing so could increase the trustworthiness of evidence used for policy making and support improvements throughout the evidence ecosystem.

"Health Psychology adopts Transparency and Openness Promotion (TOP) Guidelines": Correction to Freedland (2021).

Reports an error in "Health Psychology adopts Transparency and Openness Promotion (TOP) Guidelines" by Kenneth E. Freedland (Health Psychology, 2021[Apr], Vol 40[4], 227-229). In the article, the TOP levels were mislabeled throughout. Level 2 should have been Level 1, and Level 1 should have been Level 2. The online version of this article has been corrected. (The following abstract of the original article appeared in record 2021-35537-001.) The Editors are pleased to announce that Health Psychology has adopted the Transparency and Openness Promotion (TOP) Guidelines (Center for Open Science, 2021). We and the other core American Psychological Association (APA) journals are implementing these guidelines at the direction of the APA Publications and Communications Board. Their decision was made with the support of the APA Council of Editors and the APA Open Science and Methodology Committee. The TOP Guidelines were originally published in Science (Nosek et al. 2015) to encourage journals to incentivize open research practices. They are being implemented by a wide range of scientific publications, including some of the leading behavioral and medical research journals. The TOP guidelines for Health Psychology are outlined in our recently revised Instructions to Authors. We are implementing most of the standards at TOP Level 1, which means that adherence is required. We are implementing other standards at Level 2, which means that authors are encouraged to adhere to them and required to disclose whether and how they have done so. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Evaluating implementation of the Transparency and Openness Promotion\xa0(TOP) guidelines:\xa0the TRUST process for rating journal policies, procedures, and practices

BackgroundThe Transparency and Openness Promotion (TOP) Guidelines describe modular standards that journals can adopt to promote open science. The TOP Factor is a metric to describe the extent to which journals have adopted the TOP Guidelines in their policies. Systematic methods and rating instruments are needed to calculate the TOP Factor. Moreover, implementation of these open science policies depends on journal procedures and practices, for which TOP provides no standards or rating instruments.MethodsWe describe a process for assessing journal policies, procedures, and practices according to the TOP Guidelines. We developed this process as part of the Transparency of Research Underpinning Social Intervention Tiers (TRUST) Initiative to advance open science in the social intervention research ecosystem. We also provide new instruments for rating journal instructions to authors (policies), manuscript submission systems (procedures), and published articles (practices) according to standards in the TOP Guidelines. In addition, we describe how to determine the TOP Factor score for a journal, calculate reliability of journal ratings, and assess coherence among a journal’s policies, procedures, and practices. As a demonstration of this process, we describe a protocol for studying approximately 345 influential journals that have published research used to inform evidence-based policy.DiscussionThe TRUST Process includes systematic methods and rating instruments for assessing and facilitating implementation of the TOP Guidelines by journals across disciplines. Our study of journals publishing influential social intervention research will provide a comprehensive account of whether these journals have policies, procedures, and practices that are consistent with standards for open science and thereby facilitate the publication of trustworthy findings to inform evidence-based policy. Through this demonstration, we expect to identify ways to refine the TOP Guidelines and the TOP Factor. Refinements could include: improving templates for adoption in journal instructions to authors, manuscript submission systems, and published articles; revising explanatory guidance intended to enhance the use, understanding, and dissemination of the TOP Guidelines; and clarifying the distinctions among different levels of implementation.Research materials are available on the Open Science Framework: https://osf.io/txyr3/.

Make like a glass frog: In support of increased transparency in herpetology

Across many scientific disciplines, direct replication efforts and meta-analyses have fuelled concerns on the replicability of findings. Ecology and evolution are similarly affected. Investigations into the causes of this lack of replicability have implicated a suite of research practices linked to incentives in the current publishing system. Other fields have taken great strides to counter incentives that can reward obfuscation –chiefly by championing transparency. But how prominent are protransparency (open science) policies in herpetology journals? We use the recently developed Transparency and Openness Promotion (TOP) Factor to assess the transparency promotion of 19 herpetology journals, and compare the TOP scores to broader science. We find promotion of transparent practices currently lacking in many herpetological journals; and encourage authors, students, editors, and publishers to redouble efforts to bring open science practices to herpetology by changing journal policy, peer-review, and personal practice. We promote an array of options –developed and tested in other fields– demonstrated to counter publication bias, boost research uptake, and enable more transparent science, to enrich herpetological research.

Forest restoration or propaganda? The need for Transparency and Openness Promotion (TOP) scores to uphold research integrity

In a time of environmental crisis and ‘fake news’, there are calls for scientists to engage in public debate or advocacy. Some are wary, fearing that revealing subjective views poses a risk to scientific credibility or erodes trust in scholarly publishing. Others are less concerned, seeing it as their duty to society or an opportunity to boost their profile. Ideally, we need better checks and balances that allow scientists to contribute to public discourse without fear of compromising the credibility of their science, while avoiding subjective views influencing the outcomes of peer-reviewed research. For better or worse, scientists have personal views. The question is not whether they should be condoned or condemned, but how they should be managed in the context of scholarly publishing to maximise benefits and minimise negative outcomes. Using the recent contention around global tree ‘restoration’ potential as an example, I propose we score journals and articles based on the Transparency and Openness Promotion (TOP) guidelines and associated criteria. A high TOP score means readers have sufficient access to information to assess the objectivity and credibility of scientific publications and their authors. I show that current practice provides very little access to information, and readers are essentially being asked to have faith in the scholarly publication system. We must do better.
 Significance:
 
 Science is predicated upon objectivity, yet readers are rarely given enough information to assess the objectivity, and thus integrity, of peer-reviewed research.
 To address this issue, a scoring system is proposed, which is based on the principles of transparency and openness.
 Improving transparency and openness in scholarly publishing is essential for allowing readers to assess the objectivity of published research and researchers, growing public trust, and allowing researchers to engage in public debates without fear of loss of scientific credibility.

Does Truth Have a Future?

Does Truth Have a Future?

Open Science Initiatives at Emerging Adulthood

This editorial reports on concrete changes in expectations and practices at Emerging Adulthood that are effective immediately. These include: aligning with Level 1 of the Transparency and Openness Promotion (TOP) Guidelines, adding Registered Reports as a submission option, accepting commentaries on articles published in any journal, and making open science badges available to authors.

Editorial: Improving Scientific Practices and Increasing Access

Editorial: Improving Scientific Practices and Increasing Access

Peer Review – the future is here

Several divergent standpoints are hosted under the umbrella of Peer Review. Authors, who wish to see the results of scrupulous work published, in the anticipation of career promotion or secure funding. Editors, who are under pressure to identify sound and novel research. Reviewers, who try to fit a thoughtful and time-consuming process within busy schedules, often receiving no credit for it. And Publishers competing in a transforming landscape of fast and abundant science publishing. This article is protected by copyright. All rights reserved.

Enhancing reproducibility for computational methods.

Data, code, and workflows should be available and cited