Transparency In Research Research Articles

Are We Becoming More Transparent? Survey Reporting Trends in Top Journals of Social Sciences

Abstract Transparency is essential for objectively evaluating or replicating any study’s findings. Although several reporting standards and guidelines exist for survey research, these standards are not adopted or enforced by most academic journals, and thus, transparency in survey research has been relatively low. This study documents and compares trends in transparency in survey research. We reviewed 899 articles that used 1,237 original surveys and were published in top journals in the fields of political science, sociology, and public opinion research between 2011 and 2021. The results show small changes in reporting different survey characteristics during the observed period. Nevertheless, we found that the majority of the articles did not provide responses or other rates, and a strong decrease was observed in transparency regarding response rates. Our results underscore the importance of promoting reporting standards and enforcing rules more rigorously.

CSDMS Data Components: data–model integration tools for Earth surface processes modeling

Abstract. Progress in better understanding and modeling Earth surface systems requires an ongoing integration of data and numerical models. Advances are currently hampered by technical barriers that inhibit finding, accessing, and executing modeling software with related datasets. We propose a design framework for Data Components, which are software packages that provide access to particular research datasets or types of data. Because they use a standard interface based on the Basic Model Interface (BMI), Data Components can function as plug-and-play components within modeling frameworks to facilitate seamless data–model integration. To illustrate the design and potential applications of Data Components and their advantages, we present several case studies in Earth surface processes analysis and modeling. The results demonstrate that the Data Component design provides a consistent and efficient way to access heterogeneous datasets from multiple sources and to seamlessly integrate them with various models. This design supports the creation of open data–model integration workflows that can be discovered, accessed, and reproduced through online data sharing platforms, which promotes data reuse and improves research transparency and reproducibility.

Quality Management and Animal Welfare as Culture of Care for Animal Experiments under Biosafety Level-3, taking COVID-19 as a Practical Example

Abstract Since the start of the COVID‐19 pandemic, a tremendous amount of work has been done by scientists to understand the pathogenesis of the severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2). Several animal models have been reported to be useful for the development and testing of vaccines, antivirals, or therapeutics. These in vivo experiments have to be performed at high containment biosafety level (BSL)-3. Thus, an appropriate biosafety, facility and data management as well as high standard of animal welfare, thorough study design, and staff training combined with an overall research transparency and open communication need to be implemented to achieve high quality reproducible research. Such an approach combined with the well-known 3R principle (Replacement, Reduction, and Refinement) and the 3S principle (Good Science, Good Sense, and Good Sensibilities) can contribute to culture of care in scientific research. The perspective reviews the key points of optimized biosafety, data, quality and animal welfare management in high risk biosafety-settings to conduct animal experiments with infectious pathogens in the context of a culture of care.

The State of Open Science Practices in Psychometric Studies of Suicide: A Systematic Review.

The adoption of open science practices (OSPs) is crucial for promoting transparency and robustness in research. We conducted a systematic review to assess the frequency and trends of OSPs in psychometric studies focusing on measures of suicidal thoughts and behavior. We analyzed publications from two international databases, examining the use of OSPs such as open access publication, preregistration, provision of open materials, and data sharing. Our findings indicate a lack of adherence to OSPs in psychometric studies of suicide. The majority of manuscripts were published under restricted access, and preregistrations were not utilized. The provision of open materials and data was rare, with limited access to instruments and analysis scripts. Open access versions (preprints/postprints) were scarce. The low adoption of OSPs in psychometric studies of suicide calls for urgent action. Embracing a culture of open science will enhance transparency, reproducibility, and the impact of research in suicide prevention efforts.

Completeness of intervention reporting of nutrition-focused weight management interventions adjunct to metabolic and bariatric surgery: effect of the TIDieR checklist.

Nutrition-focused interventions are essential to optimize the bariatric care process and improve health and weight outcomes over time. Clear and detailed reporting of these interventions in research reports is crucial for understanding and applying the findings effectively in clinical practice and research replication. Given the importance of reporting transparency in research, this study aimed to use the Template for Intervention Description and Replication (TIDieR) checklist to evaluate the completeness of intervention reporting in nutritional weight management interventions adjunct to metabolic and bariatric surgery (MBS). The secondary aim was to examine the factors associated with better reporting. A literature search in PubMed, PsychINFO, EMBASE, Scopus, and the Cochrane Controlled Register of Trials was conducted to include randomized controlled trials (RCT), quasi-RCTs and parallel group trials. A total of 22 trials were included in the final analysis. Among the TIDieR 12 items, 6.6 ± 1.9 items were fully reported by all studies. None of the studies completely reported all intervention descriptors. The main areas where reporting required improvement were providing adequate details of the materials and procedures of the interventions, intervention personalization, and intervention modifications during the study. The quality of intervention reporting remained the same after vs. before the release of the TIDieR guidelines. Receiving funds from industrial organizations (p = 0.02) and having the study recorded within a registry platform (p = 0.08) were associated with better intervention reporting. Nutritional weight management interventions in MBS care are still below the desirable standards for reporting. The present study highlights the need to improve adequate reporting of such interventions, which would allow for greater replicability, evaluation through evidence synthesis studies, and transferability into clinical practice.

A data-driven approach to choosing privacy parameters for clinical trial data sharing under differential privacy.

Clinical trial data sharing is crucial for promoting transparency and collaborative efforts in medical research. Differential privacy (DP) is a formal statistical technique for anonymizing shared data that balances privacy of individual records and accuracy of replicated results through a "privacy budget" parameter, ε. DP is considered the state of the art in privacy-protected data publication and is underutilized in clinical trial data sharing. This study is focused on identifying ε values for the sharing of clinical trial data. We analyzed 2 clinical trial datasets with privacy budget ε ranging from 0.01 to 10. Smaller values of ε entail adding greater amounts of random noise, with better privacy as a result. Comparison of rates, odds ratios, means, and mean differences between the original clinical trial datasets and the empirical distribution of the DP estimator was performed. The DP rate closely approximated the original rate of 6.5% when ε > 1. The DP odds ratio closely aligned with the original odds ratio of 0.689 when ε ≥ 3. The DP mean closely approximated the original mean of 164.64 when ε ≥ 1. As ε increased to 5, both the minimum and maximum DP means converged toward the original mean. There is no consensus on how to choose the privacy budget ε. The definition of DP does not specify the required level of privacy, and there is no established formula for determining ε. Our findings suggest that the application of DP holds promise in the context of sharing clinical trial data.

The transparency of quantitative empirical legal research published in highly ranked law journals (2018–2020): an observational study

Background Scientists are increasingly concerned with making their work easy to verify and build upon. Associated practices include sharing data, materials, and analytic scripts, and preregistering protocols. This shift towards increased transparency and rigor has been referred to as a “credibility revolution.” The credibility of empirical legal research has been questioned in the past due to its distinctive peer review system and because the legal background of its researchers means that many often are not trained in study design or statistics. Still, there has been no systematic study of transparency and credibility-related characteristics of published empirical legal research. Methods To fill this gap and provide an estimate of current practices that can be tracked as the field evolves, we assessed 300 empirical articles from highly ranked law journals including both faculty-edited journals and student-edited journals. Results We found high levels of article accessibility (86%, 95% CI = [82%, 90%]), especially among student-edited journals (100%). Few articles stated that a study’s data are available (19%, 95% CI = [15%, 23%]). Statements of preregistration (3%, 95% CI = [1%, 5%]) and availability of analytic scripts (6%, 95% CI = [4%, 9%]) were very uncommon. (i.e., they collected new data using the study’s reported methods, but found results inconsistent or not as strong as the original). Conclusion We suggest that empirical legal researchers and the journals that publish their work cultivate norms and practices to encourage research credibility. Our estimates may be revisited to track the field’s progress in the coming years.

Best practices for data management and sharing in experimental biomedical research.

Effective data management is crucial for scientific integrity and reproducibility, a cornerstone of scientific progress. Well-organized and well-documented data enable validation and building on results. Data management encompasses activities including organization, documentation, storage, sharing, and preservation. Robust data management establishes credibility, fostering trust within the scientific community and benefiting researchers' careers. In experimental biomedicine, comprehensive data management is vital due to the typically intricate protocols, extensive metadata, and large datasets. Low-throughput experiments, in particular, require careful management to address variations and errors in protocols and raw data quality. Transparent and accountable research practices rely on accurate documentation of procedures, data collection, and analysis methods. Proper data management ensures long-term preservation and accessibility of valuable datasets. Well-managed data can be revisited, contributing to cumulative knowledge and potential new discoveries. Publicly funded research has an added responsibility for transparency, resource allocation, and avoiding redundancy. Meeting funding agency expectations increasingly requires rigorous methodologies, adherence to standards, comprehensive documentation, and widespread sharing of data, code, and other auxiliary resources. This review provides critical insights into raw and processed data, metadata, high-throughput versus low-throughput datasets, a common language for documentation, experimental and reporting guidelines, efficient data management systems, sharing practices, and relevant repositories. We systematically present available resources and optimal practices for wide use by experimental biomedical researchers.

Artificial intelligence-assisted repair of peripheral nerve injury: a new research hotspot and associated challenges.

Artificial intelligence can be indirectly applied to the repair of peripheral nerve injury. Specifically, it can be used to analyze and process data regarding peripheral nerve injury and repair, while study findings on peripheral nerve injury and repair can provide valuable data to enrich artificial intelligence algorithms. To investigate advances in the use of artificial intelligence in the diagnosis, rehabilitation, and scientific examination of peripheral nerve injury, we used CiteSpace and VOSviewer software to analyze the relevant literature included in the Web of Science from 1994-2023. We identified the following research hotspots in peripheral nerve injury and repair: (1) diagnosis, classification, and prognostic assessment of peripheral nerve injury using neuroimaging and artificial intelligence techniques, such as corneal confocal microscopy and coherent anti-Stokes Raman spectroscopy; (2) motion control and rehabilitation following peripheral nerve injury using artificial neural networks and machine learning algorithms, such as wearable devices and assisted wheelchair systems; (3) improving the accuracy and effectiveness of peripheral nerve electrical stimulation therapy using artificial intelligence techniques combined with deep learning, such as implantable peripheral nerve interfaces; (4) the application of artificial intelligence technology to brain-machine interfaces for disabled patients and those with reduced mobility, enabling them to control devices such as networked hand prostheses; (5) artificial intelligence robots that can replace doctors in certain procedures during surgery or rehabilitation, thereby reducing surgical risk and complications, and facilitating postoperative recovery. Although artificial intelligence has shown many benefits and potential applications in peripheral nerve injury and repair, there are some limitations to this technology, such as the consequences of missing or imbalanced data, low data accuracy and reproducibility, and ethical issues (e.g., privacy, data security, research transparency). Future research should address the issue of data collection, as large-scale, high-quality clinical datasets are required to establish effective artificial intelligence models. Multimodal data processing is also necessary, along with interdisciplinary collaboration, medical-industrial integration, and multicenter, large-sample clinical studies.

F1000Research collaboration with the Research on Research Registry & Hub

F1000Research is proud to announce the new collaborative agreement between the Research on Research, Policy & Culture Gateway and the online repository for research on research (RoR) activity, the Research on Research Registry & Hub. This collaboration aims to connect scholars and organisations in the growing RoR discipline (the study of research practices also referred to as ‘meta-research’ or ‘science of science’), facilitating knowledge exchange and research transparency through project registration and open publication of outputs.

Can lateral tenodesis improve the rotational stability of the ACL reconstruction? A finite element analysis.

One of the most common knee injuries is the Anterior Cruciate Ligament (ACL) rupture with severe implications on knee stability. The usual treatment is the ACL Reconstruction (ACLR) surgery where the surgeon replaces the torn ligament with a graft in an effort to restore knee kinematics. In case of excessive rotatory instability, Lateral Extra-Articular Tenodesis (LET) can be performed in combination with ACLR. Additionally, LET appears to reduce ACLR graft forces minimizing graft failure chances. However, there are concerns about overconstraining physiological rotation. To gain insight in this controversial topic, we developed an automatic, open-source tool to create a series of Finite Element (FE) models attempting to investigate the interactions of ACLR and LET through simulation. We started by creating a validated model of the healthy knee joint that served as reference for subsequent FE simulations. Then, we created FE models of standalone ACLR and combined ACLR-LET. Each model was assessed by applying a loading profile that resembles the reduction phase of the Pivot-Shift clinical exam. We measured the External Tibia Rotation (ETR), the Posterior Tibia Translation (PTT) of the lateral tibial compartment, and the ACLR graft stress developed around the femoral tunnel insertion site. We observed the following: a) LET reduces ETR and PTT compared to isolated ACLR, b) combined ACLR-LET is more sensitive to LET graft pretension with lower values showcasing performance closer to the healthy joint, c) LET reduces ACLR graft forces for the same pretension values, d) LET exhibits significant overconstraint for higher pretension values. In general, these findings are in agreement with relevant clinical studies and accentuate the potential of the developed framework as a tool that can assist orthopaedists during surgery planning. We provide open access for the FE models of this study to enhance research transparency, reproducibility and extensibility.

Cartographic Metadata for Improving Accessibility and Facilitating Knowledge Extraction and Validation in Planetary Mapping Based on Remote-Sensing Observations

The field of planetary mapping and cartography builds almost exclusively on remote-sensing data and can be defined by three distinct concepts: systematic imaging as performed through spacecraft surveying, reference mapping as performed through the compilation of reference maps, i.e., regional to global image and topographic maps, and thematic mapping, which aims at abstracting and contextualizing spatial information to generate complex thematic maps, such as geologic or geomorphologic maps. While thematic mapping represents the highest form of abstraction of information that is provided through systematic mapping, thematic mapping also provides scientific reasoning in support of systematic mapping and exploration through spatially contextualized knowledge. For the development of knowledge, it is paramount to manage and exploit the value of thematic maps as research products, and to design a reliable and transparent development process from the beginning of the mapping phase as there is almost no validation for thematic maps. A key element in accomplishing these objectives is well-designed structures and metadata which are maintained within spatial data infrastructures (SDI) and shared as a coordinated process in research data management through data models. In this contribution, we focus on the need to transfer planetary thematic maps into findable, accessible, interoperable, reusable (FAIR), as well as transparent research data assets to facilitate improved knowledge extraction and also to compensate for limitations caused by the lack of conventional validation options. We review the current status of planetary thematic mapping, and we discuss the principles and roles of mappers and publishers in the process of creating and stewarding digital planetary maps and associated data products. We then present and discuss a set of recommendations that are closely tied to the FAIR concepts in research data management to accomplish such tasks.

Releasing synthetic data from the Avon Longitudinal Study of Parents and Children (ALSPAC): Guidelines and applied examples

The Avon Longitudinal Study of Parents and Children (ALSPAC) is a prospective birth cohort. Since its inception in the early 1990s, the study has collected over thirty years of data on approximately 15,000 mothers, their partners, and their offspring, resulting in over 100,000 phenotype variables to date. Maintaining data security and participant anonymity and confidentiality are key principles for the study, meaning that data access is restricted to bona fide researchers who must apply to use data, which is then shared on a project-by-project basis. Despite these legitimate reasons for restricting data access, this does run counter to emerging best scientific practices encouraging making data openly available to facilitate transparent and reproducible research. Given the rich nature of the resource, ALSPAC data are also a valuable educational tool, used for teaching a variety of methods, such as longitudinal modelling and approaches to modelling missing data. To support these efforts and to overcome the restrictions in place with the study’s data sharing policy, we discuss methods for generating and making openly available synthesised ALSPAC datasets; these synthesised datasets are modelled on the original ALSPAC data, thus maintaining variable distributions and relations among variables (including missing data) as closely as possible, while at the same time preserving participant anonymity and confidentiality. We discuss how ALSPAC data can be synthesised using the ‘synthpop’ package in the R statistical programming language (including an applied example), present a list of guidelines for researchers wishing to release such synthesised ALSPAC data to follow, and demonstrate how this approach can be used as an educational tool to illustrate longitudinal modelling methods.

Advocating for Generalizability: Accepting Inherent Variability in Translation of Animal Research Outcomes.

Advancing scientific discovery requires investigators to embrace research practices that increase transparency and disclosure about materials, methods, and outcomes. Several research advocacy and funding organizations have produced guidelines and recommended practices to enhance reproducibility through detailed and rigorous research approaches; however, confusion around vocabulary terms and a lack of adoption of suggested practices have stymied successful implementation. Although reproducibility of research findings cannot be guaranteed due to extensive inherent variables in attempts at experimental repetition, the scientific community can advocate for generalizability in the application of data outcomes to ensure a broad and effective impact on the comparison of animals to translation within human research. This report reviews suggestions, based upon work with National Institutes of Health advisory groups, for improving rigor and transparency in animal research through aspects of experimental design, statistical assessment, and reporting factors to advocate for generalizability in the application of comparative outcomes between animals and humans.

Is replication possible in qualitative research? A response to Makel et al. (2022)

ABSTRACT There has been much debate in recent years about how open research practices, which have been promoted in efforts to improve research robustness, may (not) be appropriate for qualitative methodologies, particularly in educational research. Among these is a concern for replication efforts. Makel et al. (2022) make the case that “replication is relevant to qualitative research”. The authors argue that concerns surrounding the transferability, intentionality, and transparency of qualitative research may be eased, or responded to, by replication studies. Here, I argue that there are three fundamental questions that need unpacking before declarative claims can be made about the relevance of replication to qualitative research. This includes critical questioning of what we mean by: replication, qualitative research, and rigour. I address each of these issues and encourage a more nuanced appraisal of how replication may, or may not, be epistemologically, ontologically, or methodologically compatible with the goals of qualitative research.

Improving Palliative Care Research Reporting: A Guide to Reporting Guidelines.

Improving the quality of research reporting is crucial for addressing current challenges in palliative care, with academic journals playing a crucial role in promoting clear and comprehensive reporting through structured guidelines. These guidelines, such as Appraisal of Guidelines, Research, and evaluation, Consolidated Standards of Reporting Trials, Case Reports (CARE) guidelines, transparent reporting of evaluations with nonrandomized designs (TREND), transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD), meta-analysis of observational studies in epidemiology (MOOSE) Checklist, methods of researching end-of-life care Statement, Preferred Reporting Items for Systematic Reviews and Meta-analyses, Standards for Quality Improvement Reporting Excellence 2.0, Strengthening the Reporting of Observational Studies in Epidemiology, standard protocol items: recommendations for interventional trials (SPIRIT), template for intervention description and replication (TIDieR) Consolidated Criteria for Reporting Qualitative Research and Standards for Qualitative Research, are instrumental in ensuring transparency by furnishing essential details for comprehending, replicating and applying research findings in clinical decision-making and systematic reviews. The Enhancing the quality and transparency of health research (EQUATOR) network champions trustworthy health research literature globally by advocating for transparent and accurate reporting, thereby enhancing the reliability and utility of research outcomes research outcomes.

Here, There and Everywhere: On the Responsible Use of Artificial Intelligence (AI) in Management Research and the Peer‐Review Process

AbstractThis editorial introduces and explains the Journal of Management Studies’ (JMS) new policy on artificial intelligence (AI). We reflect on the use of AI in conducting research and generating journal submissions and what this means for the wider JMS community, including our authors, reviewers, editors, and readers. Specifically, we consider how AI‐generated research and text could both assist and augment the publication process, as well as harm it. Consequentially, our policy acknowledges the need for careful oversight regarding the use of AI to assist in the authoring of texts and in data analyses, while also noting the importance of requiring authors to be transparent about how, when and where they have utilized AI in their submissions or underlying research. Additionally, we examine how and in what ways AI's use may be antithetical to the spirit of a quality journal like JMS that values both human voice and research transparency. Our editorial explains why we require author teams to oversee all aspects of AI use within their projects, and to take personal responsibility for accuracy in all aspects of their research. We also explain our prohibition of AI's use in peer‐reviewers’ evaluations of submissions, and regarding editors’ handling of manuscripts.

Using an expert survey and user feedback to construct PRECHECK: A checklist to evaluate preprints on COVID-19 and beyond

Background The quality of COVID-19 preprints should be considered with great care, as their contents can influence public policy. Surprisingly little has been done to calibrate the public’s evaluation of preprints and their contents. The PRECHECK project aimed to generate a tool to teach and guide scientifically literate non-experts to critically evaluate preprints, on COVID-19 and beyond. Methods To create a checklist, we applied a four-step procedure consisting of an initial internal review, an external review by a pool of experts (methodologists, meta-researchers/experts on preprints, journal editors, and science journalists), a final internal review, and a Preliminary implementation stage. For the external review step, experts rated the relevance of each element of the checklist on five-point Likert scales, and provided written feedback. After each internal review round, we applied the checklist on a small set of high-quality preprints from an online list of milestone research works on COVID-19 and low-quality preprints, which were eventually retracted, to verify whether the checklist can discriminate between the two categories. Results At the external review step, 26 of the 54 contacted experts responded. The final checklist contained four elements (Research question, study type, transparency and integrity, and limitations), with ‘superficial’ and ‘deep’ evaluation levels. When using both levels, the checklist was effective at discriminating a small set of high- and low-quality preprints. Its usability for assessment and discussion of preprints was confirmed in workshops with Bachelors students in Psychology and Medicine, and science journalists. Conclusions We created a simple, easy-to-use tool for helping scientifically literate non-experts navigate preprints with a critical mind and facilitate discussions within, for example, a beginner-level lecture on research methods. We believe that our checklist has potential to help guide decisions about the quality of preprints on COVID-19 in our target audience and that this extends beyond COVID-19.

Protocol for conducting a systematic review on diagnostic accuracy in clinical research

In the landscape of modern medicine, the ability to accurately diagnose various clinical conditions is paramount. As new diagnostic tools continue to emerge, their accuracy must be rigorously assessed before clinical implementation. This paper introduces a systematic review protocol tailored for diagnostic accuracy studies, drawing inspiration from a review on dysphagia screening in post-stroke patients. The protocol, designed with precision and transparency at its core, facilitates a thorough synthesis of evidence, employing tools such as the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) and the Standards for Reporting of Diagnostic Accuracy Studies (STARD) checklist for robust evaluation. The protocol emphasizes registration with the PROSPERO database and adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The systematic search approach encompasses a comprehensive exploration of databases and precise keyword combinations. Distinctive inclusion and exclusion criteria, coupled with a dual-reviewer methodology, ensure the selection of high-quality studies. This framework has the potential to serve as a benchmark for systematic reviews in diagnostic accuracy, highlighting the importance of standardization, transparency, and adaptability in clinical research. This approach paves the way for a research methodology that delves deeper into diagnostic tools across various clinical scenarios, promoting evidence-based advancements in patient care.

Peer review of GPT-4 technical report and systems card.

The study provides a comprehensive review of OpenAI's Generative Pre-trained Transformer 4 (GPT-4) technical report, with an emphasis on applications in high-risk settings like healthcare. A diverse team, including experts in artificial intelligence (AI), natural language processing, public health, law, policy, social science, healthcare research, and bioethics, analyzed the report against established peer review guidelines. The GPT-4 report shows a significant commitment to transparent AI research, particularly in creating a systems card for risk assessment and mitigation. However, it reveals limitations such as restricted access to training data, inadequate confidence and uncertainty estimations, and concerns over privacy and intellectual property rights. Key strengths identified include the considerable time and economic investment in transparent AI research and the creation of a comprehensive systems card. On the other hand, the lack of clarity in training processes and data raises concerns about encoded biases and interests in GPT-4. The report also lacks confidence and uncertainty estimations, crucial in high-risk areas like healthcare, and fails to address potential privacy and intellectual property issues. Furthermore, this study emphasizes the need for diverse, global involvement in developing and evaluating large language models (LLMs) to ensure broad societal benefits and mitigate risks. The paper presents recommendations such as improving data transparency, developing accountability frameworks, establishing confidence standards for LLM outputs in high-risk settings, and enhancing industry research review processes. It concludes that while GPT-4's report is a step towards open discussions on LLMs, more extensive interdisciplinary reviews are essential for addressing bias, harm, and risk concerns, especially in high-risk domains. The review aims to expand the understanding of LLMs in general and highlights the need for new reflection forms on how LLMs are reviewed, the data required for effective evaluation, and addressing critical issues like bias and risk.