Transparency In Research Research Articles

AMS Publications Support for Open, Transparent, and Equitable Research

AMS Publications Support for Open, Transparent, and Equitable Research

Data Paper's Functions in Scholarly Communication Ecosystem as Perceived by Natural Scientists

ABSTRACTData papers, a new class of scholarly publication emerging from the open‐science movement, foster data discovery and reuse by offering comprehensive descriptions of research data. Yet, despite their promising growth, the role of data papers in scholarly communication remains underexplored. This work therefore investigates the perceived contributions and functions of data papers to scholarly communication by interviewing 14 data‐paper authors operating in the field of natural science. Using conceptual frameworks adopted from Borgman (2007) and Van de Sompel et al. (2004), we identify four general functions of scholarly communication (i.e., legitimization; dissemination; access, preservation, and curation; and rewarding). Additionally, our data lead us to propose that verification is a distinct scholarly communication, underscoring the importance of data papers in validating research findings in the context of ensuring research transparency. By elucidating the crucial role that data papers now play within the scholarly communication ecosystem, this study seeks to raise the academic community's awareness of their fundamental position, as well as their co‐existence with other forms of data publication, in advancing scientific research.

Big data management algorithms in artificial Internet of Things-based fintech

Research background: Fintech companies should optimize banking sector performance in assisting enterprise financing as a result of firm digitalization. Artificial IoT-based fintech-based digital transformation can relevantly reverse credit resource misdistribution brought about by corrupt relationship chains. Purpose of the article: We aim to show that fintech can decrease transaction expenses and consolidates firm stock liquidity, enabling excess leverage decrease and cutting down information asymmetry and transaction expenses across capital markets. AI- and IoT-based fintechs enable immersive and collaborative financial transactions, purchases, and investments in relation to payment tokens and metaverse wallets, managing financial data, infrastructure, and value exchange across shared interactive virtual 3D and simulated digital environments. Methods: AMSTAR is a comprehensive critical measurement tool harnessed in systematic review methodological quality evaluation, DistillerSR is harnessed in producing accurate and transparent evidence-based research through literature review stage automation, MMAT appraises and describes study checklist across systematic mixed studies reviews in terms of content validity and methodological quality predictors, Rayyan is a responsive and intuitive knowledge synthesis tool and cloud-based architecture for article inclusion and exclusion suggestions, and ROBIS appraises systematic review bias risk in relation to relevance and concerns. As a reporting quality assessment tool, the PRISMA checklist and flow diagram, generated by a Shiny App, was used. As bibliometric visualization and construction tools for large datasets and networks, Dimensions and VOSviewer were leveraged. Search terms were “fintech” + “artificial intelligence”, “big data management algorithms”, and “Internet of Things”, search period was June 2023, published research inspected was 2023, and selected sources were 35 out of 188. Findings & value added: The growing volume of financial products and optimized operational performance of financial industries generated by fintech can provide firms with multifarious financing options quickly. Big data-driven fintech innovations are pivotal in banking and capital markets in relation to financial institution operational efficiency. Through data-driven technological and process innovation capabilities, AI system-based businesses can further automated services.

Toward open science in PLS-SEM: Assessing the state of the art and future perspectives

Driven by the high-profile failures to reproduce and replicate published findings, there have been increasing demands to adopt open science practices across scientific disciplines in order to enhance research transparency. Critics have highlighted the use of underpowered studies and researchers’ analytical degrees of freedom as factors contributing to these issues. Despite methodological advances and updated guidelines, similar concerns persist regarding studies utilizing partial least squares structural equation modeling (PLS-SEM). Open science practices can help alleviate these concerns by facilitating transparency in PLS-SEM-based studies. However, the current level of adherence to these practices remains unknown. In this article, we conduct a comprehensive literature review of leading marketing journals to assess the extent to which open science practices are implemented in PLS-SEM-based studies. Based on the observed lack of adoption, we propose a PLS-SEM-specific preregistration template that researchers can use to foster transparency in their analyses, thereby bolstering confidence in their findings.

A Survey of Submission and Review Standards across Clinical Radiation Oncology and Medical Physics Journals: Devising the Optimal Criteria for Reproducibility and Sustained Impact

Publishing and editorial policies differ substantially across the Radiation Oncology (RO) and Medical Physics (MedPhys) compendium of journals. Adoptance of modern standards in scientific publishing and data sharing have the potential to improve the impact and reliability of the RO literature. We characterized the editorial, authorship and peer reviewer policies of various prominent clinical RO (N = 16) and medical physics (N = 9) peer-reviewed journals affiliated with professional societies for characteristics that are associated with improved reproducibility and rigorous review. A combination of tools including Enhancing the QUAlity and Transparency Of health Research (EQUATOR), Findability, Accessibility, Interoperability, and Reuse (FAIR), and Quality Output Checklist and Content Assessment (QuOCCA) principles were used to quantify the value and reproducibility of journal policies. Cohen's kappa coefficient was utilized to assess agreement between reviewers. Components of the above tools were regressed against various scientometric indices (H-index, IF, etc.) to identify factors that are associated with perceived relative importance within the field. Reviewer agreement (κ) for scientometric indices was highest (1.0) for criteria for statistical review and data submission standards and lowest (-0.246) for various submission checklists. Data availability statements were endorsed (44%) or required (31%) in a higher proportion of RO journals relative to MedPhys journals (44%, 0% respectively). Data repository submission was required in <10% of journals. FAIR adoptance was poor (31%, 22%) in RO and MedPhys journals. ≥1 EQUATOR guideline checklist was endorsed or required in 76% of journals. While there were no glaring differences in editorial policies between RO and MedPhys journals, there was substantial heterogeneity of scientometrics evaluating the rigor of data submission, reproducibility standards, and statistical review criteria. Linear regression of journal impact factors indicated a predictive relationship between FAIR adoption standards, use of EQUATOR checklists, and more rigorous statistical method submission criteria. The present review documented and confirmed significant variation in submission, review, and publication policies across RO and MedPhys journals. Established scientometric standards, FAIR principle adoptance, and more rigorous statistical methodology were predictive of increasing journal impact factor.

Simulation Analysis for Influential Mechanism of Fuzzy Set Data Calibration Toward Consistency Analysis of fsQCA

Fuzzy set qualitative comparative analysis (fsQCA) is a new method to solve complex causal relationship analysis in social science, and data calibration is a core process of fsQCA. Ignoring data calibration will have an impact on the fsQCA consistency analysis, thereby undermining the rigor of the causal mechanism between fsQCA mining conditions and results. This study found that the distribution characteristics of the data did not have a significant impact on the final consistency and the corrected consistency, while the anchor point setting of the crosspoint had a significant impact on the sufficient conditional consistency of fsQCA. When the consistency study is carried out on the crosspoint anchor point setting and calibration transformation, it is found that the effect of crosspoint anchor point setting on consistency is more obvious than that of calibration transformation. The study of the consistency between fuzzy set data calibration and fsQCA can provide useful conclusions and calibration methods for the empirical analysis of fsQCA, reminding researchers that they should avoid using mechanical procedures to perform simple data calibration to obtain misleading results and standardize fuzzy set data. Calibration is beneficial to improve the transparency of fsQCA research and also provides an important reference for fsQCA practitioners to conduct robust analysis of different data-driven methods.

Automatic Extraction of Research Themes in Epidemiological Criminology From PubMed Abstracts From 1946 to 2020: Text Mining Study.

The emerging field of epidemiological criminology studies the intersection between public health and justice systems. To increase the value of and reduce waste in research activities in this area, it is important to perform transparent research priority setting considering the needs of research beneficiaries and end users along with a systematic assessment of the existing research activities to address gaps and harness opportunities. In this study, we aimed to examine published research outputs in epidemiological criminology to assess gaps between published outputs and current research priorities identified by prison stakeholders. A rule-based method was applied to 23,904 PubMed epidemiological criminology abstracts to extract the study determinants and outcomes (ie, "themes"). These were mapped against the research priorities identified by Australian prison stakeholders to assess the differences from research outputs. The income level of the affiliation country of the first authors was also identified to compare the ranking of research priorities in countries categorized by income levels. On an evaluation set of 100 abstracts, the identification of themes returned an F1-score of 90%, indicating reliable performance. More than 53.3% (11,927/22,361) of the articles had at least 1 extracted theme; the most common was substance use (1533/11,814, 12.97%), followed by HIV (1493/11,814, 12.64%). The infectious disease category (2949/11,814, 24.96%) was the most common research priority category, followed by mental health (2840/11,814, 24.04%) and alcohol and other drug use (2433/11,814, 20.59%). A comparison between the extracted themes and the stakeholder priorities showed an alignment for mental health, infectious diseases, and alcohol and other drug use. Although behavior- and juvenile-related themes were common, they did not feature as prison priorities. Most studies were conducted in high-income countries (10,083/11,814, 85.35%), while countries with the lowest income status focused half of their research on infectious diseases (47/91, 52%). The identification of research themes from PubMed epidemiological criminology research abstracts is possible through the application of a rule-based text mining method. The frequency of the investigated themes may reflect historical developments concerning disease prevalence, treatment advances, and the social understanding of illness and incarcerated populations. The differences between income status groups are likely to be explained by local health priorities and immediate health risks. Notable gaps between stakeholder research priorities and research outputs concerned themes that were more focused on social factors and systems and may reflect publication bias or self-publication selection, highlighting the need for further research on prison health services and the social determinants of health. Different jurisdictions, countries, and regions should undertake similar systematic and transparent research priority-setting processes.

The PopuList: A Database of Populist, Far-Left, and Far-Right Parties Using Expert-Informed Qualitative Comparative Classification (EiQCC)

AbstractWith a proliferation of scholarly work focusing on populist, far-left, and far-right parties, questions have arisen about the correct ways to ideologically classify such parties. To ensure transparency and uniformity in research, the discipline could benefit from a systematic procedure. In this letter, we discuss how we have employed the method of ‘Expert-informed Qualitative Comparative Classification’ (EiQCC) to construct the newest version of The PopuList (3.0) – a database of populist, far-left, and far-right parties in Europe since 1989. This method takes into account the in-depth knowledge of national party experts while allowing for systematic comparative analysis across cases and over time. We also examine how scholars have made use of the previous versions of the dataset, explain how the new version of The PopuList differs from previous ones, and compare it to other data. We conclude with a discussion of the strengths and limitations of The PopuList dataset.

Registering transparency: the making of the international clinical trial registry platform by the world health organization (2004–2006)

BackgroundThis paper examines the events and conditions that led to the creation of the International Clinical Trials Registry Platform (ICTRP) in 2006 by the World Health Organization (WHO), and how the WHO addressed the issue of transparency in global pharmaceutical research. Using historical textual analysis, I trace the scientific debates that advocated for the establishment of official clinical trial registries in medical journals, and the sequence of actions following the GSK Paxil scandal in 2004, identifying the major ethical and scientific arguments that led to the involvement of the WHO as a key actor in trial registration in the context of the Big Pharma business model.ResultsThrough the questions “Why register?” and “Why registries?” as a roadmap, I examine the issues of publication bias and selective reporting by the industry, scrutinizing two ways in which the practice of publication bias damaged transparency in industry-sponsored research. The first involved ethical concerns regarding human subject exploitation and concealing of negative results. The second addresses the deterioration of the certainty of evidence due to incomplete access to trials results. By reviewing the series of events that occurred between 2004 and 2006 –between the Paxil scandal and the launch of the ICTRP—, I analyze the actions taken by the different actors involved: (1) the International Committee of Medical Journal Editors (ICMJE) and the creation of the Ottawa Group; (2) the WHO, beginning with the Ministerial Summit on Health Research held in November of 2004, and (3) the responses of the pharmaceutical industry and specifically GSK to the call for transparency and trial registration.ConclusionsThe history of trial registration through the ICTRP as a dataveillance apparatus shows the difficulty of regulating a health enterprise turned into a global business. Moreover, it shows the challenges of globalization and how easier and faster it is to globalize business compared to good practices, raising the question of why it has been so hard to undo these trends. Indeed, the history of the movement for trial registration is not a history of regulation success, or at least not yet.

Directed acyclic graphs for clinical research: a tutorial.

Directed acyclic graphs (DAGs) are useful tools for visualizing the hypothesized causal structures in an intuitive way and selecting relevant confounders in causal inference. However, in spite of their increasing use in clinical and surgical research, the causal graphs might also be misused by a lack of understanding of the central principles. In this article, we aim to introduce the basic terminology and fundamental rules of DAGs, and DAGitty, a user-friendly program that easily displays DAGs. Specifically, we describe how to determine variables that should or should not be adjusted based on the backdoor criterion with examples. In addition, the occurrence of the various types of biases is discussed with caveats, including the problem caused by the traditional approach using p-values for confounder selection. Moreover, a detailed guide to DAGitty is provided with practical examples regarding minimally invasive surgery. Essentially, the primary benefit of DAGs is to aid researchers in clarifying the research questions and the corresponding designs based on the domain knowledge. With these strengths, we propose that the use of DAGs may contribute to rigorous research designs, and lead to transparency and reproducibility in research on minimally invasive surgery.

Are Nonprobability Surveys Fit for Purpose?

Abstract Social scientists employ survey methods to explore the contours of human behavior. Today there are more opportunities to collect survey data than at any time in recent history. Yet sample quality varies dramatically due in part to the availability of nonprobability samples (NPSs) from commercial survey organizations. While these kinds of surveys have advantages in terms of cost and accessibility, the proprietary nature of the data can be problematic. In this synthesis, we describe situations in which researchers typically employ NPSs and consider whether these data are fit for purpose. Next, we discuss use cases that are not widespread but may be appropriate for these data. We conclude that potential utility of NPSs will remain out of reach unless scholars confront the tension between the operation of online survey organizations and the goals of transparent research.

The role of global open data infrastructure in a changing policy, technology, and research environment

Watch VIDEO. The proliferation of open data sharing policies (such as the wide-reaching NIH policy and Nelson Memo in the United States) reflect a growing push for greater collaboration, transparency, and accountability in scientific research. They assert that the open sharing of data underpinning research is essential to achieving the benefits of open science, that researchers and the general public have a right to access that data, and that the value of shared data lies in its (re)usability. As the scientific community rallies to meet these new requirements, research data infrastructures like Dryad, an open data publishing platform and community, face myriad opportunities and challenges. In this talk, the presenters will summarize the impact that new policies, evolving researcher and institutional needs, coordinated support for open infrastructure, and other changes in the landscape are having on the Open Data movement and those that support it; provide insight into Dryad’s experience addressing challenges like increasing file size and complexity, and simplifying data deposition workflows; explore the opportunities and challenges presented by emerging new technologies such as machine learning, automation, and distributed data storage; outline the ongoing relevance of non-specialist data infrastructures in the rapidly evolving research landscape, and how their role will have to change through collaboration and partnerships.

It’s the incentives, stupid!

Watch VIDEO. From 2004, a new incentives system for scholarly publishing was introduced in Norway. Part of the funding of the HE institutions has since been based on the amount of scholarly publishing produced per institution, as recorded in the Norwegian CRIS system, Cristin. And the incentives worked. Publishing increased more rapidly in Norway than in comparable countries, over the years following 2004 (Aagard et al. 2014). But the sheer number of journal articles, anthologies and monographs is not necessarily what best brings science forward. And now the Norwegian government has announced that the publishing count will be dropped from the funding scheme (Meld. St. 14). Thus, we suggest that the right move would be to instead direct the incentives to stimulate good, transparent open science research practice. The advantages of open science, both for further research advancement and the benefit to society, are well documented (Miedema 2022). This includes early and obstacle-free access to research output in terms of articles, peer review reports and commentaries, as well as access to the research processes with preregistration and registered reports, methods and protocols, as well as FAIR research data that enables reproducible research. So, what is needed is an incentive system that stimulates such research practices. We suggest an incentive system, through the funding schemes of the institutions. This should favour institutions that follow open science practices. The institutions will thus be incentivized to stimulate open science, and identify and remove possible bottlenecks in their organisations that may hamper execution of open science. As employers of researchers as well as support staff, the managers of the institutions will look for ways to optimize local incentives, and also how their organisation may be designed, to best live up to the ideals of open science practices. Based on the above, we suggest a systematic evaluation of each HE institution, to document how well they practice such openness in their research. To accomplish this a list of criteria that can be easily monitored and objectively evaluated is needed. Several of the open research output parameters should be easy to monitor and measure through a reporting system. The evaluation should also acknowledge that necessary limitations to open processes and outputs exists, and thus avoid biases between institutions due to the nature of their research. Incentives directed towards the institutional level to stimulate open research practices will motivate the management to stimulate their organisation in the desired direction.

Teaching Reproducible Methods in Economics at Liberal Arts Colleges: A Survey

Economics has become increasingly empirical and, alongside this shift, has come more demand for improved transparency and reproducibility in empirical economic research. In this article, we distribute a survey to almost 1500 economics faculty from the top 161 liberal arts colleges with an economics major (according to U.S. News & World Report) in the United States to determine the prevalence of teaching reproducible methods in undergraduate economics, summarize the most-common methods of instruction, and determine the intended student learning objectives. We find that of the economics faculty at liberal arts colleges who teach these reproducible methods, most do so in advanced upper-level (42%) and basic econometrics (31%) courses. Those faculty report teaching reproducibility using the following methods: transparent coding (85%), organizational skills (78%), and producing replication documentation (47%) through individual research projects (82%), homework assignments (55%), and/or workshops (33%). We conclude with some qualitative text analysis to shed light on the intended learning objectives and find that research skills (59%) and the importance of reproducibility (37%) are the most common reasons cited for teaching these methods.

Organizing Scholarly Knowledge in the Open Research Knowledge Graph

The Open Research Knowledge Graph (ORKG) is an Open Science digital infrastructure for the production, curation, publication, and reuse of machine-actionable scholarly knowledge. Built on top of the RDF data model and extensible ontologies, the ORKG provides a common vocabulary for researchers to describe their research contributions and data, improving the discoverability and reusability of scholarly knowledge and research data. The ORKG includes tools for visualizing the relationships between different entities, making it easier to understand the connections between different pieces of research and their findings. It facilitates collaboration between researchers by providing a collaborative platform for organizing and sharing scholarly knowledge and data, reducing duplication and enabling more efficient use of resources. As research becomes increasingly data-driven, tools like the ORKG will become essential for enabling efficient, transparent, and collaborative research.

Enhancing the accessibility and interactions of regional hydrologic projections for water managers

Growing challenges of climate change require urgent shifts in scientific research to inform environmental decision-making. In the context of water management of the Colorado River Basin (CRB), the impacts of intensifying forest disturbances under climate change are not well understood but expected to have major consequences. To address this, we developed a web-based tool, CRB-Scenario-Explorer, that provides interactive visual assessments of modelled future CRB hydrology scenarios and detailed documentation of our approach to support mindful interpretations. Water manager feedback and user-experience principles were incorporated to achieve a user-centered design. Stakeholders confirmed the effectiveness of the web-based tool in assisting with the discovery that future CRB hydrology appears more sensitive to climate uncertainty than forest disturbances, which can be used to brief leaders and spark discourse around risk management. CRB-Scenario-Explorer thus exemplifies a novel and effective method to increase the accessibility, applicability, and transparency of environmental science research.

Addressing Low-Level Reproduction in Academic Publication: Implications for Integrity and Academic Environment

This paper addresses the issue of low-level reproduction in academic publication, which is considered a form of imperceptible academic corruption. The practice of low-level reproduction creates academic bubbles and undermines the integrity of the academic atmosphere. In response, this paper advocates for the banning of low-level reproduction to uphold the principles of scientific integrity within the scientific community. By doing so, it aims to foster an environment that nurtures critical thinking, creativity, and problem-solving skills among researchers, ultimately benefiting society and the nation as a whole. The paper emphasizes the importance of promoting transparent and original research in academic publishing to maintain the credibility and trustworthiness of scholarly work.

The UK Reproducibility Network: A progress report

There is growing awareness that the ways in which academic research is conducted could be improved. A number of exciting innovations are emerging, alongside a broader agenda that includes a growing emphasis on open research. This short article outlines the rationale, progress and plans of the UK Reproducibility Network, which is one of a growing number of similar initiatives internationally that promote more rigorous and transparent research.

Metaresearch study highlights the gender gap in randomized controlled trials in dentistry

Background and ObjectivesThis study aimed to assess the gender gap in randomized controlled trials (RCTs) in dentistry in terms of authorship, collaborations, metrics, funding and reporting of good research practice and transparecy. MethodsThe search was performed in PubMed for RCTs restricted to English texts in the dental field, indexed from 12/31/2016 to 12/31/2021. Two reviewers screened the studies in line with the eligibility criteria. A total of 844 articles were included. The name and gender of authors, citation metrics, funding, reporting of characteristics of good research practice and transparency were extracted. We considered "collaboration between authors" when the corresponding author was different from the first author. ResultsThe proportion of women as first authors was 46.56% and 40.12% for corresponding authors. The analysis showed that when a woman is the corresponding author, the probability of the first author also being a woman is 57% higher compared to the first author being a man. For “protocol registration” and “data sharing,” the prevalence of reporting was higher when women were first authors. ConclusionA gender gap in dentistry RCTs was identified and related to the participation of women as first and corresponding authors and the collaboration between authors.

A detailed examination of reporting procedural fidelity in the Journal of Applied Behavior Analysis.

Few reviews on procedural fidelity-the degree to which procedures are implemented as designed-provide details to gauge the quality of fidelity reporting in behavior-analytic research. This review focused on experiments in the Journal of Applied Behavior Analysis (2006-2021) with "integrity" or "fidelity" in the abstract or body. When fidelity data were collected, the coders characterized measurement details (e.g., description of calculation, report of single or multiple values, frequency of fidelity checks, checklist use). The researchers found increasing trends in describing the calculation(s), reporting multiple values, and stating the frequency of measurement. Few studies described using a checklist. Most studies reported fidelity as a percentage, with high obtained values (M = 97%). When not collecting fidelity data was stated as a limitation, authors were unlikely to provide a rationale for the omission. We discuss recommendations for reporting procedural fidelity to increase the quality of and transparency in behavior-analytic research.