Transparency and reproducibility in the journal of contextual behavioral science: An audit study

Abstract

Increasing openness, transparency, and reproducibility in contextual behavioral science (CBS) through incorporating CBS-consistent open science practices was identified as a key aim of the ACBS Research Task Force. However, little data exist on the prevalence of open science practices currently being used in CBS research. This study aimed to address this gap by auditing the prevalence of open science and reproducibility practices in studies published in the Journal of Contextual Behavioral Science across 1 year, prior to the journal's adoption of open science recommendations (July 2020–July 2021). Aims of the study were twofold: first, to characterize current use of open science and reproducibility practices in JCBS to serve as a point for future comparison; second, to compare the rate of open science and reproducibility practices in JCBS, the flagship journal for contextual behavioral science, against two recently published audits of top clinical psychology journals. Domains audited were use of pre-registration, practices to ensure adequate power, data availability statements, use of standard reporting guidelines, preprints, conflict of interest statements, and resource and code sharing. Results indicated that studies published in JCBS had low rates of pre-registration, data availability statements, preprint posting, and resource and code sharing. Use of mandated standardized reporting guidelines and conflict of interest disclosures, both required by JCBS at the time of the audit, reflected relative strengths. Power for correlational studies was superior to power for experimental studies; the latter reflected a relative weakness compared to other clinical psychology journals. Rates of practices required by JCBS were significantly higher than those not required. JCBS may consider strongly encouraging or mandating other open science practices to incentivize researchers to use them.