Abstract
The high rate of statistically significant findings in the sciences that do not replicate in a new sample has been described as a "replication crisis." Few replication attempts have been conducted in studies of alcohol use disorder (AUD), and the best method for determining whether a finding replicates has not been explored. The goal of the current study was to conduct direct replications within a multisite AUD-randomized controlled trial and to test a range of replication metrics. We used data from a large AUD clinical trial (Project Matching Alcoholism Treatments and Client Heterogeneity [Project MATCH], n = 1,726) to simulate direct replication attempts. We examined associations between drinking intensity and negative alcohol-related consequences (Model 1), sex differences in drinking intensity (Model 2), and reductions in drinking following treatment (Model 3). We treated each of the 11 data collection sites as unique studies such that each subsample was treated as an "original" study, and the remaining 10 subsamples were viewed as "replication" studies. Replicability metrics included the consistency of statistical significance, overlapping confidence intervals, and consistency of the direction of the effect. We also tested effect replication and heterogeneity using meta-analysis. We observed between 0% and 100% replicability across the replicability metrics depending on which subsample was treated as the "original" study. Meta-analyses indicated results were more similar across subsamples with no significant heterogeneity for Models 1 and 2. We recommend researchers focus on effect sizes and use meta-analysis to evaluate the level of replicability. We also encourage direct replication attempts and sharing of data and code to facilitate direct replication. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.