Assessing for Bias and Evidential Value: Introducing the P-Curve to Social Work Research

Abstract

High-quality research is essential for the creation and delivery of effective social welfare interventions and policies. Such research is a cornerstone of evidence-based practice and more generally for the cultivation of intra- and interdisciplinary knowledge. The ongoing “replication crisis” (Ioannidis, 2005; Yaffe, 2019) in the biomedical and social sciences suggests that social work’s evidence base may not be as robust as it appears. In other words, social work research may have suboptimal levels of false positive findings, bias, and irreproducibility, among other issues—which together undergird many aspects of unreplicable research. The replication crisis serves as an important reminder—that social work researchers must continuously assess the veracity of their findings (Howard & Garland, 2015) if they are to make professional, social, and epistemic progress. Indeed, the promotion and publication of poorly conducted research (Chambers, 2017; Dunleavy, 2020a, 2021), characterized in part by the selective reporting of results, p-hacking (i.e., continuing to collect or manipulating data until the associated result is statistically significant; see Head et al., 2015; Simonsohn et al., 2014), and publication bias (e.g., Chambers, 2017; Rosenthal, 1979; Simmons et al., 2011), can lead to the adoption of ineffective or harmful practices and undermine genuine advances in knowledge (Dunleavy, 2020b, 2021; Gambrill, 2011).