Generalizing findings from a randomized controlled trial to a real-world study of the iLookOut, an online education program to improve early childhood care and education providers' knowledge and attitudes about reporting child maltreatment.

Chengwu Yang,Sarah Dore,Breanna Grable,Robert M Hamm,Erik B Lehman,Benjamin Levi,Nicole Verdiglione,Carlo Panlilio,David E Bard,Richard Fiene

doi:10.1371/journal.pone.0227398

Abstract

In recent years, real-world studies (RWS) are gaining increasing interests, because they can generate more realistic and generalizable results than randomized controlled clinical trials (RCT). In 2017, we published a RCT in 741 early childhood care and education providers (CCPs). It is the Phase I of our iLookOut for Child Abuse project (iLookOut), an online, interactive learning module about reporting suspected child maltreatment. That study demonstrated that in a RCT setting, the iLookOut is efficient at improving CCPs’ knowledge of and attitudes towards child maltreatment reporting. However, the generalizability of that RCT’s results in a RWS setting remains unknown. To address this question, we design and conduct this large RWS in 11,065 CCPs, which is the Phase II of the iLookOut. We hypothesize replication of the earlier RCT findings, i.e., the iLookOut can improve CCPs’ knowledge of and attitudes toward child maltreatment reporting in a real world setting. In addition, this RWS also explores whether demographic factors affect CCPs’ performance. Results of this RWS confirmed the generalizability of the previous RCT’s results in a real world setting. It yielded similar effect sizes for knowledge and attitudes as were found in the earlier RCT. Cohen’s d for knowledge improvement was 0.95 in that RCT, 0.96 in this RWS; Cohen’s d for attitude improvement was 0.98 in that RCT, 0.80 in this RWS. Also, we found several significant differences in knowledge and attitude improvement with regard to age, race, education, and employment status. In conclusion, iLookOut improves knowledge and attitudes of CCPs about child maltreatment prevention and reporting in a real-world setting. The generalizability of the initial RCT findings to this RWS provides strong evidence that the iLookout will be effective in other real world settings. It can be a useful model for other interventions aimed at preventing child maltreatment.Clinical trial registration for the original RCT: NCT02225301 (ClinicalTrials.gov Identifier)

Highlights

While randomized controlled trials (RCT) have long been seen as the “gold standard” for evaluating the efficacy of interventions, there are well-known limitations to their generalizability [1]
There have been growing interests in real-world studies (RWS) to generate real-world evidence (RWE) that are more realistic and generalizable [2,3,4,5,6,7,8,9], and RWE is increasingly valued by regulators and payers [10]
We published data from an RCT about the online educational intervention, the iLookOut for Child Abuse, showing that it improved early childhood care and education providers (CCPs) knowledge and attitudes about child maltreatment and its reporting [11]. In this follow-up study, through an RWS, we evaluate whether these results are generalizable to a broad population of CCPs in a real-world setting

Summary

Introduction

While randomized controlled trials (RCT) have long been seen as the “gold standard” for evaluating the efficacy of interventions, there are well-known limitations to their generalizability [1]. We published data from an RCT about the online educational intervention, the iLookOut for Child Abuse (iLookOut), showing that it improved early childhood care and education providers (CCPs) knowledge and attitudes about child maltreatment and its reporting [11]. In this follow-up study, through an RWS, we evaluate whether these results are generalizable to a broad population of CCPs in a real-world setting.

Methods

Results

Discussion

Conclusion