Kaminski et al. Respond

Abstract

We appreciate the opportunity to address issues of reporting the outcomes of parenting intervention models. We concur and have previously made similar conclusions that the parenting intervention literature often lacks critical information.1 While we acknowledge that more analyses could have been included had we published in monograph form, we defend reporting the findings of our intent-to-treat analyses in this first, single article–length publication, in that intent-to-treat analyses are well accepted as the most statistically conservative method for detecting programmatic effects. As noted in our article, we collected extensive data on other outcomes, potential moderators, intervention attendance, implementation fidelity, and program costs. We have articles in progress that report on the influence of mediating and moderating factors as well as economic analyses and the influence of intervention exposure. We encourage potential implementers to consider all of these issues, with information obtained either in our future publications or via direct discussion with the Centers for Disease Control and Prevention (CDC), before concluding that Legacy for Children (Legacy) is a good fit for a particular system, population, or community. We would like to address directly the assertion that our study lacked external validity and the related criticism that program effectiveness does not accurately characterize our study. Legacy was evaluated using a randomized controlled study design, which was selected to maximize the internal validity of findings at each site. When used in isolation, we acknowledge that this approach has been said to sacrifice external validity. Thus, we also intentionally included what Chen refers to as bottom-up strategies for increasing external validity.2 This included having both sites develop their own site curriculum that was consistent with the single Legacy model (i.e., built on the same philosophy, goals, and delivery method), selection of staff with different levels of education, variations in intervention dosage and periodicity, and inclusion criteria that focused on capturing (and succeeded in recruiting) a heterogeneous population of mothers of children born into poverty. Therefore, although we exercised a high level of control over implementation throughout the evaluation, we allowed for site adaptation in key areas that promoted community fit. These design decisions increased the external validity of our findings and renders the evaluation a hybrid of both efficacy and effectiveness. We hope this response provides clarity to our approach and future directions regarding CDC’s Legacy model. We look forward to additional discussions with researchers and communities who are committed to bringing evidence-based interventions to children living in poverty.