The Use of Prospective Versus Retrospective Pretests with Child-Welfare Involved Families

Abstract

The purpose of the current study is to examine the use of both a prospective and a retrospective pretest in evaluating the impact of the Strengthening Families Program on foster care involved families affected by parental substance abuse. Debate has existed in the literature for over 60 years regarding the use of retrospective pretests in assessing self-reported program impact, with key stakeholders often arguing strongly that, due to response-shift bias, sensitization, and the nature of many human service settings, retrospective pretesting may be more appropriate. However, program evaluators must also incorporate into the evaluation design funding mandates to collect data at specific points in time. In order to mitigate potential pretest-only biases and enhance the evaluation’s rigor, this study sought to address the pretest debate through the use of both a traditional, prospective pretest and a retrospective pretest. Using data provided by 411 caregivers, program effectiveness was measured in the areas of family, child and parent functioning. Statistical significance tests and effect sizes were analyzed to compare traditional prospective pretest to posttest scores and retrospective pretests to posttest scores. Findings indicate that overall, the Strengthening Families Program positively impacted family, child and parent functioning and that there were few differences between testing approaches when testing for statistical significance; however, relative differences between prospective pretests and retrospective pretests appeared more prominent in effect size computations. This research informs the longstanding debate, and suggests that program evaluators consider the advantages and disadvantages of using a retrospective pretest in design planning.