Joining formative evaluation with translational science to assess an EBI in foster care: Examining social–emotional well-being and placement stability

Abstract

This study examined measures of placement stability and social–emotional well-being for a federally-funded demonstration project of inhome Parent Management Training-Oregon model (PMTO) for children in foster care with serious emotional disturbance (SED). Following a cultural exchange framework for translational research and a tollgate approach to evaluation, this formative study tested these measures prior to further investment in summative evaluation. The research aim was to observe whether measures of social–emotional well-being and placement stability performed as expected. Using a pretest–posttest randomized consent trial, children identified as SED within six months of entering foster care were randomly assigned to PMTO or to a services-as-usual comparison group (N=121). A multi-group structural equation model was tested to observe the relationship of baseline social–emotional wellbeing and placement stability on post-test social–emotional well-being. Results showed that post-test well-being was significantly predicted by baseline well-being for both groups and, importantly, only the intervention group demonstrated significant effects of placement stability on post-test well-being. For the intervention group, as placement stability increased, post-test social skills significantly improved, demonstrating an association between well-being and placement stability that was not evident in the comparison group. Overall, wellbeing measures performed as expected and detected relationships between variables and variation as hypothesized. Additionally, placement stability may have been mediated by the PMTO intervention. In conclusion, by using a real world example of the translational research concept of cultural exchange between university-based researchers and agency-based practitioners, this study shows that formative evaluation offers an important opportunity to test ad hoc research questions inspired by the process of implementation itself.