PROTOCOL: Effectiveness of Early Family/Parent Training Programs in Preventing Antisocial Behavior & Delinquency

Abstract

A key observation in longitudinal studies of antisocial behavior, delinquency, and crime indicates that chronic disruptive behavior that emerges early in the life course leads to serious delinquency and crime during childhood, adolescence, and adulthood (McCord, Widom, and Crowell, 2001; Piquero, Farrington, and Blumstein, 2003) and also produces negative reverberations in other, non-crime life domains such as education, employment, relationship quality, and so forth (Moffitt, 1993). Because of this strong linkage or cumulative continuity over the life course and across life domains, it is not surprising to learn that early prevention has been suggested as an important avenue of policy with respect to early childhood problem behavior (Farrington and Welsh, 2007). And because children evincing early-life behavior problems become increasingly resistant to change over the life course (Frick and Loney, 1999; Tremblay, 2000), it becomes even more important to begin such efforts as early in the life course as possible, as these efforts may have a larger benefit when focused on high-risk families.1 One such vehicle includes early family-/parent-training (EFPT) programs. Such programs generally postulate that improving the quality of parent-child relations, which is a key feature of EFPT programs, will facilitate learning of control over impulsive, oppositional, and aggressive behavior, thus reducing disruptive behavior and its long-term negative impact on social integration (Bernazzini and Tremblay, 2006:22). In practice, such interventions attempt to change the social contingencies in the family context and/or provide advice/guidance to parents on raising their children or general parent education (Tremblay and Craig, 1995; Hawkins et al., 1999; Kazdin et al., 1992).2 Although a recent meta-analysis found that the two main types of family-based programs, general parent education (i.e., home visiting programs aimed at improving health and parenting skills and parent education plus daycare services) and parent management are effective in preventing delinquency or later criminal offending (Farrington and Welsh, 2003), and other reviews of the effectiveness of home visiting programs, including a systematic review (Bilukha et al., 2005) and a narrative review (Gomby et al., 1999) found that the evidence on child behavior outcomes was a bit more mixed, the totality of the evidence on early EFPT programs is muddied (Farrington and Welsh, 2007:122), largely because of the lack of a significant number of experimentally-based EFPT programs that contain long-term information on delinquency and crime. As background, we provide a brief overview of Farrington and Welsh's (2003) meta-analysis of the effectiveness of family-based crime prevention programs (carried out in several settings: home visiting programs, daycare/preschool programs, parent training programs, school-based programs, home/community programs with older adolescents, and multi-systemic therapy programs).3 Specifically, these authors included in their review studies that met the following criteria: (a) the family was the focus of the intervention4, (b) there was an outcome measure of delinquency or antisocial child behavior, (c) the evaluation used a randomized or well-controlled experiment, and (d) the original sample size included at least fifty persons. In general, while effect sizes were generally greater in smaller scale studies, the forty studies that met their criteria had a favorable effect on delinquency, with a decrease in offending from 50% in the control group to 34% in the experimental group. Additionally, the effects persisted in long-term evaluation studies. Their review also indicated that the most effective interventions employed behavioral parent training5, while the least effective were based in schools. Finally, home-visiting, day care/preschool, home/community, and multi-systemic therapy programs were generally effective. The specific focus of the current review is on early parent training programs through age 5 (of the child) in preventing delinquency. This focus permits us to compare our results to one previous review that we extend in important ways, to which we now turn our attention to. In a systematic review of early parent training interventions designed to impact children's delinquency limited to families with a child under age three at the start of the intervention (but without limits concerning the child's age at the end of the intervention), Bernazzani and Tremblay (2006) identified seven studies.6 Although the studies varied greatly with respect to outcome measures, child's age at evaluation, the nature and duration of the intervention and sample size, and the study's geographic location and its inclusion criteria (selective vs. universal), their analysis indicated that, overall, results concerning the effectiveness of parent training in the prevention of behavior problems in children were mixed: four studies reported no evidence of effectiveness, two reported beneficial effects, and one study reported mainly beneficial effects with some very minor harmful effects (p.26).7 Only one study in their review evaluated the effectiveness of home visitation and parent training on delinquency, and it reported very positive, crime-reduction effects (Olds et al., 1998). In short, it is still too early, from their review, to draw any definitive statement as to whether early parent training and support is effective in preventing disruptive behaviors in children and delinquency during adolescence. This is so because of the limited number of adequately designed studies, the results of the well-designed studies available are mixed and where positive often modest in magnitude, and very few studies were specifically designed to prevent disruptive behaviors in children. With respect to parent management training, several narrative and comprehensive vote-counting reviews, as well as one meta-analysis (Serketich and Dumas, 1996) provided support that this is an effective early family-based intervention to prevent delinquency and offending. And while cost/benefit analyses have been rare, Greenwood et al. (2001) reported a benefit/cost ratio of 4:1 of the Elmira nurse home visitation program (i.e., the Olds et al., 1998 study). Both Greenwood (2006) and Aos et al. (2004, 2006) have recently reported similar benefit/cost ratios for nurse home visitation programs generally, and EFPT programs in particular.8 To conclude this section, it is useful to repeat Farrington and Welsh's (2007:136) summary of the evaluation literature on this issue: “parent education plus daycare services and parent management training are effective in preventing delinquency and later offending. There is seemingly less consensus among evidence-based reviews on the effectiveness of parent education in the context of home visiting. Our meta-analytic review, based on four clearly defined, well-implemented, and methodologically rigorous home visitation programs, found that this form of early intervention was effective in preventing child antisocial behavior and delinquency. None of the other reviews (one a narrative review) utilized meta-analytic techniques to assess results, and in two of the reviews, programs other than home visiting were included. In our estimation, these differences go a long way toward explaining why these reviews found mixed results regarding the efficacy of home visiting.” The objective of this systematic review is to synthesize the extant empirical evidence (published and unpublished) on the effects of EFPT programs implemented up to age 5 in preventing later antisocial behavior and delinquency. The report will conform to the systematic review methodology and will incorporate meta-analytic techniques to assess results. It will build on and update (actually add and complete) the Bernazzani et al. (2001) and Bernazzani and Tremblay (2006) systematic reviews of the effectiveness of early parenting training programs (for families with children up to age 3) in preventing child disruptive behavior (i.e., opposition to adults, truancy, aggression) and delinquency. Their review included parent education in the context of home visiting and parent education plus daycare.9 As such, the primary question of this review is: What is the effectiveness of EFPT programs implemented up to age 5 in reducing later antisocial behavior and/or delinquency? When data are available, we will also collect information on cost-effectiveness of EFPT programs and their effect on antisocial behavior, delinquency, and crime. Following the earlier Bernazzani and Tremblay (2001:92) review and the more general Campbell reviews, the scope of this review is experimental and quasi-experimental studies including pre-post evaluations of family programs. Studies lacking control groups were excluded as they cannot help differentiate intervention effects from other effects including developmental effects. The preliminary eligibility criteria are as follows: Several strategies were used to perform an exhaustive search for literature fitting the eligibility criteria. First, a key word search was performed on an array of online abstract databases (see lists of keywords and databases below). Second, reviewed the bibliographies of four past reviews of EFPT programs (Mrazek and Brown, 1999; Tremblay, LeMarquand, and Vitaro, 1999; Bernazzani et al., 2001; Farrington and Welsh, 2007). Third, we performed forward searches for works that have cited seminal studies in this area.13 Fourth, we performed hand searches of leading journals in the field.14 Fifth, we searched the publications of several research and professional agencies (see list below). Sixth, after finishing the above searches and reviewing the studies as described later, we e-mailed the list to leading scholars knowledgeable in the specific area. These scholars were defined as those who authored two or more studies that appear on our inclusion list. These experts referred us to studies we may have missed, particularly unpublished pieces such as dissertations. Finally, we consulted with an information specialist at the outset of our review and at points along the way in order to ensure that we have used appropriate search strategies. The following databases were searched: The publications of the following groups were searched: The following agencies’ publications were searched and the agencies will be contacted if necessary: The following keywords were used to search the databases listed above: Several strategies were used to obtain full-text versions of the studies found through searches of the various abstract databases listed above. First, attempted to obtain full-text versions from the electronic journals available through the John Jay/CUNY library research port as well as those from the University of Maryland and the University of Louisville. When electronic versions were not available, we used print versions of journals available at the library. If the journals were not available at the university libraries, we used of the Interlibrary Loan Office (ILL) to try to obtain the journal from the libraries of other area schools. When these methods did not work, we contacted the author(s) of the article and/or the agency that funded the research to try to obtain a copy of the full-text version of the study. The studies included in this review will use methodologies that are variations of a treatment versus comparison group research designs with a post-test measure and possible follow-up measurement points. In all cases, the participant samples will be families and children, a sample of who will participate in the program and a sample who will not participate in the program. The studies varied with respect to the method of constructing the comparison group. The studies also varied with respect to the degree to which they employed statistical controls to reduce the threat of selection bias. All studies included in the review included a post-program measure of antisocial behavior, delinquency, or crime. These included dichotomous indicators or more differentiated indicators that indicated the specific types of offenses or the frequency of offenses committed. A few studies reported on other outcomes, such as school performance. We did not code these other outcomes, but documented all reported outcomes. Some studies reported multiple findings on different outcomes and/or different samples. In the case of independent samples, the results will be treated as separate findings and all such results will be included in the analysis. Other studies reported on outcomes for the same sample. For cases such as this with multiple findings from the same sample, each was examined independently to decide how to either combine the findings or to choose the one that best represents the study (recognizing that we do not wish to select the measure with the largest or smallest effect). Our final decision here was to pool together the separate effects into a single summary estimate because (1) different studies measure different antisocial and delinquent behaviors in different ways with respect to item wording, combined scales/indices, and time period referenced, and (2) we were primarily interested in the overall effect of the EFPT program on antisocial behavior/delinquency in general. (Note: We do indicate that future evaluation studies parcel out and specifically focus on the effect of EFPT on unique antisocial/delinquent behaviors). It is the case that many interventions were designed to deal with a specific outcome, but some also targeted some secondary problems and reported outcomes for these as well. In these cases the effect size for the primary outcome was reported. To be sure, different outcome contrasts were analyzed separately, and thus the independence issue was addressed. The issue, of course, is one that deals with multiple measures of the same construct or the same measure at multiple time points. Yet, we selected the best measure of a construct and analyzed these separately. So, for example, most studies had the primary outcome measure of interest be mother-reports of child delinquency. If other outcomes were also examined in the reviewed study (i.e., teacher reports of child delinquency), we analyzed these separately.15 All eligible studies were coded (see coding protocol attached in Appendix A) on a variety of criteria (including details related to them) including: Dr. Jennings will independently code each eligible study and consult with Dr. Piquero when questions arise in order to determine the final coding decision.16 Meta-analytic procedures were used to combine data from studies. For eligible studies, with enough data present, effect sizes were calculated using the standardized measures of effect sizes as suggested in the meta-analytic literature (Lipsey and Wilson, 2001). We also used the standardized mean difference effect size or odds-ratios, depending on the outcome construct. Mean effect sizes were computed across studies and we used a correction such as the inverse variance weight for computing the associated standard error.17 Though we examined the Q statistic to assess heterogeneity of effect sizes across studies, it is our initial assumption that study random factor in our analysis, that is, that there are meaningful differences across the studies affecting the size of the effect. As such, we implemented a random effects model for all analyses involving effect sizes. This is the case because EFPT strategies are diverse and they are oftentimes designed to ameliorate different types of problems. In this context, we believed that a mixed effects model will be most appropriate in analyzing the effectiveness of EFPT programs. We had originally hoped to examine contextual or moderating features of these programs. Though it was difficult to know at the outset, we thought it important to explore the differential effects of EFPT programs across different outcomes and across different types of treatments. Our goal was to assess this using the analog to the ANOVA method of moderator analysis (Lipsey and Wilson, 2001) for categorical moderator variables and meta-analytic regression analysis for continuous moderator variables or analyses involving multiple moderators. We also initially recognized that such analyses would be dependent on the number of studies that are available for inclusion in the meta-analysis. For reasons discussed above with respect to different outcomes, we did not continue further on this issue, but identified it as an important future research effort. Methodological quality will also be examined. For example, we will report the results of experimental and quasi-experimental studies combined and separately. We also plan to conduct moderator analysis of other method variables, such as attrition, similarity at baseline, or other method variations that are important within the more general prevention literature. Finally, publication bias is a concern in every meta-analysis. As such, we use traditional methods to test for the sensitivity of the findings to publication bias in the experimental and quasi-experimental studies. These methods include a comparison of the mean effect size for published and unpublished studies and a trim-and-fill analysis. We recognize that it may be difficult to rely upon traditional quantitative methods to deal with publication bias. Qualitative studies will not be included in the current review. Although we recognize that assessing study quality can be both an objective and a subjective exercise, we attempt to assess the quality of the studies in terms of research design, sample bias, and attrition bias, and make notation of study quality at various points in the review. The review process will adhere to the following schedule (and will likely be completed sooner than projected): The authors expect to update the review every five years. Drs. Piquero, Farrington, Welsh, Tremblay, and Jennings have no financial interest in any existing or planned family/parent program. Dr. Tremblay has been involved in several intervention and prevention programs in Canada; thus, the only potential conflict of interest is consistent with prior scholarly publications. The research team will strive to avoid any potential conflict. A study must meet the following criteria in order to be eligible. Answer each question with a “yes” or a “no” If the study does not meet the criteria above, answer the following question: Eligibility status: ____ Eligible ____ Not eligible ____ Relevant review Notes: ________________________________________________________________________ ________________________________________________________________________ __________ Publication type: ______ Date range of research (when research was conducted): The following questions are about the target population of the intervention (if the intervention is not targeting groups of problem people skip to question 38): What is the target population of the treatment? _____ Gender composition of target population: Age composition of target population Socio-economic status of target population: Race/ethnicity of the sample What was the type of intervention? Type of study: ____ Was the program highly structured, that is, followed a set protocol? Did the program remain consistent over time? Were there adjustments for baseline differences? Were there adjustments for attrition? Were there adjustments for differential attrition? Outcomes reported (Note that for each outcome, a separate coding sheet is required) What is the specific outcome recorded on this coding sheet?18 ________________________________________________________________________ Was it the primary outcome of the study? _______ Was this initially intended as an outcome of the study? ____ If no, explain why: ________________________________________________________________________ ________________________________________________________________________ __________ Dependent Variable What type of data was used to measure the outcome covered on this coding sheet? If official data was used, what specific type(s) of data were used? (Select all that apply) Did the researcher assess the quality of the data collected? Did the researcher(s) express any concerns over the quality of the data? If yes, explain ________________________________________________________________________ ________________________________________________________________________ Does the evaluation data correspond to the initially stated problem? (i.e., if the problem is delinquency, does the evaluation data look at whether delinquency decreased) If no, explain the discrepancy: ________________________________________________________________________ ________________________________________________________________________ If self-reports are used, were outcome data: Dependent Measure Descriptors Sample size Was attrition a problem in the analysis for this outcome? If attrition was a problem, provide details (e.g., how many cases lost and why they were lost). ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ _______________ Effect Size Data Raw difference favors (i.e., shows more success for): Did a test of statistical significance indicate statistically significant differences between either the control and treatment groups or the pre and post tested treatment group? ____ Was a standardized effect size reported? If no, is there data available to calculate an effect size? Type of data effect size can be calculated from: Means and Standard Deviations Proportions or frequencies Significance Tests Calculated Effect Size Note that the following questions refer to conclusions about the effectiveness of the intervention in regards to the current outcome/problem being addressed on this coding sheet. Conclusion about the impact of the intervention? ____ Did the author(s) conclude that the parent/family intervention beneficial? ____ Did the author(s) conclude there a relationship between the parent/family intervention and a reduction in delinquency/crime? ____ Additional notes about conclusions: ________________________________________________________________________ ________________________________________________________________________ _______________