Effectiveness Of Evidence-based Programs Research Articles

Implementing psychosocial support models in contexts of extreme adversity: lessons from a process evaluation in Colombia.

High quality investments during early childhood allow children to achieve their full potential by setting developmental foundations. However, challenges in the scale-up of evidence-based interventions make across-the-board implementation a non-trivial matter. Moreover, extreme contextual conditions -such as community violence, forced displacement, and poverty- impose a double threat. First, by directly affecting early childhood development (ECD), forced displacement and exposure to violence during early childhood, coupled with deficits in nurturing relationships, can trigger toxic stress, affecting children's mental health and social and emotional learning. Second, contexts of extreme adversity exacerbate common implementation pitfalls in the scale-up of interventions. Recognizing and documenting "what it takes" to successfully implement "what works" can contribute to the expansion and effectiveness of evidence-based programs that promote ECD in these settings. Semillas de Apego (SA, onward), a community-based psychosocial support model for caregivers, materialized as a strategy to promote ECD in communities affected by violence and forced displacement. This article presents the results of the process evaluation of SA during the 2018-2019 implementation in Tumaco, a violence ridden municipality in the south-west border of Colombia, South America. In this phase, the program reached 714 families, 82% direct victims of violence and 57% were internally displaced. The process evaluation combined qualitative and quantitative methodological approaches to produce evidence of factors that promoted implementation quality. Findings identified salient components of the program that promoted the program's acceptability, adoption, appropriateness, fidelity and sustainability: a rigorous cultural adaptation; well-structured team selection and training methodologies; and a team support and supervision protocol to provide continuous capacity building and prevent burn-out and other occupational hazards common among professionals in mental health and psychosocial support interventions. The statistical analysis using monitoring data identified key predictors of the dosage delivered (a measure of fidelity). Evidence suggests that initial attendance to the program and observable characteristics -such as educational attainment, violence victimization and employment status-predict a successful compliance (in terms of dosage to benefit from the program). This study provides evidence for the development of structural, organizational, and procedural processes for the adoption, appropriate adaptation, and high-fidelity delivery of psychosocial support models delivered in territories affected by extreme adversity.

A Randomized-Controlled Trial of the Effects of Online Training on Implementation Fidelity

The effectiveness of evidence-based programs can be compromised if the program is not implemented with fidelity. This article describes the evaluation of an online training to improve implementation fidelity to an evidence-based HIV prevention curriculum, Reducing the Risk, through online skills-based instruction and support. Educators were randomly assigned to either the online training or a self-preparation control group. The primary outcome was the proportion of lesson activities implemented completely. Measurement of the main outcome was obtained through self-report logs, which educators were asked to complete following each of the 16 lessons taught. Data were analyzed from 162 educators. The online training group reported significantly higher overall implementation fidelity compared to the standard control group with modest effect sizes (p =.04–.051 and Cohen's d = 0.31 – 0.33). Additionally, when fidelity scores were compared for just the core activities, implementation fidelity was significantly higher for the training group compared to the control group (p =.023; d = 0.36). The online training group also reported significant improvements in knowledge of the curriculum and perceptions of knowledge and skills in role-play improvement (p <.001–.013; d = 0.40–0.62). However, many of the other secondary outcomes analyzed were not significant. Online training with video demonstrations may be an effective way to enhance implementation fidelity in educators.

Preliminary Psychometric Properties of the BEST in CLASS Adherence and Competence Scale

The measurement of treatment integrity is critical to evaluate the efficacy and effectiveness of evidence-based programs (EBPs) designed to improve the developmental outcomes of young children at risk of emotional/behavioral disorders. Unfortunately, the science of treatment integrity measurement lags behind the development and evaluation of EBP for young, high-risk children. This article describes the development and preliminary psychometric properties of the BEST in CLASS Adherence and Competence Scale (BiCACS), designed to measure the adherence and competence of delivery of the BEST in CLASS prevention program. Independent observers coded videotaped ( n = 116) and live ( n = 289) observations of teachers delivering the BEST in CLASS program. The BiCACS showed good interrater reliability and analyses provided some support for the validity of the measure. Implications for future research and integrity measurement work are discussed.