Treatment Fidelity Workgroup Research Articles

Developing the Workforce of the Digital Future: mHealth Competency and Fidelity Measurement in Community-Based Care

Integrating mobile health (mHealth) interventions into settings that serve diverse patient populations requires that prerequisite professional competencies are delineated and that standards for clinical quality assurance can be pragmatically assessed. Heretofore, proposed mHealth competencies have been broad and have lacked a framework to support specific applications. We outline the meta-competencies identified in the literature relevant to mHealth interventions and demonstrate how these meta-competencies can be integrated with population- and intervention-related competencies to help guide a pragmatic approach to competency assessment. We present a use case based on FOCUS—an evidence-based mHealth intervention designed for individuals with serious mental illness and currently being implemented in geographically and demographically diverse community behavioral health settings. Subsequent to identifying the cross-cutting competencies relevant to the target population (outpatients experiencing psychotic symptoms), substratal intervention (Cognitive Behavioral Therapy for psychosis), and treatment modality (mHealth), we detail the development process of an mHealth fidelity monitoring system (mHealth-FMS). We adhered to a published sequential 5-step process to design a fidelity monitoring system that aligns with our integrated mHealth competency framework and that was guided by best practices prescribed by the Treatment Fidelity Workgroup of the National Institutes of Health Behavior Change Consortium. The mHealth-FMS is intended to enhance both clinical and implementation outcomes by grounding the mHealth interventionist and the system of care in which they operate in the core functions, tasks, knowledge, and competencies associated with system-integrated mHealth delivery. Future research will explore acceptability and feasibility of the mHealth-FMS.

Reporting of Treatment Fidelity in Mindfulness-Based Intervention Trials: A Review and New Tool using NIH Behavior Change Consortium Guidelines.

Mindfulness-based interventions (MBIs) are in the fourth decade of adaptation and testing, yet little is known about their level of treatment fidelity. Treatment fidelity is a methodological strategy used to monitor and enhance the reliability and validity of behavioral interventions. The Treatment Fidelity Workgroup of the National Institutes of Health Behavior Change Consortium (BCC) put forth recommendations covering five components of treatment fidelity: design, training, delivery, receipt, and enactment. We conducted a literature review to describe how these five components of treatment fidelity are reported in published main outcomes articles of MBI efficacy trials among adult participants. Our search yielded 202 articles and we identified 25 (12%) described study treatment fidelity. All 25 studies reported on design, n=24 (96%) reported on training, n=23 (92%) reported on delivery, n=23 (92%) reported on receipt, and n=16 (64%) reported on enactment. Eleven (44%) articles analyzed measures of receipt and enactment with a participant outcome. Fourteen (56%) articles reported on all five fidelity components. There was high variation in the way each component was conducted and/or reported, making comparisons across articles difficult. To address the prevailing limitation that the majority of MBI efficacy studies did not detail treatment fidelity, we offer the Treatment Fidelity Tool for MBIs adapted from the BCC guidelines to help researchers monitor and report these methods and measures in a simple and standardized format. By using this tool, researchers have the opportunity to improve the transparency and interpretability of the MBI evidence base.

Strategies for supporting intervention fidelity in the rehabilitation therapy in older acute heart failure patients (REHAB-HF) trial

IntroductionAcute decompensated heart failure (ADHF) is the leading cause of hospitalization in older adults. Rehabilitation Therapy in Older Acute Heart Failure Patients (REHAB-HF) trial is a multi-site clinical trial to determine if physical rehabilitation intervention in older patients with ADHF improves physical function and reduces rehospitalizations. The REHAB-HF intervention aims to improve functional performance utilizing reproducible and progressive exercises that are individually tailored to the patient's physiological and physical capabilities. Fidelity of the intervention is essential to the trial's integrity and success. Maintaining fidelity is challenged by the complex, multi-domain design of the intervention implemented across multiple sites and delivered to an older, heterogeneous participant pool with severe underlying disease and multi-morbidity. Methods/designGiven the dynamic nature of the REHAB-HF intervention, rigorous fidelity strategies were formulated. In this paper we summarize the specific strategies that REHAB-HF is using to meet the National Institutes of Health (NIH) Behavior Change Consortium Treatment Fidelity Workgroup recommendations in 5 key areas: 1) ensuring the intervention dose is consistent across participants, 2) standardizing interventionist training, 3) monitoring intervention delivery, 4) evaluating participants' understanding of information provided, and 5) ensuring that participants use the skills taught in the intervention. DiscussionEffective intervention fidelity strategies are essential to the reliability and validity of physical function intervention trials. The REHAB-HF trial has developed comprehensive, specific strategies to ensure intervention fidelity despite a challenging study population and a complex intervention to meet NIH recommendations. This experience provides a strong working model for future physical function intervention trials.

Fidelity considerations in translational research: Eating As Treatment - a stepped wedge, randomised controlled trial of a dietitian delivered behaviour change counselling intervention for head and neck cancer patients undergoing radiotherapy.

BackgroundThe confidence with which researchers can comment on intervention efficacy relies on evaluation and consideration of intervention fidelity. Accordingly, there have been calls to increase the transparency with which fidelity methodology is reported. Despite this, consideration and/or reporting of fidelity methods remains poor. We seek to address this gap by describing the methodology for promoting and facilitating the evaluation of intervention fidelity in The EAT (Eating As Treatment) project: a multi-site stepped wedge randomised controlled trial of a dietitian delivered behaviour change counselling intervention to improve nutrition (primary outcome) in head and neck cancer patients undergoing radiotherapy.Methods/DesignIn accordance with recommendations from the National Institutes of Health Behaviour Change Consortium Treatment Fidelity Workgroup, we sought to maximise fidelity in this stepped wedge randomised controlled trial via strategies implemented from study design through to provider training, intervention delivery and receipt. As the EAT intervention is designed to be incorporated into standard dietetic consultations, we also address unique challenges for translational research.DiscussionWe offer a strong model for improving the quality of translational findings via real world application of National Institutes of Health Behaviour Change Consortium recommendations. Greater transparency in the reporting of behaviour change research is an important step in improving the progress and quality of behaviour change research.Trial registration numberACTRN12613000320752 (Date of registration 21 March 2013)

Establishing Treatment Fidelity in Evidence-Based Parent Training Programs for Externalizing Disorders in Children and Adolescents

The current review evaluates the use of treatment fidelity strategies in evidence-based parent training programs for treating externalizing disorders. We used a broad framework for evaluating treatment fidelity developed by the National Institutes of Health Treatment Fidelity Workgroup that includes the aspects of treatment design, treatment delivery, training providers, and assessment of participant receipt of treatment and enactment of treatment skills. Sixty-five articles reporting outcome trials of evidence-based parent training programs met inclusion criteria and were coded for treatment fidelity strategies. The mean adherence to fidelity strategies was .73, which was higher than two previous review studies employing this framework in the health literature. Strategies related to treatment design showed the highest mean adherence (.83), whereas training of providers and enactment of treatment skills had the lowest (.58). In light of an increasing emphasis on effectiveness and dissemination trials, the broader treatment fidelity framework as applied in this review focuses needed attention on areas often overlooked in fidelity practices, such as training providers and generalization of treatment skills. We discuss the strengths and limitations of fidelity practices in parent training studies, implications of these findings, and areas for future research.

The Reporting of Fidelity Measures in Primary Prevention Programmes for Eating Disorders in Schools

The aim of this paper was to examine the extent to which controlled trials of face-to-face school-based primary prevention for eating disorders report their strategies for maintaining fidelity. A systematic review located 38 articles eligible for inclusion. These studies were assessed using 18 criteria for reporting fidelity maintenance strategies based on those recommended by the Treatment Fidelity Workgroup of the National Institute of Health Behavior Change Consortium. Fidelity reporting was generally poor. The studies reported between 22% and 56% of fidelity criteria. Detailed reporting of curriculum-as-usual control conditions was generally lacking, as were methods to ensure high-quality training and mechanisms to assess provider adherence to intervention protocol. Poor fidelity assessment and reporting is a problem in school-based primary prevention programmes for eating disorders. Recommendations for improving fidelity maintenance and reporting practices are provided.

Abstract 1361: Treatment fidelity evaluation of the cancer prevention intervention in African American, Latina, and Arab women.

Abstract Background: Cancer education is important, especially for those facing health disparities. CHWs are increasingly utilized to educate and monitor cancer screening and surveillance for underserved populations. Here, we evaluate a randomized controlled treatment, utilizing Community Health Workers (CHW) to deliver breast and cervical cancer education intervention to African American (herein called Black), Latina, and Arab women in Detroit and Dearborn, Michigan. The main objectives of the study are to: 1) examine fidelity and consistency of treatment delivery and 2) assess qualitative elements of the intervention. Methods: CHWs and their clients recruited additional family members into the intervention. After randomization at total of 514 women were included in the study (305 intervention, 209 control). We surveyed only the intervention group because they received a behavioral intervention whereby the control group was given only reading materials. Family clusters received two home visits. The first session related to breast cancer education and the second session to cervical cancer. After each session, all women completed an evaluation of the fidelity and integrity of the education using the NIH Treatment Fidelity Workgroup Tool (Borelli, 2005). The CHW Survey included questions regarding their training, fidelity in treatment delivery and qualitative items. Results: The intervention group (n = 305) was made up of 48% Black, 11% Latina, and 41% Arab women. Almost all (> 90%) women agreed that they received the treatment in the way that it was intended. All 16 CHWs responded affirmatively as well. The greater majority (>90%), felt that they delivered the treatment the way it was intended and delivered it in the same way other Kin KeeperSM CHWs did. Qualitatively, 91% of CHWs felt they made an impact on participants’ cancer literacy and were satisfied with the project. In addition, CHWs indicated that “serving and educating their community” was personally important. Conclusions: First, both participants and CHWs indicated that the program was mutually rewarding, indicating that there was “cross fertilization and cross benefit” of working with each other. These benefits served to endorse and confirm that CHWs are a very important mechanism in increasing health literacy in the community and referring underserved individuals to health providers. Second, with strong treatment fidelity, these findings further validate the Kin KeeperSM program and its instruments (Williams, et al., 2011) and thus has the potential to be replicated for of number of diseases and in a variety of venues especially for those facing health disparities. Citation Format: Sabrina Ford, Cristian Meghea, Tamika Estes, Hiam Hamade, Mulisa Lockett, Karen Patricia Williams. Treatment fidelity evaluation of the cancer prevention intervention in African American, Latina, and Arab women. [abstract]. In: Proceedings of the 104th Annual Meeting of the American Association for Cancer Research; 2013 Apr 6-10; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2013;73(8 Suppl):Abstract nr 1361. doi:10.1158/1538-7445.AM2013-1361

Intervention fidelity: Ensuring application to practice for youth and families

Limited information on intervention fidelity is available in published studies with youth and families. The components of intervention fidelity, the complexity of measurement in these studies, and strategies for measuring intervention fidelity are described. Strategies for ensuring intervention fidelity according to the Treatment Fidelity Workgroup of the Behavior Change Consortium in the areas of study design, provider training, treatment delivery, treatment receipt, and treatment enactment provide guidance for evaluating or developing intervention fidelity plans. Ensuring the quality of intervention fidelity in evidence-based reviews or when developing new interventions is essential for translating findings into practice.

Tailoring a Treatment Fidelity Framework for an Intensive Care Unit Clinical Trial

Treatment fidelity (TF) refers to methodological strategies and practices used to monitor and enhance the reliability and validity of behavioral interventions. Treatment fidelity monitoring enhances internal and external validity and is needed for study replication and generalizability. The aim of this study was to describe the implementation, monitoring, and impact of TF in an intensive-care-unit-based clinical trial testing music for anxiety self-management with mechanically ventilated patients. Development of the criteria was based on the Five-Component Treatment Fidelity Framework from the Treatment Fidelity Workgroup. Descriptive statistics were used to evaluate adherence rates to the key TF criteria and the reasons criteria were unmet. Descriptive and nonparametric statistics were used to evaluate the impact of TF on participants' use of the assigned intervention. The Treatment Fidelity Framework was adapted easily to fit the study interventions. After the initial implementation phase of monitoring, adherence to key criteria was maintained at the targeted level of 80%. The majority of barriers to adherence affected the research nurses' opportunity to interact with the participant and encourage use of the intervention. There was a trend toward increased use of equipment associated with the assigned condition after the initiation of TF; however, this difference was not statistically significant. Treatment fidelity monitoring is an iterative process that requires ongoing vigilance. Identification of barriers and the implementation of methods to enhance protocol adherence are needed to enhance the reliability, validity, and generalizability of clinical trials in the dynamic and challenging research environment of the intensive care unit.

Ensuring treatment fidelity in a multi‐site behavioral intervention study: implementing NIH behavior change consortium recommendations in the SMART trial

The Stories and Music for Adolescent/Young Adult Resilience during Transplant (SMART) study (R01NR008583; U10CA098543; U10CA095861) is an ongoing multi-site Children's Oncology Group randomized clinical trial testing the efficacy of a therapeutic music video intervention for adolescents/young adults (11-24 years of age) with cancer undergoing stem cell transplant. Treatment fidelity strategies from our trial are consistent with the National Institutes of Health (NIH) Behavior Change Consortium Treatment Fidelity Workgroup (BCC) recommendations and provide a successful working model for treatment fidelity implementation in a large, multi-site behavioral intervention study. In this paper, we summarize 20 specific treatment fidelity strategies used in the SMART trial and how these strategies correspond with NIH BCC recommendations in five specific areas: (1) study design, (2) training providers, (3) delivery of treatment, (4) receipt of treatment, and (5) enactment of treatment skills. Increased use and reporting of treatment fidelity procedures is essential in advancing the reliability and validity of behavioral intervention research. The SMART trial provides a strong model for the application of fidelity strategies to improve scientific findings and addresses the absence of published literature, illustrating the application of BCC recommendations in behavioral intervention studies.

Intervention Fidelity

The Treatment Fidelity Workgroup (TFW) established by the National Institutes of Health provides a 5-point structure for intervention fidelity: dosing, interventionists' consistency, intervention delivery, receipt, and enactment of the intervention. Using our reflexology trial, we apply the first 3 points. Study objectives were to (1) evaluate key dosage dimensions associated with complementary and alternative medicine (CAM) research, (2) evaluate approaches to interventionists' consistency of delivery of CAM protocols, and (3) evaluate and discuss data that reflect CAM intervention fidelity. Women with late-stage breast cancer (N = 318) were randomly assigned to either 4 weeks of reflexology, placebo, or standard care. Dosing consists of 3 dimensions: frequency (4 sessions), duration (30 minutes), and interval between sessions (5-9 days). Interventionist consistency revealed more than a 90% accuracy rate in following the protocol; 84% and 89% completion rate of the 4 sessions in the reflexology and placebo groups, respectively; and no differences in attrition after randomization between reflexology and placebo groups (17% and 15%, respectively). Intervention delivery, examined through debriefing data, indicated a significantly higher rate of correct guesses on group assignment in the reflexology group as compared with the placebo (82% vs 46%, P = .0002). This study points out the relevance of dosing, interventionists' consistency, and delivery data within a CAM clinical trial, as well as the challenges of blinding. Monitoring intervention fidelity by using the key areas identified by the TFW ensures that findings from a clinical trial are meaningful and have the potential to be translated to clinical practice.

Examples of implementation and evaluation of treatment fidelity in the BCC studies: Where we are and where we need to go

Treatment fidelity plays an important role in the research team's ability to ensure that a treatment has been implemented as intended and that the treatment has been accurately tested. Developing, implementing, and evaluating a treatment fidelity plan can be challenging. The treatment fidelity workgroup within the Behavior Change Consortium (BCC) developed guidelines to comprehensively evaluate treatment fidelity in behavior change research. The guidelines include evaluation of treatment fidelity with regard to study design, training of interventionists, delivery and receipt of the intervention, and enactment of the intervention in real-life settings. This article describes these guidelines and provides examples from four BCC studies as to how these recommended guidelines for fidelity were considered. Future work needs to focus not only on implementing treatment fidelity plans but also on quantifying the evaluations performed, developing specific criteria for interpretation of the findings, and establishing best practices of treatment fidelity.

Enhancing Treatment Fidelity in Health Behavior Change Studies: Best Practices and Recommendations From the NIH Behavior Change Consortium.

Treatment fidelity refers to the methodological strategies used to monitor and enhance the reliability and validity of behavioral interventions. This article describes a multisite effort by the Treatment Fidelity Workgroup of the National Institutes of Health Behavior Change Consortium (BCC) to identify treatment fidelity concepts and strategies in health behavior intervention research. The work group reviewed treatment fidelity practices in the research literature, identified techniques used within the BCC, and developed recommendations for incorporating these practices more consistently. The recommendations cover study design, provider training, treatment delivery, treatment receipt, and enactment of treatment skills. Funding agencies, reviewers, and journal editors are encouraged to make treatment fidelity a standard part of the conduct and evaluation of health behavior intervention research.