Evaluate Treatment Fidelity Research Articles

Effectiveness of full Pulpotomy compared with Root canal treatment in managing teeth with signs and symptOms indicative of irreversible pulpitis: a protocol for prospectiVE meta-analysis of individual participant data of linked randomised clinical trials (PROVE)

BackgroundFull pulpotomy has been proposed as an alternative to root canal treatment in teeth with signs and symptoms indicative of irreversible pulpitis (IRP), but the evidence is limited, relying on underpowered studies with a high risk of bias. The aim of this study is to conduct a prospective meta-analysis (PMA) of individual participant data of a series of individual randomised trials to provide robust evidence on the clinical and cost-effectiveness of pulpotomy compared with root canal treatment.MethodsIndividual participant data will be obtained from a series of randomised trials designed and conducted by a consortium of multi-national investigators with an interest in vital pulp treatment. These individualised trials will be conducted using a specified protocol, defined outcomes, and outcome measures. Ten parallel-group randomised trials currently being conducted in 10 countries will provide data from more than 500 participants. The primary outcome is a composite measure defined as (1) the absence of pain indicative of IRP, (2) the absence of signs and symptoms indicative of acute or chronic apical periodontitis, and (3) the absence of radiographic evidence of failure including radiolucency or resorption. Individual participant data will be obtained, assessed, and checked for quality by two independent reviewers prior to the PMA. Pooled estimates on treatment effects will be generated using a 2-stage meta-analysis approach. The first stage involves a standard regression analysis in each trial to produce aggregate data on treatment effect estimates followed by an inverse variance weighted meta-analysis to combine these aggregate data and produce summary statistics and forest plots. Cost-effectiveness analysis based on the composite outcome will be undertaken as a process evaluation to evaluate treatment fidelity and acceptability by patients and dentists.ResultsThe research question and trial protocol were developed and approved by investigators in all 10 sites. All sites use shared resources including study protocols, data collection forms, participant information leaflets, and consent forms in order to improve flow, consistency, and reproducibility. Each site obtained its own Institutional Review Board approval, and trials were registered in appropriate open access platforms. Patient recruitment has started in most sites, as of July 2023.DiscussionPMA offers a rigorous, flexible, and efficient methodology to answer this important research question and provide results with improved generalisability and external validity compared with traditional trials and retrospective meta-analyses. The results of this study will have implications for both the delivery of clinical practice and structured clinical guidelines’ development.Trial registrationPROSPERO CRD42023446809. Registered on 08 February 2023.

Using Adherence and Competence Measures Based on Practice Elements to Evaluate Treatment Fidelity for Two CBT Programs for Youth Anxiety

Measures designed to assess the quantity and quality of practices found across treatment programs for specific youth emotional or behavioral problems may be a good fit for evaluating treatment fidelity in effectiveness and implementation research. Treatment fidelity measures must demonstrate certain reliability and validity characteristics to realize this potential. This study examines the extent to which two observational measures, the Cognitive-Behavioral Treatment for Anxiety in Youth Adherence Scale (CBAY-A) and the CBAY Competence Scale (CBAY-C), can assess the quantity (the degree to which prescribed therapeutic techniques are delivered as intended) or quality (the competence with which prescribed techniques are delivered) of practices found in two distinct treatment programs for youth anxiety. Treatment sessions (N = 796) from 55 youth participants (M age = 9.89 years, SD = 1.71; 46% female; 55% White) with primary anxiety problems who participated in an effectiveness study were independently coded by raters who coded quantity, quality, and the youth–clinician alliance. Youth received one of three treatments: (a) standard (i.e., cognitive-behavioral therapy program), (b) modular (i.e., a cognitive-behavioral and parent-training program), and (c) usual clinical care. Interrater reliability for the CBAY-A items was good across the standard and modular conditions but mixed for the CBAY-C items. Across the standard and modular conditions, the CBAY-A Model subscale scores demonstrated evidence of construct validity, but the CBAY-C Model subscale scores showed mixed evidence. The results provide preliminary evidence that the CBAY-A can be used across different treatment programs but raise concerns about the generalizability of the CBAY-C.

Treatment Fidelity of the Family-Centered Function-Focused Care Intervention.

The purpose of the current study was to evaluate treatment fidelity (i.e., the extent to which an intervention is provided as intended) in the Family-Centered Function-Focused Care (Fam-FFC) intervention. This was a descriptive study using data collected during intervention activities throughout the course of the Fam-FFC study. Specific measures included Environment and Policy Assessments, Fam-FFC Knowledge Test, Goal Attainment Scale, Function-Focused Care Behavior Checklist, and completion of the FamPath Audit. Delivery was provided as intended. Staff demonstrated intervention skills with only one Fam-FFC research nurse needing retraining. Receipt was based on Fam-FFC Knowledge Test scores >80%, with the majority of participants reporting goal achievement as expected or higher than expected and slight improvement in environments and policies to better support Fam-FFC. Lastly, enactment was based on evidence that in 67% of observations staff provided at least one function-focused care intervention. Findings from this study will be used to adapt the intervention to reach all staff, increase ways to change environments and policies, consider ways to more comprehensively evaluate enactment of function-focused care during real-world interactions, and consider the characteristics of nursing staff and whether a relationship exists between staff characteristics and providing function-focused care. [Research in Gerontological Nursing, 16(4), 165-171.].

Development of an audit checklist to evaluate treatment fidelity of a complex rehabilitation intervention

Purpose The process of examining treatment fidelity is critical to the successful implementation of rehabilitation interventions. Videotaping is considered the ‘gold standard’ assessment; however, may be considered resource-intensive and intrusive for intervention providers and patients. An audit checklist is an alternative approach recommended in the literature. The purpose of this study was to develop a documentation audit checklist for assessing treatment fidelity during delivery of SENSe therapy, a complex rehabilitation intervention targeting upper limb somatosensory impairment post-stroke. Methods Checklist development comprised: content determination and design; checklist testing via audit of 38 therapy records from an existing data set; and exploration of rater agreement between two assessors, using a subset of 10 therapy records. Results The developed audit checklist comprised 29 components core to the delivery of SENSe therapy. Six SENSe therapy records were delivered with high fidelity (>80% adherence to core components), and 32 with moderate fidelity (51–79%). Rater agreement was 80% across the subset of 10 records. Conclusion Findings highlight the importance of using a theoretically-guided approach to checklist development, with the use of rater agreement to identify areas for refinement. A documentation audit checklist was developed that can be used to evaluate treatment fidelity of complex rehabilitation interventions. IMPLICATIONS FOR REHABILITATION Development of an audit checklist that evaluates clinician documentation of therapy delivery, is feasible as one strategy to measure and enhance the treatment fidelity of complex rehabilitation interventions. The process of audit checklist development should be structured and based on conceptual frameworks, to ensure it accurately measures quality of delivery and adherence to core intervention components. Audit checklists can be used to support clinicians delivering complex rehabilitation interventions.

Keeping the Faith While Keeping It Real: Practical, Empirical Approaches to Evaluating Treatment Fidelity

Practical approaches towards fidelity measurement are critical for large-scale uptake of evidence-based practices (EBPs). This descriptive review summarizes approaches to evaluation of EBP fidelity that are empirically tested and implemented in “real-world” settings. The systematic literature search covered a 30-year period (1988–2018) and included published papers describing treatment fidelity strategies for psychosocial interventions. Characterization of articles included “scientific-empirical,” “practical-empirical,” or “practical non-empirical.” Articles characterized as “practical-empirical” were the focus of the review. Twenty-five articles met inclusion criteria for “practical empirical.” These articles were coded for fidelity data collection methods, the source of fidelity information, who arbitrates or determines fidelity, if the approach is at the practice or program level, if the approach is for a specific EBP or is generic, and if fidelity is measured at the individual practitioner versus team level. The type of empirical evidence provided and the extensiveness of fidelity domains assessed was characterized for each study. Results indicated a wide variety of fidelity measurement approaches. Most commonly, outside experts assessed treatment fidelity. Half of the approaches examined fidelity at the practice level and half at the broader program level. A similar ratio examined fidelity at the individual practitioner level compared with the team level. Most approaches were specific to particular interventions. The vast majority focus on the adherence subdomain of fidelity. To date, there is no single predominant approach to fidelity assessment in real-world settings.

Pain-related fear in adolescents with chronic musculoskeletal pain: process evaluation of an interdisciplinary graded exposure program

BackgroundFor studying the effectiveness of treatment, it is important to check whether a new treatment is performed as originally described in the study-protocol.ObjectivesTo evaluate whether an interdisciplinary graded exposure program, for adolescents with chronic musculoskeletal pain reporting pain-related fear, was performed according to protocol, and whether it is feasible to implement the program in rehabilitation care.MethodsA process evaluation where quantitative and qualitative data on participant characteristics (adolescents, parents and therapists), attendance and participants’ opinion on the program were collected, by means of registration forms, questionnaires and group interviews. To evaluate treatment fidelity, audio and video recordings of program sessions were analyzed.ResultsThirty adolescents were offered the program, of which 23 started the program. Adolescents attended on average 90% of the sessions. At least one parent per adolescent participated in the program. Analysis of 20 randomly selected recordings of treatment sessions revealed that treatment fidelity was high, since 81% of essential treatment elements were offered to the adolescents. The program was considered client-centered by adolescents and family-centered by parents. Treatment teams wished to continue offering the program in their center.ConclusionThe interdisciplinary graded exposure program was performed largely according to protocol, and therapists, adolescents and their parents had a favorable opinion on the program. Implementation of the program in rehabilitation care is considered feasible.Trial registrationClinicaltrials.gov ID: NCT02181725 (7 February 2014).

Protocol for a pilot randomised controlled trial of an educational programme for adults on chronic haemodialysis with fatigue (Fatigue-HD)

IntroductionFatigue is a pervasive symptom of end-stage renal disease (ESRD) that is associated with low quality of life, disability and mortality, and has been identified as a top research priority...

Fidelity review: a scoping review of the methods used to evaluate treatment fidelity in behavioural change interventions

Objectives: To identify the definitions used for treatment fidelity in the behaviour change literature and to explore the extent to which the assessment of fidelity has been reported according to the five domains by Bellg et al.Methods: Three databases (Scopus, Medline Ovid and CINAHL) were searched. Results were limited to studies published between 2012 and 2015. Definitions/summaries of treatment fidelity used were recorded. Methods for assessing/monitoring treatment fidelity were extracted, summarised and categorised according to the five domains.Results: Sixty-five papers were included for analysis. A definition of treatment fidelity was provided by n = 34 studies; n = 9 defined fidelity according to Bellg et al. In the context of treatment fidelity, n = 9 (13.8%) reported on study design; n = 22 (33.8%) reported on an element of training of providers; n = 59 (90.7%) papers reported on delivery of treatment; n = 13 (20%) reported on receipt of treatment; and n = 10 (15.3%) reported on enactment of treatment skills.Conclusion: The definitions of treatment fidelity in the literature and the extent to which it has been reported were limited. Delivery of treatment was the most frequently reported component of treatment fidelity but other important aspects were poorly reported. The potential consequence of this is that translation of research interventions into clinical practice may not be optimised.

Reporting of treatment fidelity in behavioural paediatric obesity intervention trials: a systematic review.

Behavioural interventions for paediatric obesity are promising, but detailed information on treatment fidelity (i.e. design, training, delivery, receipt and enactment) is needed to optimize the implementation of more effective interventions. Little is known about current practices for reporting treatment fidelity in paediatric obesity studies. This systematic review, in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, describes the methods used to report treatment fidelity in randomized controlled trials. Treatment fidelity was double-coded using the National Institutes of Health Fidelity Framework checklist. Three hundred articles (N = 193 studies) were included. Mean inter-coder reliability across items was 0.83 (SD = 0.09). Reporting of treatment design elements within the field was high (e.g. 77% of studies reported designed length of treatment session), but reporting of other domains was low (e.g. only 7% of studies reported length of treatment sessions delivered). Few reported gold standard methods to evaluate treatment fidelity (e.g. coding treatment content delivered). General study quality was associated with reporting of treatment fidelity (p < 0.01) as was the number of articles published for a given study (p < 0.01). The frequency of reporting treatment fidelity components has not improved over time (p = 0.26). Specific recommendations are made to support paediatric obesity researchers in leading health behaviour disciplines towards more rigorous measurement and reporting of treatment fidelity.

Establishing Treatment Fidelity in Evidence-Based Parent Training Programs for Externalizing Disorders in Children and Adolescents

The current review evaluates the use of treatment fidelity strategies in evidence-based parent training programs for treating externalizing disorders. We used a broad framework for evaluating treatment fidelity developed by the National Institutes of Health Treatment Fidelity Workgroup that includes the aspects of treatment design, treatment delivery, training providers, and assessment of participant receipt of treatment and enactment of treatment skills. Sixty-five articles reporting outcome trials of evidence-based parent training programs met inclusion criteria and were coded for treatment fidelity strategies. The mean adherence to fidelity strategies was .73, which was higher than two previous review studies employing this framework in the health literature. Strategies related to treatment design showed the highest mean adherence (.83), whereas training of providers and enactment of treatment skills had the lowest (.58). In light of an increasing emphasis on effectiveness and dissemination trials, the broader treatment fidelity framework as applied in this review focuses needed attention on areas often overlooked in fidelity practices, such as training providers and generalization of treatment skills. We discuss the strengths and limitations of fidelity practices in parent training studies, implications of these findings, and areas for future research.

Exercise Intervention Research in Stroke: Optimizing Outcomes Through Treatment Fidelity

Background: Demonstrating the treatment fidelity of an intervention is a key methodological requirement of any trial testing the impact of the intervention. Purpose: The purpose of this report was to serve as a model for evaluating treatment fidelity in stroke exercise intervention studies and to provide evidence for treatment fidelity in the Exercise Training for Hemiparetic Stroke Intervention Development Study. Methods: Treatment fidelity was evaluated based on study design, training of interventionists, and delivery and receipt of the intervention. Results: There were some concerns about design fidelity as the control group and intervention group traveled to the study location together and received different exercise programs in the same facility. With regard to training of interventionists, observations were utilized to help maintain adherence to the protocol. There was strong support for the delivery and receipt of the intervention with participants exposed to the anticipated number of exercise sessions. As per study protocol, there was evidence that progress toward the proposed physical activity goal was made. Treatment fidelity findings are reported using cumulative data rather than at points throughout the study. Thus the findings may be conservative with regard adherence, for example, to time spent in exercise. Conclusion: In addition to assurance of true testing of the intervention in this study, treatment fidelity of this work provides a useful model for replication and critically important information to better understand the type, dose, and length of exposure to exercise interventions that is needed to optimize stroke recovery.

Staff Training in Assisted Living: Evaluating Treatment Fidelity

To investigate a protocol for identifying and evaluating treatment fidelity in Staff Training in Assisted-living Residences (STAR), a structured but flexible program to train direct care staff to improve care of residents with dementia. Multisite feasibility trial. Assisted living facilities (ALFs). Forty-four direct care staff and 36 leadership staff. STAR is a comprehensive, dementia-specific training program to teach direct care staff in ALFs to improve care and reduce affective and behavioral problems in residents with dementia. It is conducted on-site over 2 months through 2 half-day group workshops and four individualized sessions. Treatment fidelity was assessed following the National Institute of Health Behavior Change Consortium model utilizing observations and self-report of trainers, direct care staff, and leadership. Each key area of treatment fidelity was identified, measured, and yielded significant outcomes. For example, significant increases included direct care staff identifying activators, behaviors, and consequences (an essential component of training); understanding basics of dementia care; and applying STAR techniques. Results support that STAR is ready to be translated and disseminated into practice. Because ALFs will continue to provide care for individuals with dementia, the need for effective, practical, and sustainable staff training programs is clear. STAR offers one such option. It is hoped that this report will encourage others to conduct comprehensive evaluations of the treatment fidelity of their programs and thereby increase the availability of such programs to enhance care.

Development of a tool to assess fidelity to a psycho‐educational intervention

This paper is a description of a method to develop and conduct a customized psycho-educational intervention fidelity assessment as part of pilot work for an efficacy study. A tool designed to assess treatment fidelity to a psycho-education intervention for patients with end-stage renal disease and their surrogate decision makers, Sharing the Patient's Illness Representations to Increase Trust, is presented as an illustration. Despite the specificity and idiosyncrasy of individual interventions and the call to systematically evaluate treatment fidelity, how to accomplish this goal has not been clarified. Tools to adequately measure treatment fidelity are lacking. We developed the Sharing the Patient's Illness Representations to Increase Trust Treatment Fidelity Assessment tool by identifying elements that were idiosyncratic to the intervention and those that could be adapted from existing tools. The tool has four components: overall adherence to the intervention content elements; pacing of the intervention delivery; overall dyad responsiveness; and, overall quality index of intervention delivery. The study was undertaken between 2006 and 2008. Inter-rater reliability ranged from 0.80 to 0.87 for the four components. The tool showed utility in training and monitoring, such as detecting unplanned content elements delivered and the use of proscribed communication behaviours. Psycho-educational interventions are one of the most common types of nursing interventions worldwide. Use of fidelity assessment tools customized to the individual interventions may enhance systematic evaluation of training and monitoring treatment fidelity.

Treatment Fidelity in Nursing Home Research: The Res-Care Intervention Study

Treatment fidelity plays an important role in the research team's ability to assure that a treatment has been implemented as intended. This is particularly important in long-term care, given the challenges associated with engaging in research in these sites. The purpose of this article is to describe the treatment fidelity plan and treatment fidelity outcomes for the Res-Care Intervention Study, which included a two-tiered motivational intervention examining behaviors in both nursing assistants and residents. We considered treatment fidelity with regard to design, training, delivery, receipt, and enactment. There was some evidence of treatment fidelity across all five areas. Future work, however, should consider additional ways in which to evaluate treatment fidelity, including using technology as a way to expose nursing assistants to new information and the inclusion of a Goal Attainment Form for enactment of resident activities. In doing so, assurance of treatment fidelity related to restorative care interventions would be strengthened.

FIDELITY OF AN EDUCATIONAL PROCESS: ROLE OF THE INTERVENTIONISTS

Objective: to evaluate treatment fidelity strategies used in a randomized controlled trail study such as record and standardization of intervention, selection and training of interventionists, and implementation of criteria for adherence and monitoring. Methodology: strategies analysis through pre and post training evaluation, monitor reporting, day-after evaluation and participants comments. Results: the analysis shows the selection and training of able interventionists with sufficient knowledge of curriculum delivery, self-efficient in processes and trustworthy in results. The day-after evaluations and participant´s comment reflect a dynamic, assertive and adhered-to-curriculum performance of the interventionists. Conclusions: observation of the elements that define randomized controlled trail study adherence and delivery guarantee intervention integrity and therefore reduce bias possibility because of this source.

Assessing fidelity in individual and family therapy for adolescent substance abuse

This study introduces an observational measure of fidelity in evidence-based practices for adolescent substance abuse treatment. The Therapist Behavior Rating Scale—Competence (TBRS-C) measures adherence and competence in individual cognitive–behavioral therapy and multidimensional family therapy for adolescent substance abuse. The TBRS-C assesses fidelity to the core therapeutic goals of each approach and also contains global ratings of therapist competence. Study participants were 136 clinically referred adolescents and their families observed in 437 treatment sessions. The TBRS-C demonstrated strong interrater reliability for goal-specific ratings of treatment adherence, and modest reliability for goal-specific and global ratings of therapist competence, evidence of construct validity, and discriminant validity with an observational measure of therapeutic alliance. The utility of the TBRS-C for evaluating treatment fidelity in field settings is discussed.

Development of the Distraction Coaching Index

Children's distress during medical procedures can be decreased when professionals or parents provide distraction coaching, but the quality of distraction may contribute to the variation in the results. This article describes the development of the Distraction Coaching Index (DCI), a tool for measuring the quality and frequency of distraction coaching. Nominal group technique and consensus agreement were used for concept clarification and development of behavioral indicators and descriptors. Videotapes of untrained parents, trained parents, and expert professional coaches, who provided distraction to children (4–10 years old) undergoing peripheral intravenous catheter insertion, were used to assess interrater reliability and construct validity. DCI scores were significantly higher for the experts versus untrained parents and trained parents versus untrained parents. The DCI can be used by researchers to evaluate treatment fidelity of distraction coaching and by clinicians for teaching health professionals and parents how to perform distractions.

Project Liberty: New York's Crisis Counseling Program Created in the Aftermath of September 11, 2001

Project Liberty: New York's Crisis Counseling Program Created in the Aftermath of September 11, 2001

A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research.

A. Bellg, B. Borrelli, et al. (2004) previously developed a framework that consisted of strategies to enhance treatment fidelity of health behavior interventions. The present study used this framework to (a) develop a measure of treatment fidelity and (b) use the measure to evaluate treatment fidelity in articles published in 5 journals over 10 years. Three hundred forty-two articles met inclusion criteria; 22% reported strategies to maintain provider skills, 27% reported checking adherence to protocol, 35% reported using a treatment manual, 54% reported using none of these strategies, and 12% reported using all 3 strategies. The mean proportion adherence to treatment fidelity strategies was .55; 15.5% of articles achieved greater than or equal to .80. This tool may be useful for researchers, grant reviewers, and editors planning and evaluating trials.

Examples of implementation and evaluation of treatment fidelity in the BCC studies: Where we are and where we need to go

Treatment fidelity plays an important role in the research team's ability to ensure that a treatment has been implemented as intended and that the treatment has been accurately tested. Developing, implementing, and evaluating a treatment fidelity plan can be challenging. The treatment fidelity workgroup within the Behavior Change Consortium (BCC) developed guidelines to comprehensively evaluate treatment fidelity in behavior change research. The guidelines include evaluation of treatment fidelity with regard to study design, training of interventionists, delivery and receipt of the intervention, and enactment of the intervention in real-life settings. This article describes these guidelines and provides examples from four BCC studies as to how these recommended guidelines for fidelity were considered. Future work needs to focus not only on implementing treatment fidelity plans but also on quantifying the evaluations performed, developing specific criteria for interpretation of the findings, and establishing best practices of treatment fidelity.