Measure Of Treatment Fidelity Research Articles

Treatment Fidelity Over the Last Decade in Psychosocial Clinical Trials Outcome Studies: A Systematic Review

Background: Treatment fidelity tools are frequently used in clinical trials, promoting treatment consistency and therefore validity of trial findings. However, treatment fidelity procedures have not been included within international clinical trial guidelines such as the Consolidated Standards of Reporting Trials (CONSORT 2010). Aim: This study systematically reviews psychological clinical trials that incorporate Treatment Fidelity procedures and appraises their implementation using the Implementation of Treatment Integrity Procedures Scale (ITIPS). Method: Using the PRISMA Checklist as a guide for systematic review, a comprehensive search of the Medline, PsychINFO, Ovid, Cochrane Library, Scopus, PUBMED databases for the period 2004 to 2014 resulted in retrieval of 3186 potential articles. Thirty-two studies meeting inclusion criteria were analysed against the ITIPS. Results: Sixteen studies were assessed as ‘approaching adequacy’ in implementing Treatment Fidelity procedures as measured by the ITIPS scale, 8 studies were assessed as ‘adequate’ whilst a further 8 studies were deemed ‘inadequate’ against this assessment. Treatment Fidelity tools generally increased the intensity of the intervention or program within which they were used, resulting in improved levels of Treatment Fidelity. Conclusion: Current evidence supporting the inclusion of Treatment Fidelity tools is limited since there have been relatively few published studies examining the effectiveness of Treatment Fidelity tools. Further research into the efficacy, feasibility and measurement of Treatment Fidelity in implementing treatments is recommended, in tandem with additions to the CONSORT Guidelines to better support the inclusion of Treatment Fidelity procedures within clinical trials.

Implementation of Best Practices Regarding Treatment Fidelity in the Family Colorectal Cancer Awareness and Risk Education Randomized Controlled Trial

Treatment fidelity is associated with improvement in research outcomes and increased confidence in significant findings. However, few studies report on recommended areas of treatment fidelity (i.e., study design, training, treatment delivery, treatment receipt, and treatment enactment), leaving a dearth of information about implementation components that contributed to a study’s success. Without such information, it is difficult for researchers to correctly assess previous findings and for practitioners to correctly implement findings into practice. Thus, it is crucial that studies assess both treatment fidelity and applicability of treatment fidelity findings. We report measures of treatment fidelity in a randomized controlled trial of an intervention promoting colonoscopy in at-risk relatives of colorectal cancer (CRC) patients. We describe assessments related to both treatment delivery and treatment receipt. We conducted separate ANCOVAs to model the change in each of the treatment receipt variables, comparing the two intervention arms. Compared with the control group, the intervention group had significantly greater improvements in CRC knowledge (f = 17.46, p < .0001), perceptions about susceptibility (f = 15.08, p = .0002), response efficacy (f = 7.46, p = .0076), self-efficacy (f = 8.16, p = .0053), and reduced decisional uncertainty (f = 19.59, p < .0001) from baseline to 1-month follow-up. Overall, our study adhered to most of the best-practice guidelines for behavioral intervention fidelity. This demonstrates that our intervention was delivered as intended and positively affected the cognitive processes that are purported to be predictive of adherent behavioral outcomes.

Development and evaluation of a treatment fidelity instrument for family-based treatment of adolescent anorexia nervosa.

This study provides data on the psychometric properties of a newly developed measure of treatment fidelity in Family-Based Treatment (FBT) for adolescent anorexia nervosa (AN). The Family Therapy Fidelity and Adherence Check (FBT-FACT) was created to evaluate therapist adherence and competency on the core interventions in FBT. Participants were 45 adolescents and their families sampled from three randomized clinical trials evaluating treatment for AN. Trained fidelity raters evaluated 19 therapists across 90 early session recordings using the FBT-FACT. They also rated an additional 15 session 1 recordings of an alternate form of family therapy-Systemic Family Therapy for the purpose of evaluating discriminant validity of the FBT-FACT. The process of development and the psychometric properties of the FBT-FACT are presented. Overall fidelity ratings for each session demonstrated moderate to strong inter-rater agreement. Internal consistency of the measure was strong for sessions 1 and 2 and poor for session 3. Principal components analysis suggests sessions 1 and 2 are distinct interventions. The FBT-FACT demonstrates good reliability and validity as a measure of treatment fidelity in the early phase of FBT.

Inter-rater reliability of treatment fidelity and therapeutic alliance measures for psychological therapies for anxiety in young people with autism spectrum disorders

Objectives: This article presents work undertaken to establish inter-rater reliability for a measure of treatment fidelity and a measure of therapeutic alliance for therapies for anxiety for young people with autism spectrum disorders. The discussion and decision-making processes behind achieving consensus of raters are rarely published. Margolin et al. (1998) have highlighted this issue and called for researchers to communicate the details of their observational and rating procedures. This article is a response to their call for greater transparency so that these methods are readily accessible for comparison with other studies.Methods: Participants were young people with autism spectrum disorders receiving treatment for anxiety, clinical staff treating these young people and the independent raters assessing the treatment sessions. We report: (i) the processes involved in establishing inter-rater reliability for two instruments, (ii) the results obtained with a sample of young people with autism spectrum disorders using these instruments.Results and conclusions: Results demonstrate that it was possible to attain satisfactory inter-rater reliability with each of these two instruments with a client group with autism spectrum disorders, even though the instruments were originally designed for typically-developing populations.

Establishing Empirically-Informed Practice With Caregivers: Findings From the CARES Program

There is increasing interest in delivering efficacious interventions to caregivers. This study reports on the processes and findings from a modified approach of the Wraparound System of Care. The practice model is implemented over 4 phases and is based on 3 theoretical constructs and 10 principles. The evaluation utilized treatment fidelity measures and employed a quasi-experimental pretest/posttest design to examine caregiver outcomes (N = 120). Findings suggest significant improvement among caregivers’ self-reported health, life satisfaction, well-being, and quality of life, as well as a reduction in caregiver risk and burden. Delineation of the processes used in the model are presented.

Health coaching for low back pain: a systematic review of the literature.

To systematically review the evidence for health coaching for patients with low back pain and describe the diversity of health coach training and interventions. Electronic databases Medline, CINAHL, EMBASE, PsycINFO, AMED and the Cochrane Central Register of Controlled Trials were searched to 24 June 2013 using individually devised strategies. Randomised or quasi randomised controlled trials (RCTs) of health coaching for adults with low back pain of any duration were considered. The overall quality of the body of evidence was assessed using the GRADE approach. Treatment effects were presented as the difference in mean scores with 95% confidence intervals and standardised mean difference at follow-up between health coaching and control groups. Health coaching interventions were compared narratively by their theoretical principles and the training and assessment of heath coaches. Five publications describing three RCTs and one cluster RCT met the criteria for review. A rating of very low was assigned to the body of evidence using the GRADE approach. One RCT found significant improvements in lifting capacity and exercise compliance in favour of the health coaching group at both follow-up points with a large and moderate standardised mean difference. All included studies based health coaching interventions on the transtheoretical model of change however, the content of counselling programmes varied between studies and measures of treatment fidelity were inconclusive. Variability in health coaching interventions and a lack of assessment of treatment fidelity in addition to the very low rating of the overall body of evidence identified in the current review renders any estimates of the effect of health coaching on low back pain uncertain. Well-designed RCTs of patients with sub-acute low back pain are required that incorporate clearly described protocols for health coaching interventions and include standardised measures of treatment fidelity.

An intervention for sensory difficulties in children with autism: a randomized trial.

This study evaluated a manualized intervention for sensory difficulties for children with autism, ages 4–8 years, using a randomized trial design. Diagnosis of autism was confirmed using gold standard measures. Results show that the children in the treatment group (n = 17) who received 30 sessions of the occupational therapy intervention scored significantly higher (p = 0.003, d = 1.2) on Goal Attainment Scales (primary outcome), and also scored significantly better on measures of caregiver assistance in self-care (p = 0.008 d = 0.9) and socialization (p = 0.04, d = 0.7) than the Usual Care control group (n = 15). The study shows high rigor in its measurement of treatment fidelity and use of a manualized protocol, and provides support for the use of this intervention for children with autism. Findings are discussed in terms of their implications for practice and future research.

A Comparison of Assertive Community Treatment Fidelity Measures and Patient-Centered Medical Home Standards

OBJECTIVE This study compared program measures of assertive community treatment (ACT) with standards of accreditation for the patient-centered medical home (PCMH) to determine whether there were similarities in the infrastructure of the two methods of service delivery and whether high-fidelity ACT teams would qualify for medical home accreditation. METHODS The authors compared National Committee for Quality Assurance PCMH standards with two ACT fidelity measures (the Dartmouth Assertive Community Treatment Scale and the Tool for Measurement of Assertive Community Treatment [TMACT]) and with national ACT program standards. RESULTS PCMH standards pertaining to enhanced access and continuity, management of care, and self-care support demonstrated strong overlap across ACT measures. Standards for identification and management of populations, care coordination and follow-up, and quality improvement demonstrated less overlap. The TMACT and the program standards had sufficient overlap to score in the range of a level 1 PCMH, but no ACT measure sufficiently detailed methods of population-based screening and tracking of referrals to satisfy "must-pass" elements of the standards. CONCLUSIONS ACT measures and medical home standards had significant overlap in innate infrastructure. ACT teams following the program standards or undergoing TMACT fidelity review could have the necessary infrastructure to serve as medical homes if they were properly equipped to supervise general medical care and administer activities to improve management of chronic diseases.

Treatment Fidelity: Its Importance and Reported Frequency in Aphasia Treatment Studies

Treatment fidelity is a measure of the reliability of the administration of an intervention in a treatment study. It is an important aspect of the validity of a research study, and it has implications for the ultimate implementation of evidence-supported interventions in typical clinical settings. Aphasia treatment studies published in the last 10 years in 3 journals were reviewed using coding techniques that were adapted from Gresham, Gansle, Noell, Cohen, and Rosenblum (1993). The following items were noted: identifying information, study design, description of both the dependent and independent variables, and whether a measure of treatment fidelity was explicitly included. Of the aphasia treatment studies published in the last 10 years, 14% explicitly reported treatment fidelity. Most studies reporting treatment fidelity used checking of videotaped sessions by independent raters. Of the reviewed studies, 45% provided sufficient treatment description to support replication. Treatment fidelity is widely acknowledged as being critical to research validity and is a foundation for the implementation of evidence-based practices, but only a small percentage of aphasia treatment studies published in the last 10 years explicitly reported treatment fidelity. Recommendations for research practices include increased attention to matters of treatment fidelity in the peer review process and explicit incorporation of 3 levels of treatment fidelity in treatment research.

Reliability and Validity of a Treatment Fidelity Assessment for Motivational Interviewing Targeting Sexual Risk Behaviors in People Living with HIV/AIDS

This study psychometrically evaluates the Motivational Interviewing Treatment Integrity Code (MITI) to assess fidelity to motivational interviewing to reduce sexual risk behaviors in people living with HIV/AIDS. 74 sessions from a pilot randomized controlled trial of motivational interviewing to reduce sexual risk behaviors in people living with HIV were coded with the MITI. Participants reported sexual behavior at baseline, 3-month, and 6-months. Regarding reliability, excellent inter-rater reliability was achieved for measures of behavior frequency across the 12 sessions coded by both coders; global scales demonstrated poor intraclass correlations, but adequate percent agreement. Regarding validity, principle components analyses indicated that a two-factor model accounted for an adequate amount of variance in the data. These factors were associated with decreases in sexual risk behaviors after treatment. The MITI is a reliable and valid measurement of treatment fidelity for motivational interviewing targeting sexual risk behaviors in people living with HIV/AIDS.

Reflections on treatment integrity in a dissemination and implementation framework.

This commentary reflects on issues related to linking the conceptual frame for treatment fidelity presented in the special issue articles to the most current thinking on treatment fidelity within the emerging field of implementation science. The commentary presents several aspects of treatment fidelity in implementation science that generally were not discussed or addressed in the special issue articles, such as the placement of treatment fidelity as only one among several critical implementation outcomes, the relationship of treatment fidelity to design elements such as external validity as well as internal validity, and the importance of considering implementation issues in the measurement of treatment fidelity even in the earlier stages of efficacy and effectiveness research. The excellent in-depth discussion of treatment fidelity in the special series is a promising start to linking this critical concept to the emerging field of implementation science.

Increasing Social Interaction Skills of Secondary School Students with Autism and/or Intellectual Disability: A Review of Interventions

We reviewed studies to identify strategies effective at increasing social interaction skills across a range of secondary school students with autism and/or intellectual disability who experienced limited peer interaction. We were particularly interested in identifying strategies that involved peers and were effective at increasing peer interaction beyond the instructional setting. We identified 13 intervention studies that we analyzed by (a) participant characteristics, settings, and outcomes measured; (b) effective instructional strategies, including role of peers and programming for generalization, related to generalized effects of social interaction interventions; (c) the relation of participant characteristics, type of intervention, and generalization effects; (d) social validation and treatment fidelity measures; and (e) methodological limitations. Findings allowed us to provide recommendations for future research and practice.

A Cross-Cultural Three-Step Process Model for Assessing Motivational Interviewing Treatment Fidelity in Thailand

The provision of culturally relevant yet evidence-based interventions has become crucial to global HIV prevention and treatment efforts. In Thailand, where treatment for HIV has become widely available, medication adherence and risk behaviors remain an issue for Thai youth living with HIV. Previous research on motivational interviewing (MI) has proven effective in promoting medication adherence and HIV risk reduction in the United States. However, to test the efficacy of MI in the Thai context a feasible method for monitoring treatment fidelity must be implemented. This article describes a collaborative three-step process model for implementing the MI Treatment Integrity (MITI) across cultures while identifying linguistic issues that the English-originated MITI was not designed to detect as part of a larger intervention for Thai youth living with HIV. Step 1 describes the training of the Thai MITI coder, Step 2 describes identifying cultural and linguistic issues unique to the Thai context, and Step 3 describes an MITI booster training and incorporation of the MITI feedback into supervision and team discussion. Throughout the process the research team collaborated to implement the MITI while creating additional ways to evaluate in-session processes that the MITI is not designed to detect. The feasibility of using the MITI as a measure of treatment fidelity for MI delivered in the Thai linguistic and cultural context is discussed.

Responsive Interaction Interventions for Children With or at Risk for Developmental Delays

The purpose of this article is to synthesize the available studies regarding responsive interaction intervention (RII) for children with or at risk for developmental delays with a focus on six dimensions: (a) the characteristics of participants, (b) the features of RII, (c) the measurement of treatment fidelity, (d) the overall effectiveness of RII as reflected by the percentage of studies reporting significant changes in adult or child outcomes, (e) the measurement of maintenance and generalization of RII effects, and (f) the social validity or level of acceptability. Through a search of articles from 1990 to 2010, the authors identified 26 studies (31 articles) employing group experimental or quasiexperimental designs incorporating these dimensions. Overall, the results of the reviewed studies indicated that implementation of RII resulted in significant positive changes in adults’ responsive behaviors and children’s emotional and social-communicative outcomes. Although the most frequently reported child outcomes were in the social-communication domain, the most consistently significant positive outcomes for parent and child outcomes were in the emotional domain. The authors identify several gaps in this literature and suggest areas where research is needed.

Treatment of Childhood Obesity: A Systematic Review

Childhood obesity trends have increased dramatically over the past three decade’s. The purpose of this quantitative systematic review is to provide an update of the evidence, illustrating the efficacy of childhood obesity treatment, considering whether treatment fidelity has been measured and/or reported and whether this related to the treatment effect size. Searches revealed 61 relevant articles published from January 2000 to 2009, including both randomized and non-randomized controlled trials of childhood and adolescent obesity treatment interventions. The review identified scant measurement and reporting of issues around treatment fidelity, an increase in the number of interventions outside of the US, (particularly across Europe) and an emergence of studies involving older children and adolescents in a range of settings. Interventions comprising a dietary, exercise, and behavioral component, supported by family involvement and delivered by trained interventionists in specialized or supervised settings, appeared to offer a potentially effective treatment for obesity. However, concern remains over study quality (particularly sample size), dropout rates and study design. Variations in outcome measures and intervention designs means generalizable conclusions cannot be easily be made. Of greater concern is the lack of consideration for treatment fidelity, which has implications for the transfer of knowledge and the reliability of interventions. Conclusions from the review include; the need for increased accuracy of reporting and objective measurement of treatment fidelity; the need for further investigation of potential cost-effective treatment options (including maintenance strategies to enhance sustainability of current interventions); and an increase in the number of longitudinal trials beyond 1 year in duration.

The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials

OBJECTIVES: To discuss methods of preservation of treatment fidelity in health behavior change trials conducted in public health contexts. METHODS: The treatment fidelity framework provided by the NIH's Behavioral Change Consortium (BCC) (1) includes five domains of treatment fidelity (Study Design, Training, Delivery, Receipt, and Enactment). A measure of treatment fidelity was previously developed and validated using these categories. RESULTS: Strategies for assessment, monitoring, and enhancing treatment fidelity within each of the five treatment fidelity domains are discussed. The previously created measure of treatment fidelity is updated to include additional items on selecting providers, additional confounders, theory testing, and multicultural considerations. CONCLUSIONS: Implementation of a treatment fidelity plan may require extra staff time and costs. However, the economic and scientific costs of lack of attention to treatment fidelity are far greater than the costs of treatment fidelity implementation. Maintaining high levels of treatment fidelity with flexible adaptation according to setting, provider, and patient is the goal for public health trials.

Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems.

This study examined the impact of treatment adherence and therapist competence on treatment outcome in a controlled trial of individual cognitive-behavioral therapy (CBT) and multidimensional family therapy (MDFT) for adolescent substance use and related behavior problems. Participants included 136 adolescents (62 CBT, 74 MDFT) assessed at intake, discharge, and 6-month follow-up. Observational ratings of adherence and competence were collected on early and later phases of treatment (192 CBT sessions, 245 MDFT sessions) by using a contextual measure of treatment fidelity. Adherence and competence effects were tested after controlling for therapeutic alliance. In CBT only, stronger adherence predicted greater declines in drug use (linear effect). In CBT and MDFT, (a) stronger adherence predicted greater reductions in externalizing behaviors (linear effect) and (b) intermediate levels of adherence predicted the largest declines in internalizing behaviors, with high and low adherence predicting smaller improvements (curvilinear effect). Therapist competence did not predict outcome and did not moderate adherence-outcome relations; however, competence findings are tentative due to relatively low interrater reliability for the competence ratings. Clinical and research implications for attending to both linear and curvilinear adherence effects in manualized treatments for behavior disorders are discussed.

Measuring School-wide Positive Behavior Support Implementation

School-wide positive behavior support (SWPBS) has been implemented in more than 4,000 schools as a means of addressing problem behavior in a systemic fashion. Preliminary outcomes (e.g., office discipline referrals, suspensions) indicate the effectiveness of SWPBS in decreasing school-wide behavior problems and creating a positive school climate. Although the results of a majority of the program evaluations yielded significant findings, there has been a lack of measurement of treatment fidelity, possibly due to the absence of expedient, effective assessment tools. This article describes the theoretical background and development, including a qualitative pilot study and psychometric properties, of the School-wide Benchmarks of Quality (BoQ; Kincaid, Childs, &amp; George, 2005), a tool intended to measure the implementation of SWPBS. Descriptive data on the instrument, including internal consistency, test—retest reliability, interrater reliability, and concurrent validity, were collected and analyzed. Results indicate that the BoQ for SWPBS is a reliable, valid, efficient, and useful instrument for measuring the fidelity of implementation of the primary or universal level of PBS application in individual schools. Future considerations for evaluating the psychometric properties of the BoQ include extending the data collection and analysis to many more schools across multiple states.

What is this thing called motivational interviewing?

What is this thing called motivational interviewing?

Intervention Research

Internal validity of a randomized clinical trial of a nursing intervention is dependent on intervention fidelity. Although several methods have been developed, evaluating audio or audiovisual tapes for prescribed and proscribed interventionist behaviors is considered the gold standard test of treatment fidelity. This approach requires development of a psychometrically sound instrument to meaningfully categorize and quantify interventionist behaviors. To outline critical steps necessary to develop a treatment fidelity instrument. A comprehensive literature review was conducted to determine procedures used by other researchers. The literature review produced five quantitative studies of treatment fidelity, all in the field of psychotherapy, and two replication studies. A synthesis of methodologies across studies combined with researchers' experiences resulted in identification of the steps necessary to develop a treatment fidelity measure. Seven sequential steps were identified as essential to the development of a valid and reliable measure of treatment fidelity. These steps include (a) identification of the essential elements of the experimental and control treatment modalities; (b) construction of scale items; (c) development of item scaling; (d) identification of the units for coding; (e) item testing and revision; (f) specification of rater qualifications and development of rater training program; and (g) development and completion of pilot testing to test psychometric properties. Development of the Possibilities Project Psychotherapy Coding Questionnaire is described as an illustration of the seven-step process. The results show the essential steps that are unique to the development of treatment fidelity measures and show the feasibility of using these steps to construct a psychometrically sound treatment-specific fidelity measure.