A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol.

Rinad S Beidas,Sonja K Schoenwald,Shannon Dorsey,Judy A Shea,Jessica Fishman,Steven C Marcus,Emily M Becker-Haimes,Bryce D Mcleod,Danielle R Adams,Michael T French,Aaron Hogue,Johanna Catherine Maclean,David S Mandell,Adina Lieberman

doi:10.1186/s12888-016-1034-z

Abstract

BackgroundThis randomized trial will compare three methods of assessing fidelity to cognitive-behavioral therapy (CBT) for youth to identify the most accurate and cost-effective method. The three methods include self-report (i.e., therapist completes a self-report measure on the CBT interventions used in session while circumventing some of the typical barriers to self-report), chart-stimulated recall (i.e., therapist reports on the CBT interventions used in session via an interview with a trained rater, and with the chart to assist him/her) and behavioral rehearsal (i.e., therapist demonstrates the CBT interventions used in session via a role-play with a trained rater). Direct observation will be used as the gold-standard comparison for each of the three methods.Methods/designThis trial will recruit 135 therapists in approximately 12 community agencies in the City of Philadelphia. Therapists will be randomized to one of the three conditions. Each therapist will provide data from three unique sessions, for a total of 405 sessions. All sessions will be audio-recorded and coded using the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale. This will enable comparison of each measurement approach to direct observation of therapist session behavior to determine which most accurately assesses fidelity. Cost data associated with each method will be gathered. To gather stakeholder perspectives of each measurement method, we will use purposive sampling to recruit 12 therapists from each condition (total of 36 therapists) and 12 supervisors to participate in semi-structured qualitative interviews.DiscussionResults will provide needed information on how to accurately and cost-effectively measure therapist fidelity to CBT for youth, as well as important information about stakeholder perspectives with regard to each measurement method. Findings will inform fidelity measurement practices in future implementation studies as well as in clinical practice.Trial registrationNCT02820623, June 3rd, 2016.

Highlights

This randomized trial will compare three methods of assessing fidelity to cognitive-behavioral therapy (CBT) for youth to identify the most accurate and cost-effective method
Direct observation of therapist behavior, considered by some to be the gold standard for measuring fidelity to psychosocial treatments, requires extensive resources
We are most interested in the potential of the two innovative methods used in the medical literature that may represent a reasonable compromise between direct observation and self-report and provide resolution to the fidelity measurement quandary of the past two decades

Summary

Introduction

This randomized trial will compare three methods of assessing fidelity to cognitive-behavioral therapy (CBT) for youth to identify the most accurate and cost-effective method. Research to improve client outcomes in community mental health has been hindered by an inability to accurately and inexpensively measure fidelity [1]. Direct observation of therapist behavior, considered by some to be the gold standard for measuring fidelity to psychosocial treatments, requires extensive resources. When fidelity is measured in community settings, which occurs rarely, the most commonly used method is therapists’ self-report [10]. There is a critical need to identify and evaluate methods of fidelity measurement that are both accurate (i.e., measure what they intend) and cost-effective [7]. Innovative alternatives from other disciplines, medicine, may present advantages for the measurement of fidelity accurately and cost-effectively [10, 12,13,14,15,16]

Objectives

Methods

Conclusion