Using Survival Analysis to Understand Patterns of Sustainment within a System-Driven Implementation of Multiple Evidence-Based Practices for Children's Mental Health Services.

Lauren Brookman-Frazee,Scott Roesch,Debbie Innes-Gomberg,David Sommerfeld,Lillian Bando,Gregory A Aarons,Nicole Stadnick,Anna S Lau,Chanel Zhan

doi:10.3389/fpubh.2018.00054

Abstract

Evidence-based practice (EBP) implementation requires substantial resources in workforce training; yet, failure to achieve long-term sustainment can result in poor return on investment. There is limited research on EBP sustainment in mental health services long after implementation. This study examined therapists’ continued vs. discontinued practice delivery based on administrative claims for reimbursement for six EBPs [Cognitive Behavioral Interventions for Trauma in Schools (CBITS), Child–Parent Psychotherapy, Managing and Adapting Practices (MAP), Seeking Safety (SS), Trauma-Focused Cognitive Behavior Therapy (TF-CBT), and Positive Parenting Program] adopted in a system-driven implementation effort in public mental health services for children. Our goal was to identify agency and therapist factors associated with a sustained EBP delivery. Survival analysis (i.e., Kaplan–Meier survival functions, log-rank tests, and Cox regressions) was used to analyze 19 fiscal quarters (i.e., approximately 57 months) of claims data from the Prevention and Early Intervention Transformation within the Los Angeles County Department of Mental Health. These data comprised 2,322,389 claims made by 6,873 therapists across 88 agencies. Survival time was represented by the time elapsed from therapists’ first to final claims for each practice and for any of the six EBPs. Results indicate that therapists continued to deliver at least one EBP for a mean survival time of 21.73 months (median = 18.70). When compared to a survival curve of the five other EBPs, CBITS, SS, and TP demonstrated a higher risk of delivery discontinuation, whereas MAP and TF-CBT demonstrated a lower risk of delivery discontinuation. A multivariate Cox regression model revealed that agency (centralization and service setting) and therapist (demographics, discipline, and case-mix characteristics) characteristics were significantly associated with risk of delivery discontinuation for any of the six EBPs. This study illustrates a novel application of survival analysis to administrative claims data in system-driven implementation of multiple EBPs. Findings reveal variability in the long-term continuation of therapist-level delivery of EBPs and highlight the importance of both agency and workforce characteristics in the sustained delivery of EBPs. Findings direct the field to potential targets of sustainment interventions (e.g., strategic assignment of therapists to EBP training and strategic selection of EBPs by agencies).

Highlights

In response to a national call for an increased delivery of evidence-based practices (EBPs) in routine-care settings to improve the quality of care [1,2,3,4], mental health systems have increasingly mandated or incentivized the implementation of EBPs
The purpose of this study was to use EBP-specific claims data to [1] characterize therapists’ continued vs. discontinued delivery of six EBPs [Cognitive Behavioral Interventions for Trauma in Schools (CBITS), Child–Parent Psychotherapy (CPP), Managing and Adapting Practice (MAP), Seeking Safety (SS), Trauma-Focused Cognitive Behavioral Therapy (TF-CBT), and Positive Parenting Program (Triple P)] and [2] identify factors associated with a sustained EBP delivery
2,387 (34.7%) therapists claimed for one practice; 29.7% claimed for two EBPs, 21.1% claimed for three EBPs, 12.1% claimed for four EBPs, and 2.4% claimed for five or six EBPs

Summary

Introduction

In response to a national call for an increased delivery of evidence-based practices (EBPs) in routine-care settings to improve the quality of care [1,2,3,4], mental health systems have increasingly mandated or incentivized the implementation of EBPs. Evidence-based practice implementation requires substantial investments to support the mental health workforce [9, 10]. Such costs are incurred through clinicians’ time spent on attending trainings (i.e., lost revenue for the agency) and costs to facilitate the supervision and fidelity monitoring of newly trained staff, including payments to external consultants or trainers [11]. Workforce costs may be especially high given the complexity of multicomponent psychosocial EBPs, many of which have intensive certification requirements [12]. An even greater investment is needed when multiple complex interventions are rolled out in tandem in a given service system [14]

Objectives

Methods

Results

Conclusion