Stratified Care in Cognitive Behavioural Therapy: A Comparative Evaluation of Predictive Modelling Approaches for Individualized Treatment: La stratification des soins pour l'orientation vers une thérapie cognitivo-comportementale: une évaluation comparative des approches de modélisation prédictive pour un traitement individualisé.

Abstract

In response to high demand and prolonged wait times for cognitive behavioural therapy (CBT) in Ontario, Canada, we developed predictive models to stratify patients into high- or low-intensity treatment, aiming to optimize limited healthcare resources. Using client records (n = 953) from Ontario Shores Centre for Mental Health Sciences (January 2017-2021), we estimated four binary outcome models to assign patients into complex and standard cases based on the probability of reliable improvement in Patient Health Questionnaire-9 (PHQ-9) and Generalized Anxiety Disorder-7 (GAD-7) scores. We evaluated two choices of cut-offs for patient complexity assignment: models at an ROC (receiver operating characteristic)-derived cut-off and a 0.5 probability cut-off. Final model effectiveness was assessed by assigning treatment intensity (high-intensity or low-intensity CBT) based on the combined performance of both GAD-7 and PHQ-9 models. We compared the treatment assignment recommendations provided by the models to those assigned by clinical assessors. The predictive models demonstrated higher accuracy in selecting treatment modality compared to provider-assigned treatment selection. The ROC cut-off achieved the highest prediction accuracy of case complexity (0.749). The predictive models exhibited large sensitivity and specificity trade-offs (which influence the rates of patient assignment to high-intensity CBT) despite having similar accuracy statistics (ROC cut-off = 0.749, 0.5 cut-off = 0.690), emphasizing the impact of cut-off choices when implementing predictive models. Overall, our findings suggest that the predictive model has the potential to improve the allocation of CBT services by shifting incoming clients with milder symptoms of depression to low-intensity CBT, with those at highest risk of not improving beginning in high-intensity CBT. We have demonstrated that models can have significant sensitivity and specificity trade-offs depending on the chosen acceptable threshold for the model to make positive predictions of case complexity. Further research could assess the use of the predictive model in real-world clinical settings.