Actuarial prediction versus clinical prediction of exits from a national supported housing program.

Abstract

The accurate identification of persons at risk of exiting permanent supportive housing could help maximize client success and minimize attrition and premature exits from such housing. Thus, in the present study, we developed and tested multivariable prediction models of negative and positive exits from the U.S. Department of Housing and Urban Development-Veterans Affairs Supportive Housing (HUD-VASH) program using logistic regression and random forests. We compared the performance of these models with clinical predictions made by HUD-VASH program case managers. We selected a cohort of all 92,196 Veterans who entered HUD-VASH nationwide between October 1, 2014 and September 30, 2019, 70% of whom were randomly selected to serve as the development cohort and the remaining 30% of whom served as the validation cohort. Negative and positive exits were measured until September 30, 2019. A subset of 1,264 Veterans was used to compare performance of models with clinical judgment. Predictor variables included sociodemographic characteristics, health and behavioral health diagnoses, homeless/housing history, and VA service utilization history. Performance of models and clinical judgment were assessed using an array of metrics including area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, and positive predictive value. The logistic regression and random forest models had similar, modest performance in predicting negative and positive exits. These models were substantially more sensitive, yet far less specific in predicting exits than clinician ratings. Study findings highlight the challenges and tradeoffs in using actuarial models or case manager predictions to target interventions to Veterans at risk of exiting HUD-VASH. (PsycInfo Database Record (c) 2022 APA, all rights reserved).