Predicting patient outcomes in psychiatric hospitals with routine data: a machine learning approach

J Wolff,M Franz,D Jung,A Klimke,H Binder,K Domschke,C Normann,K Kaier,A Gary

doi:10.1186/s12911-020-1042-2

Abstract

BackgroundA common problem in machine learning applications is availability of data at the point of decision making. The aim of the present study was to use routine data readily available at admission to predict aspects relevant to the organization of psychiatric hospital care. A further aim was to compare the results of a machine learning approach with those obtained through a traditional method and those obtained through a naive baseline classifier.MethodsThe study included consecutively discharged patients between 1st of January 2017 and 31st of December 2018 from nine psychiatric hospitals in Hesse, Germany. We compared the predictive performance achieved by stochastic gradient boosting (GBM) with multiple logistic regression and a naive baseline classifier. We tested the performance of our final models on unseen patients from another calendar year and from different hospitals.ResultsThe study included 45,388 inpatient episodes. The models’ performance, as measured by the area under the Receiver Operating Characteristic curve, varied strongly between the predicted outcomes, with relatively high performance in the prediction of coercive treatment (area under the curve: 0.83) and 1:1 observations (0.80) and relatively poor performance in the prediction of short length of stay (0.69) and non-response to treatment (0.65). The GBM performed slightly better than logistic regression. Both approaches were substantially better than a naive prediction based solely on basic diagnostic grouping.ConclusionThe present study has shown that administrative routine data can be used to predict aspects relevant to the organisation of psychiatric hospital care. Future research should investigate the predictive performance that is necessary to provide effective assistance in clinical practice for the benefit of both staff and patients.

Highlights

The individual needs of patients are central to decision making in hospital care [1]
A further aim was to compare the results of a machine learning approach with those obtained through a traditional method and those obtained through a naive baseline classifier
Three different modelling approaches were compared: The chosen machine learning approach was a stochastic gradient boosting algorithm implemented in the CARET package in R based on the gradient boosting machine (GBM) by Friedman [31,32,33], The traditional method was logistic regression with the full set of feature variables used in the machine learning approach

Summary

Introduction

The individual needs of patients are central to decision making in hospital care [1]. The identification of common patterns of needs and the prediction of relevant aspects of patient care were found to be more complex in hospital psychiatry than in other medical disciplines [3,4,5]. Machine learning is a potent approach to identify and quantify multidimensional patterns in patient and hospital service data [11]. It has gained increasing attention in health care by achieving impressive results, for instance, in early prediction and diagnosis of breast cancer [12], acute kidney injury [13], skin cancer [14], prostate. A further aim was to compare the results of a machine learning approach with those obtained through a traditional method and those obtained through a naive baseline classifier

Objectives

Methods

Results

Discussion

Conclusion