Application of machine learning approaches to administrative claims data to predict clinical outcomes in medical and surgical patient populations.

Emily J Mackay,Michael D Stubna,Peter W Groeneveld,Nimesh D Desai,Corey Chivers,William J Hanson,Michael E Draugelis,Thippa Reddy Gadekallu

doi:10.1371/journal.pone.0252585

Abstract

ObjectiveThis study aimed to develop and validate a claims-based, machine learning algorithm to predict clinical outcomes across both medical and surgical patient populations.MethodsThis retrospective, observational cohort study, used a random 5% sample of 770,777 fee-for-service Medicare beneficiaries with an inpatient hospitalization between 2009–2011. The machine learning algorithms tested included: support vector machine, random forest, multilayer perceptron, extreme gradient boosted tree, and logistic regression. The extreme gradient boosted tree algorithm outperformed the alternatives and was the machine learning method used for the final risk model. Primary outcome was 30-day mortality. Secondary outcomes were: rehospitalization, and any of 23 adverse clinical events occurring within 30 days of the index admission date.ResultsThe machine learning algorithm performance was evaluated by both the area under the receiver operating curve (AUROC) and Brier Score. The risk model demonstrated high performance for prediction of: 30-day mortality (AUROC = 0.88; Brier Score = 0.06), and 17 of the 23 adverse events (AUROC range: 0.80–0.86; Brier Score range: 0.01–0.05). The risk model demonstrated moderate performance for prediction of: rehospitalization within 30 days (AUROC = 0.73; Brier Score: = 0.07) and six of the 23 adverse events (AUROC range: 0.74–0.79; Brier Score range: 0.01–0.02). The machine learning risk model performed comparably on a second, independent validation dataset, confirming that the risk model was not overfit.Conclusions and relevanceWe have developed and validated a robust, claims-based, machine learning risk model that is applicable to both medical and surgical patient populations and demonstrates comparable predictive accuracy to existing risk models.

Highlights

Estimating risk is critical for decision making in both surgical and medical patient populations [1]
We have developed and validated a robust, claims-based, machine learning risk model that is applicable to both medical and surgical patient populations and demonstrates comparable predictive accuracy to existing risk models
Existing risk models are limited by an inability to rapidly obtain accurate information regarding patient risk [2,3,4], only apply to certain subsets of patient populations [2,3,4,5,6,7,8], and become outdated quickly because these models are not build to be continuously updated with new data [2,3,4,5,6,7,8]

Summary

Introduction

Estimating risk is critical for decision making in both surgical and medical patient populations [1]. While logistic regression risk models exist for both medical [5,6,7,8] and surgical populations [2,3,4], these risk models require time-consuming, manual data entry [2,3,4], only apply to limited subsets of patients Robust ML algorithms for prognostication have been developed primarily using electronic medical record (EMR) data [16,17,18] Such models’ reliance on EMR data, which is often proprietary and unique to a particular health system, makes implementing these prognostic tools across multiple health systems costly and challenging

Methods

Results

Discussion

Conclusion