Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review.

Siqi Liu,Mengling Feng,Xingzhi Sun,Leo Anthony Celi,Kay Choong See,Kee Yuan Ngiam

doi:10.2196/18477

Abstract

BackgroundDecision support systems based on reinforcement learning (RL) have been implemented to facilitate the delivery of personalized care. This paper aimed to provide a comprehensive review of RL applications in the critical care setting.ObjectiveThis review aimed to survey the literature on RL applications for clinical decision support in critical care and to provide insight into the challenges of applying various RL models.MethodsWe performed an extensive search of the following databases: PubMed, Google Scholar, Institute of Electrical and Electronics Engineers (IEEE), ScienceDirect, Web of Science, Medical Literature Analysis and Retrieval System Online (MEDLINE), and Excerpta Medica Database (EMBASE). Studies published over the past 10 years (2010-2019) that have applied RL for critical care were included.ResultsWe included 21 papers and found that RL has been used to optimize the choice of medications, drug dosing, and timing of interventions and to target personalized laboratory values. We further compared and contrasted the design of the RL models and the evaluation metrics for each application.ConclusionsRL has great potential for enhancing decision making in critical care. Challenges regarding RL system design, evaluation metrics, and model choice exist. More importantly, further work is required to validate RL in authentic clinical environments.

Highlights

In the health care domain, clinical processes are dynamic because of the high prevalence of complex diseases and dynamic changes in the clinical conditions of patients
In an intensive care unit (ICU), critically ill patients may benefit from deviation from established treatment protocols and from personalizing patient care using means not based on rules [5,6]
Only 9% of treatment recommendations in the ICU are based on randomized controlled trial reinforcement learning (RL) (RCT) [7], and the vast majority of RCTs in critical care have negative findings [8]

Summary

Introduction

Background In the health care domain, clinical processes are dynamic because of the high prevalence of complex diseases and dynamic changes in the clinical conditions of patients. Existing treatment recommendation systems are mainly implemented using rule-based protocols defined by physicians based on evidence-based clinical guidelines or best practices [1,2,3]. These protocols and guidelines may not consider https://www.jmir.org/2020/7/e18477 XSLFO RenderX. To aid clinical decisions in ICUs, we need other methods, including the use of large observational data sets. Given the dynamic nature of critically ill patients, one machine learning method called reinforcement learning (RL) is suitable for ICU settings. This paper aimed to provide a comprehensive review of RL applications in the critical care setting

Methods

Results

Discussion

Conclusion