Electronic health records based reinforcement learning for treatment optimizing

Abstract

Electronic Health Records (EHRs) have become one of the main sources of evidence to evaluate clinical actions, improve medical quality, predict disease-risk, and optimize treatment effects. However, EHRs present several modeling challenges, including heterogeneous data types and dynamic characteristics. Reinforcement learning provides an efficient way for sequential decision-making. Powered by model-based reinforcement learning approach, we propose an EHRs-based reinforcement learning algorithm to optimize sequential treatment strategies for diseases, such as sepsis, diabetes, and their complications. We conduct our experiments with this algorithm to optimize physicians’ historical treatment strategies and achieve better glucose control for diabetic ketoacidosis (DKA) patients, which is one serious complication of diabetes. The research includes the modeling process and reinforcement learning process. During the EHRs modeling process, besides considering the necessary physiological variables, we also consider the major disease factors to enhance the interpretability of the model. In the reinforcement learning process, a deep Q network is employed to explore the optimal insulin dose for patients. Moreover, inspired by the real medical scenes, we extend the algorithm to cooperative learning environment. We use the joint policy of the two agents to simulate doctor consultations, and achieve better treatment performances in terms of policy and blood glucose control than single agent and clinicians.