Optimal scheduling in cloud healthcare system using Q-learning algorithm

Abstract

Cloud healthcare system (CHS) can provide the telemedicine services, which is helpful to cope with the difficulty of patients getting medical service in the traditional medical systems. However, resource scheduling in CHS has to face with a great of challenges since managing the trade-off of efficiency and quality becomes complicated due to the uncertainty of patient choice behavior. Motivated by this, a resource scheduling problem with multi-stations queueing network in CHS is studied in this paper. A Markov decision model with uncertainty is developed to optimize the match process of patients and scarce resources with the objective of minimizing the total medical costs that consist of three conflicting sub-costs, i.e., medical costs, waiting time costs and the penalty costs caused by unmuting choice behavior of patients. For solving the proposed model, a three-stage dynamic scheduling method is designed, in which an improved Q-learning algorithm is employed to achieve the optimal schedule. Numerical experimental results show that this Q-learning-based scheduling algorithm outperforms two traditional scheduling algorithms significantly, as well as the balance of the three conflicting sub-costs is kept and the service efficiency is improved.