Scheduling Elective Surgeries with Markov Decision Process and Approximate Dynamic Programming

Abstract

This paper deals with the dynamic advance scheduling of elective surgeries with multiple sources of uncertainties taken into consideration. A waiting list is established to facilitate the management of elective patients from different specialties. Each patient in the waiting list is assigned a dynamic priority which is dependent on the relative importance of specialty, urgency level, and actual waiting time. At the end of each week, the number and type of elective surgeries to be performed in the following week should be properly determined to minimize an integrated cost function, including the costs incurred by performing and delaying surgeries as well as the penalties for overuse of operating rooms and shortage of recovery beds. The studied problem is formulated as an infinite-horizon Markov decision process (MDP) model. Considering that conventional dynamic programming algorithms cannot efficiently solve MDP models for real-sized problems, we develop an approximate dynamic programming (ADP) approach that combines recursive least-squares temporal difference learning and mixed integer programming. Results of numerical experiments validate the efficiency and accuracy of the proposed ADP approach and indicate that this approach can be employed by hospital managers in the future to efficiently solve real-sized surgery scheduling problems.