A Deep Reinforcement Learning-Enabled Dynamic Redeployment System for Mobile Ambulances

Abstract

Protecting citizens' lives from emergent accidents (e.g. traffic accidents) and diseases (e.g. heart attack) is of vital importance in urban computing. Every day many people are caught in emergent accidents or diseases and thus need ambulances to transport them to hospitals. In this paper, we propose a dynamic ambulance redeployment system to reduce the time needed for ambulances to pick up patients and to increase the probability of patients being saved in time. For patients in danger, every second counts. Specifically, whenever there is an ambulance becoming available (e.g. finishing transporting a patient to a hospital), our dynamic ambulance redeployment system will redeploy it to a proper ambulance station such that it can better pick up future patients. However, the dynamic ambulance redeployment is challenging, as when we redeploy an available ambulance we need to simultaneously consider each station's multiple dynamic factors. To trade off these multiple factors using handcrafted rules are almost impossible. To deal with this issue, we propose using a deep neural network, called deep score network, to balance each station's dynamic factors into one score, leveraging the excellent representation ability of deep neural networks. And then we propose a deep reinforcement learning framework to learn the deep score network. Finally, based on the learned deep score network, we provide an effective dynamic ambulance redeployment algorithm. Experiment results using data collected in real world show clear advantages of our method over baselines, e.g. comparing with baselines, our method can save ~100 seconds (~20%) of average pickup time of patients and improve the ratio of patients being picked up within 10 minutes from 0.786 to 0.838. With our method, people in danger can be better saved.