A Reinforcement Learning Model of Multiple UAVs for Transporting Emergency Relief Supplies

Abstract

In large-scale disasters, such as earthquakes and tsunamis, quick and sufficient transportation of emergency relief supplies is required. Logistics activities conducted to quickly provide appropriate aid supplies (relief goods) to people affected by disasters are known as humanitarian logistics (HL), and play an important role in terms of saving the lives of those affected. In the previous last-mile distribution of HL, supplies are transported by trucks and helicopters, but these transport methods are sometimes not feasible. Therefore, the use of unmanned aerial vehicles (UAVs) to transport supplies is attracting attention due to their convenience regardless of the disaster conditions. However, existing transportation planning that utilizes UAVs may not meet some of the requirements for post-disaster transport of supplies. Equitable distribution of supplies among affected shelters is particularly important in a crisis situation, but it has not been a major consideration in the logistics of UAVs in the existing study. Therefore, this study proposes transportation planning by introducing three crucial performance metrics: (1) the rapidity of supplies, (2) the urgency of supplies, and (3) the equity of supply amounts. We formulated the routing problem of UAVs as the multi-objective, multi-trip, multi-item, and multi-UAV problem, and optimize the problem with Q-learning (QL), one of the reinforcement learning methods. We performed reinforcement learning for multiple cases with different rewards and quantitatively evaluated the transportation of each countermeasure by comparing them. The results suggest that the model improved the stability of the supply of emergency relief supplies to all evacuation centers when compared to other models.