A Q-learning based iterated local search algorithm for human-UAV cooperation in restoring transmission network

Abstract

The power transmission network is easily to be destroyed when natural or man-made disasters occur. Restoration of power supply under disaster environments faces difficulties since a large-scale network typically contains many uninspected faulty nodes. Utilizing unmanned aerial vehicles (UAVs) to inspect these unknown faulty nodes can significantly improve the efficiency for subsequent restoration work performed by human-teams. Nevertheless, efficient cooperation of UAV and human-team is a complicated work due to the complexity of network structure and correlation between UAV scheduling and human-team scheduling. In this paper, a mathematical model is established to describe the considered problem aiming at maximizing the restored power supply in a limited response time. Then a Q-learning based iterated local search (Q ILS) algorithm is proposed to formulate the collaborative scheduling problem. Firstly, an initialization method is designed to assign UAVs for inspecting unknown faulty nodes and human-teams for repairing faulty nodes, which ensures each unknown faulty node is inspected before maintenance. Secondly, searching operators including perturbation and local search procedures are designed to ensure exploration and exploitation capability. Thirdly, Q-learning method is utilized as a learning engine to guide the direction of solution evolution. Moreover, the parameters of Q ILS are calibrated by multi-factor analysis of variance method to determine proper values. The computational simulations and comparison experiments validate the superiority of proposed algorithm.