A reinforcement learning-based cluster routing scheme with dynamic path planning for mutli-UAV network

Abstract

Unmanned Aerial Vehicles (UAVs) with visual sensors are widely used for area mapping, management of crops and traffic, rescuing lives, and many other applications that need to cover a large area. The process of coverage can be improved with the use of efficient path planning and data transfer algorithms. Numerous research studies have been performed by concentrating on each of the aforementioned elements separately. However, to tackle the rapidly changing environmental situations, this paper proposes a cluster-based routing approach by incorporating a dynamic planning algorithm. The proposed model is composed of four modules, such as an online path planning algorithm, clustering-based network topology construction, reinforcement learning-based cluster management, and a data routing mechanism. Firstly, to maximise the coverage output, an optimal set of waypoints has been generated for all UAVs. For each UAV to complete the coverage task, it needs to completely cover its own set of waypoints. Since the environment is changing, a static path planning approach might fail to achieve complete coverage. Therefore, to drive the mission without getting stuck, a dynamic path planning approach is proposed that decides the next waypoint for a UAV based on the current waypoint. The main purpose of the algorithm is to cover all the waypoints in a polynomial amount of time. Secondly, the topology construction module consists of the initialization process, cluster head election, and cluster formation. Based on five parameters such as degree of centrality, surplus energy, link stability time, connectivity with the backbone UAV, and velocity, a set of cluster heads are selected. Then the cluster management process is performed by the optimal re-clustering policy determined using an approach of reinforcement learning called State Action Reward State Action (SARSA) in ground control station. Finally, the introduction of inter-cluster forwarders and selective flooding of route requests makes the routing scheme enhance the packet delivery ratio and reduce the delay. The result shows that the proposed work performs better than the existing result in terms of different generic performance metrics used for path planning and routing.