Joint routing and computation offloading based deep reinforcement learning for Flying Ad hoc Networks

Abstract

Flying ad-hoc networks (FANETs) consisting of multiple Unmanned Aerial Vehicles (UAVs) are widely used due to their flexibility and low cost. In scenarios such as crowdsensing and data collection, data collected by UAVs are transmitted to base stations for processing and then sent to data centers. Still, the deployment of base stations is costly and inflexible. To address this issue, this paper introduces a position-based Computing First Routing (CFR) protocol designed for efficient task transmission and computation offloading in FANETs. This protocol facilitates task processing during data transfer and ensures the delivery of fully processed results to the data center. Considering the dynamically changing topology of the FANETs and the uneven distribution of the UAVs’ computation power, deep reinforcement learning is used to make multi-objective decisions based on the Q-values computed by the model. FANETs are centerless clusters, and two-hop neighbor tables containing position and computing power information are used to make less costly decisions. Simulation experiments demonstrate that CFR outperforms other benchmark schemes with an approximately 6% higher packet delivery rate, an approximately 21% reduction in end-to-end delay, and about a 34% decrease in total cost. Furthermore, it effectively ensures the completion of task offloading before reaching the destination node. This occurs due to the design of a hierarchical reward function that takes into account dynamic changes in delay and energy consumption, as well as the injection of neighbor computing power information into the two-hop neighbor table.