Energy-Efficient UAV-Enabled Data Collection via Wireless Charging: A Reinforcement Learning Approach

Abstract

In this article, we study the application of unmanned aerial vehicle (UAV) for data collection with wireless charging, which is crucial for providing seamless coverage and improving system performance in the next-generation wireless networks. To this end, we propose a reinforcement learning-based approach to plan the route of UAV to collect sensor data from sensor devices scattered in the physical environment. Specifically, the physical environment is divided into multiple grids, where one spot for UAV hovering as well as the wireless charging of UAV is located at the center of each grid. Each grid has a spot for the UAV to hover, and moreover, there is a wireless charger at the center of each grid, which can provide wireless charging to UAV when it is hovering in the grid. When the UAV lacks energy, it can be charged by the wireless charger at the spot. By taking into account the collected data amount as well as the energy consumption, we formulate the problem of data collection with UAV as a Markov decision problem, and exploit $Q$ -learning to find the optimal policy. In particular, we design the reward function considering the energy efficiency of UAV flight and data collection, based on which $Q$ -table is updated for guiding the route of UAV. Through extensive simulation results, we verify that our proposed reward function can achieve a better performance in terms of the average throughput, delay of data collection, as well as the energy efficiency of UAV, in comparison with the conventional capacity-based reward function.