Long-Term Energy Collection in Self-Sustainable Sensor Networks: A Deep Q-Learning Approach

Abstract

This article reports on the development of a deep Q-learning approach to long-term energy collection in self-sustainable sensor networks, which consists of two static chargers (SCs) and one rechargeable mobile sensor (MS). In particular, we assume that the SCs can harvest energy from the ambient environment and charge the MS via electromagnetic (EM) radiation. As the energy harvesting (EH) process is random and the radiated energy fades over distance, the achievable energy by the MS at each slot demonstrates spatiotemporal dynamics in a certain area. Thus, we focus on the problem of trajectory optimization for an autonomous MS to maximize the long-term average achievable energy per slot from both chargers. Due to the inaccessible charger-side information, such as the EH profile and locations of SCs as well as the transmit power, we introduce deep Q-learning, a model-free reinforcement learning approach, based on which the MS can learn and optimize the moving trajectory by intelligently tracking the aggregated received EM signal without any other explicit external information. Simulation results show that the MS can identify the best energy collecting location and finally moves there along the learned trajectory. We also investigate the impact of system parameters, such as initial position and moving cost per unit distance on the performance of the proposed training algorithm, such as convergence rate and stability via extensive numerical evaluations.