Learning to Transmit Fresh Information in Energy Harvesting Networks

Abstract

We study age of information (AoI) minimization in an ad hoc network consisting of energy harvesting transmitters that are scheduled to send status updates to their intended receivers. The transmission scheduling with power allocation problem over a communication session is first studied assuming apriori knowledge of channel state information, harvested energy, and update packet arrivals, i.e., the offline setting. The global optimal scheduling policy in this case is the solution of a mixed integer linear program which is known to be computationally hard. We propose a supervised-learning-based algorithm to mitigate the high computational complexity. A bidirectional recurrent neural network that interprets user scheduling as a time-series classification problem is trained and tested to achieve near-optimal AoI. Next, we consider online scheduling and power allocation with causal knowledge of the system state, which is an infinite-state Markov decision problem. In this case, the related reinforcement learning problem is solved by a model-free on-policy deep reinforcement learning, where the actor-critic algorithm with deep neural network function approximation is implemented. Comparable AoI to the optimal is demonstrated and faster runtime of learning solvers is observed, verifying the efficacy of learning in terms of both optimality and computational energy efficiency for AoI-focused scheduling and resource allocation problems in wireless networks.