An optimal policy for joint compression and transmission control in delay-constrained energy harvesting IoT devices

Abstract

Energy-efficient communication remains one of the key requirements of the Internet of Things (IoT) platforms. The concern on energy consumption can be mitigated by exploiting technical ploys to reduce the volume of data for transmission (e.g., via sensing data compression) as well as by resorting to technological advancements (e.g., energy harvesting). However, these mitigating measures carry their own cost, which is the additional complexity of control and optimization in the digital communication chain. In particular, compression ratio is another control knob that needs adjusting besides the usual transmission parameters. Also, with the random and sporadic nature of the harvested energy, the goal shifts from mere energy conservation to judicious consumption of the renewable energy in a foresighted manner. In this paper, we assume an energy-harvesting IoT device that is tasked with (loss-lessly) compressing and reporting delay-constrained sensing events to an IoT gateway over a time-varying wireless channel. We are interested in computing an optimal policy for joint compression and transmission control adaptive to the node’s energy availability, transmission buffer length, as well as its wireless channel conditions. We cast the problem as a Constrained Markov Decision Process (CMDP), and propose a two-timescale model-free reinforcement learning (RL) algorithm that is able to shape the optimal control policy in the absence of the statistical knowledge of the underlying system dynamics. Exhaustive simulation experiments are conducted to investigate the convergence of the learning algorithm, to explore the impacts of different system parameters (such as: the rate of sensing events, the energy arrival rate, and battery capacity) on the performance of the proposed policy, as well as to compare against some baseline schemes.