Status Update for Correlated Energy Harvesting Sensors: A Deep Reinforcement Learning Approach

Abstract

For many real-time applications provided by the Internet of Things (IoT) networks, it is necessary to integrate the information generated by multiple correlated sensors and hence, how to guarantee the freshness of the integrated information by designing efficient dynamic status update strategies becomes the key issue. In this paper, we consider an IoT network with multiple correlated sensors powered by energy harvesting (EH) techniques, and focus on improving the information freshness by appropriately activating the sensors to update the status. Particularly, we adopt the concept of age of correlated information (AoCI) to characterize the information freshness for correlated sensors and then, formulate a dynamic status update optimization problem to minimize the observed long-term average AoCI, where the transmission resource constraint and energy causality are jointly considered. To solve this problem, a Markov Decision Process (MDP) is formulated to cast the status update procedure, and a deep reinforcement learning based status update algorithm is devised by imbedding the action elimination mechanism into the standard deep Q-network, with which the challenges from the unknown of the environmental dynamics, curse of dimensionality, and coupling between the valid actions and states can be simultaneously addressed.