Optimal sensor scheduling for remote state estimation with limited bandwidth: a deep reinforcement learning approach

Abstract

This paper considers co-scheduling for multiple sensors to observe multiple dynamical systems. The measurements obtained by the sensors need to be transmitted to the remote estimator over a shared network with packet dropouts, where only a part of sensors can access the network due to its limited bandwidth. To improve the estimation performance, a scheduler is adopted to determine which systems to be observed and which sensors to access the network based on the real-time information of each system. For this issue, the co-scheduling protocol is formulated as an associated Markov decision process (MDP) and the existence of an optimal deterministic and stationary policy (DSP) is proved. Then a Deep Q-Network (DQN) is developed to solve the MDP in a scalable and model-free manner. A practical example of vehicle moving is presented to compare the DQN method with some other existing scheduling protocols, and the results show that the developed approach significantly outperforms other protocols in all kinds of situations.