Optimizing Energy Harvesting in Wireless Body Area Networks: A Deep Reinforcement Learning Approach to Dynamic Sampling

Abstract

Listen

Translate article

In energy-harvesting wireless body area networks (EH-WBAN), the self-sustainability of body sensor nodes without compromising the service quality criteria is very important. In remote vital signs monitoring applications, the sampling rate of body nodes in each period should be determined to achieve this goal. There are two fundamental challenges in determining the sampling rate: 1) In EH-WBAN, the rate of the harvestable energy is time-dependent and unpredictable. 2) The vital signs of the patient have different change rates relevant to the time. Therefore, a technique is required that can learn the changes of the harvestable energy and the change rates of vital signs simultaneously and specify the proper sampling rate for BNs. The reinforcement learning algorithms are among the efficient solutions for this problem because they are capable of learning environmental uncertainties. Previous RL-based methods for determining the BN's sampling rate have three fundamental problems: 1) focus on the optimization of the transmission energy and ignore the sensing energy, 2) only affect one of the two aspects of energy or sensed data in determining the sampling rate and 3) discretization of the problem space does not ensure the determination of the optimal sampling rate. Therefore, this paper proposes a method named “deep reinforcement learning-based dynamic sampling” (DRDS) which first formulates the sampling rate determination problem as a Markov decision process (MDP) and proposes a deep deterministic policy gradient algorithm (DDPG) to solve it. The proposed method considers both energy and data variability aspects in determining the sampling rate. Two parameters, the super-capacitor’s voltage level, and ambient light intensity, are considered for the energy aspect; for the data variability aspect, the long short-term memory (LSTM) algorithm is developed to predict the change rate of data in the next round. Simulations indicate that by preserving the data integrity, the proposed method can decrease the sampling rate and unnecessary data transmission by about 49.95% and 89.7%, respectively, compared with state-of-the-art methods, and allow the sensor to achieve self-sustainability.