Reinforcement Learning (RL)-Based Energy Efficient Resource Allocation for Energy Harvesting-Powered Wireless Body Area Network.

Yi-Han Xu,Jing-Wei Xie,Min Hua,Wen Zhou,Yang-Gang Zhang

doi:10.3390/s20010044

Abstract

Wireless body area networks (WBANs) have attracted great attention from both industry and academia as a promising technology for continuous monitoring of physiological signals of the human body. As the sensors in WBANs are typically battery-driven and inconvenient to recharge, an energy efficient resource allocation scheme is essential to prolong the lifetime of the networks, while guaranteeing the rigid requirements of quality of service (QoS) of the WBANs in nature. As a possible alternative solution to address the energy efficiency problem, energy harvesting (EH) technology with the capability of harvesting energy from ambient sources can potentially reduce the dependence on the battery supply. Consequently, in this paper, we investigate the resource allocation problem for EH-powered WBANs (EH-WBANs). Our goal is to maximize the energy efficiency of the EH-WBANs with the joint consideration of transmission mode, relay selection, allocated time slot, transmission power, and the energy constraint of each sensor. In view of the characteristic of the EH-WBANs, we formulate the energy efficiency problem as a discrete-time and finite-state Markov decision process (DFMDP), in which allocation strategy decisions are made by a hub that does not have complete and global network information. Owing to the complexity of the problem, we propose a modified Q-learning (QL) algorithm to obtain the optimal allocation strategy. The numerical results validate the effectiveness of the proposed scheme as well as the low computation complexity of the proposed modified Q-learning (QL) algorithm.

Highlights

Recent advances in sensors and wireless communication technology have resulted in a promising development of wireless body area networks (WBANs) [1]
The resource allocation problem jointly considers the transmission mode, relay selection, allocated time slots, transmission power, and energy status to make the optimal allocation decision; We formulate the energy efficiency problem to be a discrete-time and finite-state Markov decision process (DFMDP) and a modified Q-learning algorithm, which reduces the state-action space in the original Q-learning algorithm, is proposed to solve the modeled problem; From the numerical analysis, we show that the proposed scheme can obtain the best energy efficiency and with the more rapid convergence speed by eliminating the irrelevant exploration space in the Q-table as compared with the classical Q-learning algorithm
We suppose that only body sensors are equipped with the energy harvesting function, and the energy harvesting process is Poisson-distributed with a rate λe at arrival instants tk

Summary

Introduction

Recent advances in sensors and wireless communication technology have resulted in a promising development of wireless body area networks (WBANs) [1]. Several previous works in the literature were proposed to investigate the energy-saving technologies from the aspects of the media access control (MAC) protocol design, power control, and cross-layer resource allocation strategies to make efforts to prolong the lifetime of WBANs [6,7,8,9,10,11,12]. In the work of [7], the authors presented a time division multiple access (TDMA)-based technique to improve WBANs’ reliability and energy efficiency by adaptively synchronizing nodes while tackling channel and buffer status. The authors considered the signal-to-interference plus noise ratio, transmission priority, battery level, and transmission delay to make a decision on the access time and transmit power. In the work of [10], a multi-hop routing protocol and routing decision strategies in WBANs for medical application were proposed

Objectives

Results

Conclusion