Deep reinforcement learning-based spectrum resource allocation for the web of healthcare things with massive integrating wearable gadgets

Abstract

With the development of the future Web of Healthcare Things (WoHT), there will be a trend of densely deploying medical sensors with massive simultaneous online communication requirements. The dense deployment and simultaneous online communication of massive medical sensors will inevitably generate overlapping interference. This will be extremely challenging to support data transmission at the medical-grade quality of service level. To handle the challenge, this paper proposes a hypergraph interference coordination-aided resource allocation based on the Deep Reinforcement Learning (DRL) method. Specifically, we build a novel hypergraph interference model for the considered WoHT by analyzing the impact of the overlapping interference. Due to the high complexity of directly solving the hypergraph interference model, the original resource allocation problem is converted into a sequential decision-making problem through the Markov Decision Process (MDP) modeling method. Then, a policy and value-based resource allocation algorithm is proposed to solve this problem under simultaneous online communication and dense deployment. In addition, to enhance the exploration ability of the optimal allocation strategy for the agent, we propose a resource allocation algorithm with an asynchronous parallel architecture. Simulation results verify that the proposed algorithms can achieve higher network throughput than the existing algorithms in the considered WoHT scenario.