Dual-attention assisted deep reinforcement learning algorithm for energy-efficient resource allocation in Industrial Internet of Things

Abstract

This paper investigates the distributed resource allocation problem for energy-efficient data forwarding in resource-constrained Industrial Internet of Things (IIoT). We formulate this problem as a decentralized partially observable Markov decision process (Dec-POMDP) and propose a novel energy-efficient resource allocation algorithm based on the dual-attention assisted deep reinforcement learning (DRL) model, namely DADR, which adopts the centralized training and distributed executed (CTDE) framework to provide the intelligent decision-making ability for resource-constrained nodes. In DADR, we design a multi-scale convolutional attention module (CAM) in actor network that can extract the local state’s feature information from different dimensions; meanwhile, we propose a novel critic network based on dual-attention module and experience reconstruction module which provides a more objective and correct state evaluation from a global perspective. Moreover, the proposed critic network can solve the partially observable and non-stationary problems in multi-agent systems and can flexibly scale without changing the model structure even in a dynamic environment. Furthermore, the cooperation between CAM and multi-head self-attention (MHSA) in DADR improves the representation learning ability of DRL and offers better optimization direction for the DRL model to maximize the energy-efficient and data transmission reliability. Simulation results demonstrate that the proposed DADR algorithm outperforms the existing resource allocation algorithms and MARL models in terms of network lifetime, transmission reliability, and network stability, respectively.