Decentralized Decision for Multi-Band Sensing: A Deep Reinforcement Learning Approach

Abstract

This letter focuses on seeking a robust decentralized solution of multi-band sensing-decision-making (MBSDM) for cognitive wireless networks (CWN). As the MBSDM process of each agent in CWN can be regarded as a Partially Observable Markov Decision Problem (POMDP), we propose a distributed MBSDM algorithm based on distributed reinforcement learning with Multi-Agent Deep Deterministic Policy Gradient (MADDPG) strategy to overcome the partial observability and prohibitive computation. The MADDPG is implemented with offline centralized training and online decentralized execution. For the centralized training, the network of each agent is trained offline with the mixed actions and state information in experience pool. While in decentralized implementation, each agent calculates the local observation based on its belief state and takes action by the well-trained network independently. Comparing with existing algorithms, simulation results showcase the effectiveness and robustness of the proposed MADDPG based decentralized MBSDM algorithm.