Bayesian Reinforcement Learning-Based Coalition Formation for Distributed Resource Sharing by Device-to-Device Users in Heterogeneous Cellular Networks

Abstract

This paper investigates the problem of distributed resource sharing in a device-to-device enabled heterogeneous network, where the various device pairs choose their transmission channels, modes, base stations (BSs), and power levels without any control by the BSs based only on the locally-observable information. This problem is represented as a Bayesian coalition formation game, where the players (device pairs) create coalitions to maximize their long-term rewards with no prior knowledge of the values of potential coalitions and the types of their members. To minimize these uncertainties, a novel Bayesian reinforcement learning (RL) model is derived. In this model, the players update (through repeated coalition formation) their beliefs about the types and coalitional values to reach a stable coalitional agreement. The proposed Bayesian RL-based coalition formation algorithms are implemented in a long-term evolution advanced network and evaluated using simulations. The algorithms show a superior performance when compared with other relevant resource allocation schemes and achieve near-optimal results after a relatively small number of RL iterations.