Contextual Multi-Armed Bandit for Cache-Aware Decoupled Multiple Association in UDNs: A Deep Learning Approach

Abstract

The new trends in network densification and heterogeneity introduce new challenges for user association. Currently, user association is typically coupled, with which a user is constrained to associate with the same base station in both uplink (UL) and downlink (DL). In homogeneous networks, this is simple and effective. However, it could restrict the performance in heterogeneous ultra dense networks (UDNs), wherein BSs are densely deployed with highly variable transmit powers and topologies. Besides, the backhaul capacity could be the bottleneck in UDNs, and caching popular contents at BSs becomes an effective method for alleviating the backhauling traffic. In this paper, we propose a novel cache-aware decoupled multiple association mechanism for full-duplex UDNs, which allows a user to associate with multiple BSs in UL and DL, in a decoupled manner. Considering that users can form a self-learning system, a contextual multi-armed bandit (CMAB) problem is formulated where network states are unknown random variables. For obtaining the optimal strategy, an SINR-based linear upper confidence bound algorithm is developed. And deep learning is adopted when considering a large-scale network with expansion of the dimension in state space. The convergence of the algorithm is proven. Simulation results validate the feasibility and superiority of the proposed approach by comparisons.