This paper studies the application of downlink multi-carrier non-orthogonal multiple access (MC-NOMA) and device-to-device (D2D) in heterogeneous networks (HetNets). In the considered model, the small base stations (SBSs) can reuse the sub-channels (SCs) of the macro base stations (MBS). The paper aims to maximize the system sum-rate while managing co-tier and co-channel interference by optimizing joint SBS selection, fair SC allocation, and power allocation, under the constraints of guaranteeing a certain amount of resources to all cellular users (CUs)and D2D pairs and satisfying CUs’ minimum signal-to-interference-plus-noise ratio (SINR) requirements. However, obtaining an efficient exact solution is challenging due to the non-convexity of the mixed integer optimization problem. To deal with this issue, the problem is decoupled into two sub-problems: 1- SBS selection and fair SC allocation problem and 2- power allocation problem. Then, a four-sided many-to-one matching method is proposed to solve the SBS selection and fair SC allocation problem jointly. The performance of the SC fair allocation algorithm is investigated in terms of the system quality of services (QoS) using a proposed mathematical expression. Furthermore, a novel dynamic search operator is introduced to enhance the performance of the matching-based algorithm. Next, the non-convex power allocation problem is transformed into a deep reinforcement learning (DRL) based framework, and the deep deterministic policy gradient (DDPG) approach is employed to solve the problem. Moreover, a joint SBS selection, SC allocation, and power allocation iterative algorithm is proposed to improve the system sum-rate. Simulation results validate our analysis and indicate the superiority of the proposed algorithms in terms of the system sum-rate compared to the benchmark algorithms.
Read full abstract