Optimal Mode Selection in D2D Communication With Deadline Constraint: A CMDP Perspective

Abstract

Device to device (D2D) communication is a potential technology in the next generation communication system. We study the mode selection problem in a scenario where the D2D communication has a deadline constraint while the cellular user requires a minimum rate. We formulate the mode selection problem as a Constrained Markov Decision Process (CMDP). By adopting the Lagrangian multiplier technique, we first transform the CMDP into a non-constrained MDP problem. Then, we propose an optimal stationary mode selection algorithm combining the value iteration algorithm and the sub-gradient updating of the Lagrangian multiplier. Further, we propose an optimal randomized mode selection algorithm. Moreover, we contribute on deriving the closed-form policy selection probability. Simulation results have verified the optimality of the proposed algorithm.