Throughput and latency in the distributed Q-learning random access mMTC networks

Abstract

In mMTC mode, where thousands of devices try to access network resources sporadically, the problem of random access (RA) and collisions between devices that select the same resources arise. A promising approach to solve the RA problem is the use of learning mechanisms, specially Q-learning (QL) algorithm, where the devices learn about the best time-slot periods to transmit through rewards sent by the central node. In this work, we propose a distributed packet-based learning method of varying the reward given by the central node that favors devices having a larger number of remaining packets to transmit. The numerical results indicated that the proposed distributed packet-based QL method attains a better throughput–latency trade-off than the independent and collaborative techniques in practical scenarios, while the number of payload bits of the packet-based technique is reduced regarding the collaborative QL RA technique for achieving the same normalized throughput.