Performance Evaluation of Reinforcement Learning Based Distributed Channel Selection Algorithm in Massive IoT Networks

Abstract

In recent years, the demand for new applications using various Internet of Things (IoT) devices has led to an increase in the number of devices connected to wireless networks. However, owing to the limitation of available frequency resources for IoT devices, the degradation of the communication quality caused by channel congestion is a practical problem in developing IoT technology. Many IoT devices have hardware and software limitations that prevent centralized channel allocation, and congestion is even more severe in massive IoT networks without a central controller. Therefore, developing a distributed and sophisticated channel selection algorithm is necessary. In previous studies, the channel selection of each IoT device was modeled as a multi-armed bandit (MAB) problem, and a wireless channel selection method based on the MAB algorithm, which is a simple reinforcement learning, was proposed. In particular, it has been shown that the MAB algorithm of tug-of-war (TOW) dynamics can efficiently select channels with much lower computational complexity and power compared with other reinforcement learning-based channel-selection methods. This paper proposes a distributed channel selection method based on TOW dynamics in fully decentralized networks. We evaluate the effectiveness of the proposed method and other distributed channel-selection methods on the communication success rate in massive IoT networks by experiments and simulations. The results show that the proposed method improves the communication success rate more than other distributed channel selection methods even in a dense and dynamic network environment.