A Reinforcement Learning Approach to Dynamic Spectrum Access in Internet-of-Things Networks

Abstract

To support wireless communication traffic of Internet-of-Things (IoT) systems in terms of massive connectivity, dynamic spectrum access (DSA) is important issue. This paper proposes spectrum sensor-aided DSA system based on a reinforcement learning (RL) algorithm that aims at efficient spectrum usage for IoT network over the incumbent network. Due to small-form-factor of IoT devices, they do not have spectrum sensing capability. To support DSA of IoT devices, we introduce sensor-aided DSA system that enhances spatial spectrum reusability by means of RL algorithm. With the RL algorithm, proposed DSA system provides self-organizing feature for massive number of IoT devices. We show that the performance of proposed RL based DSA system in various densities of IoT devices utilizing slotted ALOHA protocol that has spectrum access probability learned by proposed DSA system. We also present the performance of proposed RL based DSA system surpass that of distributed Carrier Sensing Multiple Access with Collision Avoidance (CSMA/CA) protocol for channel access coordination. We also present the consistent performance of incumbent user when the IoT devices access to the spectrum band with learned spectrum access probability.