A Q-Learning-Based Adaptive MAC Protocol for Internet of Things Networks

Chien-Min Wu,Kai-Fu Chang,Cheng-Tai Tsai,Yen-Chun Kao,Cheng-Chun Hou

doi:10.1109/access.2021.3103718

Chien-Min Wu, Kai-Fu Chang + Show 3 more

Open Access

https://doi.org/10.1109/access.2021.3103718

Copy DOI

Abstract

In Internet of Things (IoT) applications, sometimes the quality of service (QoS) of throughput for transmitting video or the QoS of bounded delay for control of a sensor node is required. A traditional contention-based medium access control (MAC) protocol cannot meet the adaptive traffic demands of these networks and confers delay-related constraints. Q-learning (QL) is one of the reinforcement learning (RL) mechanisms and can potentially be the future machine learning scheme for spectrum MAC protocols in IoT networks. In this study, a QL-based MAC protocol is proposed to facilitate adaptive adjustment of the length of the contention period in response to the ongoing traffic rate in IoT networks. The novelty of QL-based MAC lies in its use of RL to dynamically adjust the length of the contention period according to the traffic rate. The QL-based MAC will solve the models without additional input information to adapt to environmental variations during training. We confirm that the proposed QL-based MAC protocol with node contention is robust. In addition, we showed that our proposed QL-based MAC protocol has higher system throughput, lower end-to-end delay, and lower energy consumption in MAC contention than those of contention-based MAC protocols.

Highlights

The Internet of Things (IoT) is a network with a variety of applications such as smart sensors, smart home appliances, and monitoring devices
The length of contention period for traditional medium access control (MAC) protocol is fixed in IoT networks
The proposed QL-based MAC (QL-MAC) protocol is feasible and the optimal length in the contention period can be achieved by using the Q-Learning algorithm

Summary

INTRODUCTION

The Internet of Things (IoT) is a network with a variety of applications such as smart sensors, smart home appliances, and monitoring devices. The adaptive length of the contention period regulation complies with the MDP formulation and QL was used to design a MAC protocol to enable the cluster head to select the appropriate length of the contention period in the network based on experience gained from agent-environment interactions. A. QL-BASED MAC PROTOCOL FOR IoT NETWORKS The adaptive length of the contention period problem suits to the MDP formulation. QL algorithm is used to design an IoT MAC protocol that can adaptively adjust the length of the contention period dynamically based on the obtained experience in the IoT communication area. The proposed QL-based adaptive IoT MAC protocol that adjusts the length of the contention period is based on the broadcast of the cluster head in order to avoid the contention collision. The BTC control frame has the following fields: IoTheadID−the ID of the cluster head; Lnext −the length of the contention period; IoTnodeIDi−the node IDi of the successful transmitter; SlotnodeIDi−the assigned slot for node IDi’s transmission

ACTION SELECTION

END-TO-END DELAY

ENERGY CONSUMPTION OF MAC CONTENTION

Findings

CONCLUSION