Resource Allocation in Time Slotted Channel Hopping (TSCH) Networks Based on Phasic Policy Gradient Reinforcement Learning

Abstract

The concept of the Industrial Internet of Things (IIoT) is gaining prominence due to its low-cost solutions and improved productivity of manufacturing processes. To address the ultra-high reliability and ultra-low power communication requirements of IIoT networks, Time Slotted Channel Hopping (TSCH) behavioral mode has been introduced in IEEE 802.15.4e standard. Scheduling the packet transmissions in IIoT networks is a difficult task owing to the limited resources and dynamic topology. In IEEE 802.15.4e TSCH, the design of the schedule is open to implementation. In this paper, we propose a phasic policy gradient (PPG) based TSCH schedule learning algorithm. We construct the utility function that accounts for the throughput, and energy efficiency of the TSCH network. The proposed PPG based scheduling algorithm overcomes the drawbacks of totally distributed and totally centralized deep reinforcement learning-based scheduling algorithms by employing the actor–critic policy gradient method that learns the scheduling algorithm in two phases, namely policy phase and auxiliary phase. In this method, we show that the schedule converges quickly compared to any other actor–critic method and also improves the system throughput performance by 58% compared to the minimal scheduling function, a default TSCH schedule.