Deep Reinforcement Learning-Based Deterministic Routing and Scheduling for Mixed-Criticality Flows

Abstract

Deterministic networking (DetNet) has recently drawn much attention by investigating deterministic flow scheduling. Combined with artificial intelligent (AI) technologies, it can be leveraged as a promising network technology for facilitating automated network configuration in the Industrial Internet of Things (IIoT). However, the stricter requirements of the IIoT have posed significant challenges, that is, deterministic and bounded latency for time-critical applications. This paper incorporates deep reinforcement learning (DRL) in Cycle Specified Queuing and Forwarding (CSQF) and proposes a DRL-based Deterministic Flow Scheduler (Deep-DFS) to solve the Deterministic Flow Routing and Scheduling (DFRS) problem. Novel delay aware network representations, action masking and criticality aware reward function design are proposed to make Deep-DFS more scalable and efficient. Simulation experiments are conducted to evaluate the performances of Deep-DFS, and the results show that Deep-DFS can schedule more flows than the other benchmark methods (heuristic-based and AI-based methods).