QoS-oriented media access control using reinforcement learning for next-generation WLANs

Abstract

Orthogonal frequency division multiple access (OFDMA) is introduced in IEEE 802.11ax to satisfy massive transmission demands. However, the uplink OFDMA-based random access (UORA) mechanism provides poor quality of service (QoS), and restricts channel utilization efficiency, especially in the dynamic network environment. To solve these issues, in this study, the transmission period is decoupled into a contention stage and a transmission stage, and an intelligent media access control (MAC) algorithm with QoS-guaranteed for next-generation wireless local area networks (WLANs) is presented. Specifically, we consider stochastic and various traffic in time-varying wireless communication conditions and propose two contention window (CW) optimization mechanisms based on Q-learning (QL) and a deep Q-network (DQN) (referred to as QL-MAC and DQN-MAC) for static and dynamic network scenarios, respectively. Meanwhile, the access point (AP) acts as the reinforcement learning (RL) agent and centrally optimizes the CW for all stations to eventually maximize the system throughput. Furthermore, we provide the QoS-guaranteed channel access mechanism for different priority data traffic, in which high-priority traffic can obtain more channel access opportunities in the uplink contention stage. Simulation results demonstrate that the proposed algorithms can significantly improve the network performance in terms of convergence, throughput, delay, and fairness compared to the adaptive grouping-based two-stage mechanism (BTM) and double random access QoS-oriented OFDMA MAC (DRA-OFDMA) algorithm in various scenarios.