Dynamic adaptation of contention window boundaries using deep Q networks in UAV swarms

Abstract

In flying ad hoc networks (FANET), medium access layer (MAC) protocols play an essential role in ensuring better network performance. The effective utilization of network resources and providing access fairness are key research issues in this area, and the contention window size directly influences these factors. This paper mainly focuses on addressing the upper and lower boundaries of the contention window in Carrier Sense Multiple Access/Collision Avoidance (CSMA/CA) with a binary exponential back-off algorithm. Existing solutions include static and dynamic window adjustment techniques only to update the upper boundary of the contention window size. However, the static window adjustment technique is unsuitable for changing network conditions, while the dynamic window adjustment technique may need to be more efficient in dense network environments. The proposed solution uses a Deep Q-learning algorithm(DQN) framework to adjust the contention window adaptively based on the reward function, resulting in efficient and dynamic adaptation to changing network conditions among unmanned aerial vehicles (UAV). The proposed methodology is evaluated through simulations, and performance is measured using channel utilization, efficiency, delay, and collision rate. The simulation findings show that, in comparison to generic MAC protocols, the proposed algorithm is 5% more efficient in throughput and incurs a four-fold reduction in delay.