Anypath Routing Protocol Design via Q-Learning for Underwater Sensor Networks

Abstract

As a promising technology in the Internet of Underwater Things, underwater sensor networks (UWSNs) have drawn a widespread attention from both academia and industry. However, designing a routing protocol for UWSNs is a great challenge due to high energy consumption and large latency in the underwater environment. This article proposes a Q-learning -based localization-free anypath routing (QLFR) protocol to prolong the lifetime as well as reduce the end-to-end delay for UWSNs. Aiming at optimal routing policies, the Q-value is calculated by jointly considering the residual energy and depth information of sensor nodes throughout the routing process. More specifically, we define two reward functions (i.e., depth-related and energy-related rewards) for Q-learning with the objective of reducing latency and extending network lifetime. In addition, a new holding time mechanism for packet forwarding is designed according to the priority of forwarding candidate nodes. Furthermore, mathematical analyses are presented to analyze the performance and computational complexity of the proposed routing protocol. Extensive simulation results demonstrate the superiority performance of the proposed routing protocol in terms of the end-to-end delay and the network lifetime.