Q-learning-Based Opportunistic Routing with an on-site architecture in UASNs

Abstract

Underwater Acoustic Sensor Networks (UASNs) demonstrate powerful detection capabilities. Diversified underwater applications are emerging, resulting in a substantial increase in the types and amount of perception data. Thus, UASNs need to transmit data to the data center reliably and efficiently. However, problems such as long paths, multiple hops and long delay, etc limit the quality of data transmission. Motivated by the real-time and effective upload of data, we propose a Q-learning-Based Opportunistic Routing (QBOR) protocol with an on-site architecture. The on-site architecture differs from the traditional model of placing the data center on the surface. We deploy the data center underwater closer to the data source. In order to adapt to this new architecture, the QBOR protocol is proposed, which transmits data to seabed. In QBOR, we define a reward function, in which the packet delivery probability and residual energy are considered in routing to obtain higher Packet Delivery Ratio (PDR) and energy efficiency. And a Q-value-based wait-competition mechanism derived from the opportunistic routing paradigm is proposed. This mechanism determines the forwarding priority through the competition of holding time to reduce packet redundancy and collisions. Our results show that the new architecture has obvious advantages in energy and delay. Then we analyze the performance of QBOR at varying network sparse scales from the points of the PDR, energy efficiency and end-to-end delay. Compared with other existing protocols, QBOR outperforms them in the three indicators, especially the PDR is significantly improved.