Opportunistic Routing with Q-Learning for Marine Wireless Sensor Networks

Abstract

In the next generation MWCNs, it is very challenging to monitor the vast marine areas, especially in the deep ocean. To this end, billions of sensors will be deployed over the different depths in the ocean or on the sea surface from Marine Wireless Sensor Networks (MWSNs), to monitor the marine environment and surveil the marine ecosystem. As described in Sect. 1.2.4 , energy efficiency is a more difficult challenge for routing in MWSNs due to the harsh marine environment compared with terrestrial wireless sensor networks. In this chapter, we propose an Energy-efficient Depth-based Opportunistic Routing Algorithm with Q-learning (EDORQ) for MWSNs to guarantee the energy-saving and reliable data transmission. It takes the advantages of both Q-learning technique and Opportunistic Routing (OR) algorithm without the full-dimensional location information to improve the network performance in terms of energy consumption, average network overhead, and packet delivery ratio. In EDORQ, the void detection factor, residual energy, and depth information of candidate nodes are jointly considered when defining the Q-value function, which contributes to proactively detect void nodes in advance, meanwhile, to reduce energy consumption. In addition, a simple and scalable void node recovery mode is proposed for the selection of candidate set so as to rescue packets that are stuck in void nodes unfortunately. Furthermore, we design a novel method to set the holding time for the schedule of packet forwarding based on Q-value so as to alleviate the packet collision and redundant transmission. We conduct extensive simulations to evaluate the performance of our proposed algorithm and compare it with other three routing algorithms on Aqua-sim platform (NS2). The results show that the proposed algorithm significantly improves the performance in terms of energy efficiency, packet delivery ratio, and average network overhead without sacrificing too much average packet delay.