Adaptive maximum-lifetime routing in mobile ad-hoc networks using temporal difference reinforcement learning

Abstract

Mobile ad-hoc NETworks (MANETs) are very dynamic environments. A routing protocol for MANETs should be adaptive in order to operate correctly in presence of variable network conditions. Reinforcement learning (RL) is a recently used technique to achieve adaptive routing in MANETs. In comparison to other machine learning and computational intelligence techniques, RL achieves optimal results at low processing and medium memory costs. To deal with adaptive energy-aware routing issue in MANETs, a RL-based maximum-lifetime routing strategy is proposed. Each mobile node learns how to adjust its route-request packets forwarding-rate according to its energy profile. In terms of RL-resolution methods, Q-Learning, SARSA, Q(λ) and SARSA(λ) which are Temporal difference RL-algorithms are used. The proposed RL model is implemented on the top of AODV routing protocol. Simulation results show that the RL-based AODV achieved good performances in comparison to Time-Delay and Probability based AODV. Particularly, the Q-Learning based AODV has marked the best global performances in terms of energy efficiency and end to end delay.