Multi-attribute Ad Hoc Network Routing Selection Based on Option-critic

Abstract

Single-attribute decision-making is often unable to achieve ideal results in Ad Hoc routing problems. Therefore, a multi-attributes decision-making routing algorithm for Ad Hoc network is proposed in order to achieve better optimization results. Compared with single-attribute decision-making, the main issue of multi-attributes decision-making is the Incommensurability between attributes and the contradiction produced by attributes. To deal with this problem, this paper proposes a reinforcement learning based multi-attributes decision-making routing algorithm for Ad Hoc network. We adopt option-critic framework based on hierarchical reinforcement learning to train agents and take into account the transmission bandwidth, packet loss rate, number of communication nodes, transmission delay. For each decision-making attribute, the combination weighting method is used to calculate the weight of each decision-making attribute, and a decision model based on multiple attributes is established. The mobile node selects the best next hop node to forward the data according to the multi-attributes decision model. The proposed algorithm evaluates various attributes and network conditions of mobile nodes comprehensively. Moreover, it can dynamically adjust the routing strategy in real time according to the load status of each node, and distribute the data in the network evenly. The simulation results show that compared with traditional multi-attribute routing algorithm, the algorithm not only has better convergence, but also finds a better path faster than the traditional multi-attribute algorithm when there is a preference for routing attributes. The proposed algorithm also has excellent performance in delivery ratio and cost, and improves the network performance.