Improved value iteration network for path planning

Abstract

In this study, a series of improved path planning algorithms are designed for path planning tasks in autonomous control based on deep reinforcement learning. The Value Iteration Network (VIN) is used to deal with the path planning problem. Origin VIN performs well on small size maps, but when it comes to a bigger size of map on test set, the success rate decreased. In order to solve the problem that origin VIN lacks long-distance multi-step planning ability on large maps and generalization ability is insufficient, a three-step improvement was made. First of all, in view of the inconvenient data flow and the disappearance of gradients caused by the network being too deep, the jump connection structure is used to obtain the deeper VIN, in which the accuracy of the experiment is improved. Secondly, with the purpose of solving the problem that the complexity of the model is greatly increased due to the deepening of the network, Batch normalization is used to obtain a new network with dueling architecture plus batch normalization layer, which further accelerates the convergence speed of the network. Third, to deal with the global path planning problem on the big map, the hierarchical network structure is adopted for hierarchical value iteration, and the Hierarchical Structure VIN is obtained. In Hierarchical Structure VIN, the long-term planning ability and generalization ability of the algorithm have been significantly improved, and the algorithm could figure out the large-scale and complex path planning problem.