Fast reinforcement learning algorithm for motion planning of nonholonomic autonomous underwater vehicle in disturbance

Abstract

This paper proposes a fast reinforcement learning (RL) algorithm for motion planning algorithm of nonholonomic autonomous underwater vehicle (AUV) in the strong water current with high reliability. The proposed algorithm can also be applied in an underwater environment with obstacles which are placed in arbitrary configuration. The algorithm has the ability to complete a learning process within a practical time limit. In order to accomplish a high learning speed, this paper proposes a hierarchical RL algorithm that copes with the curse of dimensionality, which comes from high complexity of dynamics of an AUV. The higher level of the algorithm refers only the position of the AUV, and learns a motion planning algorithm. The lower level of the algorithm refers to the velocity of the AUV and compensates for the undesirable nonMarkovian effects by stabilizing the dynamics motion of the AUV. The proposed hierarchical algorithm also uses a dynamics model described in the form of Bayesian-network. The dynamics model is very useful for increasing the efficiency of searching in learning process. The proposed algorithm is demonstrated by simulation. The result of the simulation shows high learning speed and performance of the proposed algorithm.