Robust Localization of Unmanned Surface Vehicle Using DDQN-AM

Abstract

A robust localization algorithm has been proposed for an Unmanned Surface Vehicle (USV) using DDQN-AM (Double Deep Q-Network with Action Memory). Inertial Navigation System (INS) and Global Positioning System (GPS) are used for collecting location information and for controlling the USV with DDQN-AM algorithm. Conventional probability-based filters such as Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF) are suitable for estimating behaviors of nonlinear systems and have been widely applied for localizing various kinds of nonlinear vehicles. However, the algorithms diverge or linearization position error increases in localizing USVs that are required to follow a desired trajectory accurately under high wave and wind conditions. Also, computational complexity is very high, which causes the instability of the vehicle system. To address these problems, DDQN with Adam optimization for error back propagation have been adopted in this research. However, if learning is performed using the training data set of the current time (current state, reward, action) in the replay buffer of DDQN, the learning performance (accuracy for states and running time) may deteriorate and learning load may occur. therefore, to improve the learning performance of DDQN, DDQN-AM has been newly proposed by adding AM in this research. To verify the effectiveness and to show the superior learning performance of the proposed DDQN-AM against the conventional approaches, a simulation using Matlab and field experiments with a prototype USV have been performed at the seashore. As a result, the position error of DDQN-AM was improved by 57.6% compared to the existing approach, and the average running time was also shortened by 1 minute, 30 seconds.