Underactuated Autonomous Underwater Vehicles Research Articles

Mapless Motion Planning System for an Autonomous Underwater Vehicle Using Policy Gradient-based Deep Reinforcement Learning

This research is concerned with the motion planning problem encountered by underactuated autonomous underwater vehicles (AUVs) in a mapless environment. A motion planning system based on deep reinforcement learning is proposed. This system, which directly optimizes the policy, is an end-to-end motion planning system. It uses sensor information as input and continuous surge force and yaw moment as output. It can reach multiple target points in a sequence while simultaneously avoiding obstacles. In addition, this study proposes a reward curriculum training method to solve the problem in which the number of samples required for random exploration increases exponentially with the number of steps needed to obtain a reward. At the same time, the negative impact of intermediate rewards can be avoided. The proposed system demonstrates good planning ability for a mapless environment and excellent ability to migrate to other unknown environments. The system also has resistance to current disturbances. The simulation results show that the proposed mapless motion planning system can guide an underactuated AUV in navigating to its desired targets without colliding with any obstacles.

Velocity free leader-follower formation control for autonomous underwater vehicles with line-of-sight range and angle constraints

This paper investigates the problem of leader-follower formation control of underactuated autonomous underwater vehicles (AUVs), where the Line-of-sight (LOS) range and angle are constrained. An observer is involved to estimate the leader velocity in finite time with zero error, alleviating the need of communications between the AUVs. Based on the finite time observer, the formation controllers of the AUVs are designed using time-varying ln-type barrier Lyapunov function (BLF) method, which can guarantee that the resulting AUVs formation control system is uniformly ultimately bounded. To deal with the problem of explosion of differentiation terms in backstepping control, we introduce a command filter to realize the derivative of virtual variables. At the same time, a compensating signal is constructed (based on the auxiliary system technique) to compensate for the errors of the command filter. Simulation results have demonstrated the effectiveness of the proposed method.

Output-constrained tracking control of an underactuated autonomous underwater vehicle with uncertainties

This paper investigates the trajectory tracking control problem of an underactuated autonomous underwater vehicle in the present of output constraints and environmental disturbances. The designed controller can ensure that the output errors will never violate constraint limitations and solve the singularity problem in the design process by employing Barrier Lyapunov function. Backstepping algorithm, combined with radial basis function neural network for solving external disturbances and the Dynamic Surface Control technique for handling the calculation of explosion problem, is introduced to design the control laws. Then, a stability analysis based on Lyapunov directly method proves that all signals are bounded and tracking errors converge to a neighborhood of the origin. At last, simulation results illustrate the effectiveness and feasibility of the proposed controller.

Robust adaptive sliding mode control of underactuated autonomous underwater vehicles with uncertain dynamics

This paper focuses on adaptive integral sliding mode control for a class of underactuated autonomous underwater vehicles (AUVs) with uncertain dynamics, where the vehicles moving in three-dimensional (3-D) space have only three available control inputs provided by the stern propellers, steering and diving rudders but five degrees of freedom to be controlled. Different from the traditional sliding mode control, the proposed dual closed-loop integral sliding mode control design can be described as comprising two distinct phases: 1) in outer-loop, the virtual velocity commands are determined for the following work; 2) in inner-loop, the actual control inputs are designed to achieve the trajectory tracking. Moreover, the practical situations that there exist systematic parametric uncertainties and external disturbances are also considered, and a novel direct adaptive neural network controller combined with a conditional integrator is presented, which provides the robustness and adaptation for the vehicle. In addition, the rigorous stability analysis based Lyapunov’s method demonstrates the uniform ultimate boundedness of all the tracking errors in the closed-loop system. Finally, simulation results are shown the effectiveness of the proposed controllers.

Model predictive and adaptive neural sliding mode control for three-dimensional path following of autonomous underwater vehicle with input saturation

With model uncertainties and input saturation, a novel control method is developed to steer an underactuated autonomous underwater vehicle that realizes the following of the planned path in three-dimensional (3D) space. Firstly, Serret–Frenet frame is applied as virtual target, and the path following errors model in 3D is built. Secondly, the control method which includes kinematic controller and dynamic controller was presented based on cascade control strategy. The kinematic controller, which is responsible for generating a series of constrained velocity signals, is designed based on model predictive control. The adaptive radial basis function neural network is used to estimate the model uncertainty caused by hydrodynamic parameters. Moreover, sliding mode control technology is applied in the design of dynamic controller to improve its robustness. Then, the control effect is compared with that of LOS guidance law and PID controller by simulation experiment. The comparison results show that the proposed algorithm can improve path following effect and reduce input saturation.

Trajectory generation and tracking on SE(3) for an underactuated AUV with disturbances

This paper describes the trajectory tracking of an underactuated autonomous underwater vehicle (AUV) with three control inputs (surge, yaw and pitch moment) that operates in the presence of time-varying disturbances. The AUV kinematics are described in global coordinates as a Hamiltonian system on the Lie group SE(3) and the boundary-value problem arising from the geometric framing of Pontryagin’s Maximum Principle is applied to the vehicle kinematics. This 6-dimensional boundary value problem is solved using a numerical shooting method and a novel semi-analytical Lie group integrator. The integrator uses Rodrigue’s formula to express the exact solution of the solution curves, is symplectic and preserves energy and momentum. Inverse dynamics is applied to construct an inner-loop controller, which accounts for constraints on maximum torque and force via time reparametrization. This inner-loop control, which would drive the AUV along the reference trajectory in perfect conditions, is combined with a disturbance observer to construct an outer-loop controller, which ensures stability in the presence of bounded disturbances. Simulation results complete the work.

Spatial Path-Following Control of Underactuated AUV With Multiple Uncertainties and Input Saturation

This paper studies the problem of spatial path-following control of underactuated autonomous underwater vehicles (AUVs) with multiple uncertainties and input saturation taken into account. Initially, the reduced-order extended state observes (ESOs) are introduced to estimate and compensate all lumped uncertainties due to the model parameters perturbations, unmodeled dynamics, environmental disturbances, and nonlinear hydrodynamic damping terms. Furthermore, the spatial path-following control strategy is established by combining with backstepping, integral sliding mode control, and estimator to cope with the position, attitude, and linear velocity tracking of the AUV. Specifically, the dynamic surface control (DSC) technique is utilized to achieve satisfactory differential performance and avoid the “explosion of complexity.” Additionally, the auxiliary dynamic compensator is presented to analyze the effect of input saturation, and the states of the auxiliary design system are used to develop the controller. It is easily proved that the proposed control scheme can guarantee that all signals of the closed-loop system are globally stable through the Lyapunov theorem, with the tracking errors converging to an arbitrarily small neighborhood of zero. Finally, the numerical simulation results are carried out to illustrate the effectiveness of the proposed controller.

Trajectory Tracking Control of an Underactuated AUV Based on Backstepping Sliding Mode With State Prediction

This paper studies the trajectory tracking problem of an underactuated autonomous underwater vehicle (AUV) under the uncertainty of model parameters and input delay. Different from previous algorithms, a novel control algorithm is proposed which is a combination of RBF neural network algorithm and state prediction using backstepping sliding mode control method. The RBF neural network algorithm is used to estimate the composite interferences of model parameter uncertainties and external disturbances. Meanwhile, an appropriate virtual control law is designed for the horizontal trajectory tracking of the underactuated AUV by backstepping technique. Further, the longitudinal and heading control laws are proposed using the nonsingular fast terminal sliding mode method. Then, it is proved that the AUV velocity tracking error can converge to zero in a finite time by the Lyapunov theory. Finally, the effectiveness and robustness of the approach is illustrated by the simulation results.

Stabilization of a Relative Equilibrium for an Underactuated AUV With Disturbances Rejection

This paper addresses the robust stabilization of a relative equilibrium for an underactuated autonomous underwater vehicle (AUV) with disturbances rejection. The AUV’s dynamics is described by the Hamiltonian equation in the presence of unknown external disturbances and uncertain hydrodynamic damping. The disturbances arising from slowly varying currents are assumed to be constant, which consists of matched disturbances and unmatched disturbances, simultaneously. The unmatched disturbances and the uncertain damping that make the realization of the system stabilization become more challenging. To achieve the control objective, first, a new Hamiltonian equation, which is stable around the desired relative equilibrium, is designed to serve as the closed-loop dynamics in which an integral action is added to suppress the disturbances. By matching the closed-loop equation and the open-loop equation, an anti-disturbance stabilizing controller is presented, which requires the knowledge of the uncertain damping. Then, the adaptive control method is employed to propose an improved controller, which enhances the robustness of the uncertain damping. The stability is analyzed by the Lyapunov method. The numerical simulations are conducted to demonstrate the obtained result.

Soft-Switching Proximate Time Optimal Heading Control for Underactuated Autonomous Underwater Vehicle

In this paper, a soft-switching proximate time-optimal control (PTOC) is proposed based on the model-compensation extended state observer (ESO) and Takagi-Sugeno fuzzy switching, for the fast set-heading tracking of underactuated autonomous underwater vehicle (AUV). First, based on Pontryagin's maximum principle, a time-optimal control (TOC) law is derived for the first-order Nomoto model of the underactuated AUV. Then, a model-compensation active disturbance rejection control (ADRC) is developed; the outstanding characteristic is that the nonlinear heading dynamic model is compensated to a first-order Nomoto model rather than by a double integral system through implementing a model-compensation ESO (MC-ESO). The regular ESO is replaced with a reduced-order ESO (RESO) to reduce complexity, and the model-compensation RESO (MC-RESO) is designed by adding the known partial model to RESO. Based on the controller scaling method, a parameter self-tuning strategy is proposed for model-compensation ADRC (MC-ADRC) with changed plant parameters at different velocities. Finally, the soft-switching PTOC is developed for heading control, and the MC-RESO is adopted to estimate the unmeasured velocity and unknown total disturbances for feedback and compensation. The TOC with an unsaturated region (RTOC) is employed to enhance the robustness by using a switching region to replace the switching curve, and a soft-switching strategy between ADRC (near the origin) and RTOC (far from origin) is designed based on the Takagi-Sugeno fuzzy mode. Several simulations are carried out, the effectiveness of self-tuning MC-ADRC is verified, and the proposed soft-switching PTOC shows better performance compared with MC-ADRC.

Trajectory Tracking Control of Underactuated Autonomous Underwater Vehicle in the Presence of Ocean Currents

Trajectory Tracking Control of Underactuated Autonomous Underwater Vehicle in the Presence of Ocean Currents

Multi Pseudo Q-Learning-Based Deterministic Policy Gradient for Tracking Control of Autonomous Underwater Vehicles.

This paper investigates trajectory tracking problem for a class of underactuated autonomous underwater vehicles (AUVs) with unknown dynamics and constrained inputs. Different from existing policy gradient methods which employ single actor critic but cannot realize satisfactory tracking control accuracy and stable learning, our proposed algorithm can achieve high-level tracking control accuracy of AUVs and stable learning by applying a hybrid actors-critics architecture, where multiple actors and critics are trained to learn a deterministic policy and action-value function, respectively. Specifically, for the critics, the expected absolute Bellman error-based updating rule is used to choose the worst critic to be updated in each time step. Subsequently, to calculate the loss function with more accurate target value for the chosen critic, Pseudo Q-learning, which uses subgreedy policy to replace the greedy policy in Q-learning, is developed for continuous action spaces, and Multi Pseudo Q-learning (MPQ) is proposed to reduce the overestimation of action-value function and to stabilize the learning. As for the actors, deterministic policy gradient is applied to update the weights, and the final learned policy is defined as the average of all actors to avoid large but bad updates. Moreover, the stability analysis of the learning is given qualitatively. The effectiveness and generality of the proposed MPQ-based deterministic policy gradient (MPQ-DPG) algorithm are verified by the application on AUV with two different reference trajectories. In addition, the results demonstrate high-level tracking control accuracy and stable learning of MPQ-DPG. Besides, the results also validate that increasing the number of the actors and critics will further improve the performance.

Neural network-based target tracking control of underactuated autonomous underwater vehicles with a prescribed performance

In this paper, the target tracking control problem is addressed for underactuated autonomous underwater vehicles (AUV) with a prescribed performance. For this purpose, the range and bearing angles of the AUV relative to an underwater target are transformed to a second-order open-loop error dynamic model by using the prescribed performance bound technique. Then, a new tracking controller is proposed such that the tracking errors converge to an arbitrary small ultimate bound and their transient performance are guaranteed with a pre-specified maximum overshoot and the convergence rate. To overcome unmodeled dynamics and external disturbances that are imposed on the vehicle by the wind, waves, and ocean currents, a multi-layer neural network and an adaptive robust controller are adopted. A Lyapunov stability synthesis shows that all signals of the control system are bounded, and tracking errors converge to a small region containing the origin with a prescribed performance. Finally, simulations are performed in MATLAB software and a comparative study verifies the theoretical results.

Robust trajectory tracking control for an underactuated autonomous underwater vehicle based on bioinspired neurodynamics

This article investigates the three-dimensional trajectory tracking control problem for an underactuated autonomous underwater vehicle in the presence of parameter perturbations and external disturbances. An adaptive robust controller is proposed based on the velocity control strategy and adaptive integral sliding mode control algorithm. First, the desired velocities are developed using coordinate transformation and the backstepping method, which is the necessary velocities for autonomous underwater vehicle to track the time-varying desired trajectory. The bioinspired neurodynamics is used to smooth the desired velocities, which effectively avoids the jump problem of the velocity and simplifies the derivative calculation. Then, the dynamic control laws are designed based on the adaptive integral sliding mode control to drive the underactuated autonomous underwater vehicle to sail at the desired velocities. At the same time, the auxiliary control laws and the adaptive laws are introduced to eliminate the influence of parameter perturbations and external disturbances, respectively. The stability of the control system is guaranteed by the Lyapunov theorem, which shows that the system is asymptotically stable and all tracking errors are asymptotically convergent. At the end, numerical simulations are carried out to demonstrate the effectiveness and robustness of the proposed controller.

Three-dimensional trajectory tracking control of an underactuated autonomous underwater vehicle based on ocean current observer

This article presents a design method for the three-dimensional trajectory tracking control of an underactuated autonomous underwater vehicle with unknown current disturbances. To simplify the complexity of the controller and avoid the singular problem induced by initial state constraints, a novel nonlinear backstepping technique based on virtual control variables is employed to design the kinematics and dynamics controllers. The control law is developed by building virtual errors, which can solve the problem of differential explosion in the traditional backstepping. Specifically, an ocean current observer based on the kinematics model is proposed to estimate unknown current disturbances, where the estimation is integrated into the autonomous underwater vehicle kinematics and dynamics equations. The convergence of tracking errors and system stability are proven by using Lyapunov stable theory. Finally, the simulation studies were provided to illustrate the effectiveness and good performance of the above trajectory tracking strategy.

A Lateral Error S-Plane Control for Path Tracking on Autonomous Underwater Vehicle

This research aims at the motion control on a certain torpedo-type underactuated AUV (Autonomous Underwater Vehicle), using the S (Sigmoid) plane control method to build the heading controller and speed controller, and analyzes the stability of them, then establishes a linear path tracking control system in horizontal plane. In order to improve the performance of the path tracking controller under a large initial disturbance, an asymptotic controller of the heading error is proposed. And to solve the curve path tracking, a curve path tracking controller with a feedforward control idea is proposed. The simulation shows that these two improved controllers have a satisfactory control effects and effectively improved the performance of underwater robot motion control.

Underwater docking of an under-actuated autonomous underwater vehicle: system design and control implementation

Underwater docking greatly facilitates and extends operation of an autonomous underwater vehicle (AUV) without the support of a surface vessel. Robust and accurate control is critically important for docking an AUV into a small underwater funnel-type dock station. In this paper, a docking system with an under-actuated AUV is presented, with special attention paid to control algorithm design and implementation. For an under-actuated AUV, the cross-track error can be controlled only via vehicle heading modulation, so both the cross-track error and heading error have to be constrained to achieve successful docking operations, while the control problem can be even more complicated in practical scenarios with the presence of unknown ocean currents. To cope with the above issues, a control scheme of a three-hierarchy structure of control loops is developed, which has been embedded with online current estimator/compensator and effective control parameter tuning. The current estimator can evaluate both horizontal and vertical current velocity components, based only on the measurement of AUV’s velocity relative to the ground; in contrast, most existing methods use the measurements of both AUV’s velocities respectively relative to the ground and the water column. In addition to numerical simulation, the proposed docking scheme is fully implemented in a prototype AUV using MOOS-IvP architecture. Simulation results show that the current estimator/compensator works well even in the presence of lateral current disturbance. Finally, a series of sea trials are conducted to validate the current estimator/compensator and the whole docking system. The sea trial results show that our control methods can drive the AUV into the dock station effectively and robustly.

Global Adaptive Neural Network Control of Underactuated Autonomous Underwater Vehicles with Parametric Modeling Uncertainty

AbstractThis paper focuses on global adaptive neural network control for a class of underactuated autonomous underwater vehicles in the presence of possibly large modeling parametric uncertainty. As the control inputs cannot directly act in the sway and heave directions, two virtual velocities defined here, plus three actual control actions provided by the thrusters and rudders, are used to achieve the convergence of the system errors to around zero. Motivated by real‐time characteristics in the trajectory tracking, the proposed controller presents a significant advantage because it contains only one adaptive parameter to be updated online rather than the neural network weights. In addition, we also consider the practical situation that the velocities of the vehicle may experience sharp speed jumps when the position tracking errors initially change suddenly, which always results in thruster saturation. The biologically inspired model is introduced to smooth the virtual velocity commands such that the vehicle satisfies the control input and velocity constraints. Finally, comparison simulations are given to show the effectiveness of the proposed scheme.

Adaptive formation control of autonomous underwater vehicles with model uncertainties

SummaryThis paper investigates the formation control problem of underactuated autonomous underwater vehicles (AUVs). What makes this problem different and challenging is that, in the AUV model, the mass and damping matrices are not diagonal with nonzero off‐diagonal terms and unknown hydrodynamic damping terms. To address the effect of nonzero off‐diagonal terms, we introduce a coordinate transformation, while the unknown hydrodynamic damping terms are approximated adaptively by a neural network. Then, a leader‐follower control strategy is presented, where each follower needs to track a virtual vehicle. This strategy translates the formation control problem into a set of position (including heading) tracking problems for the following AUVs. The position and heading tracking controller is designed based on Lyapunov direct method and backstepping technique, taking into account the effect of the nonzero off‐diagonal terms in the mass matrix and the unknown hydrodynamic damping terms. Simulations are given to demonstrate the effectiveness of the control method proposed.

Control for Dynamic Positioning and Way-point Tracking of Underactuated Autonomous Underwater Vehicles Using Sliding Mode Control

Autonomous Underwater Vehicles (AUVs) are used in many applications such as the exploration of oceans, scientific and military missions, etc. Developing control schemes for AUVs is considered to be a very challenging task due to the complexity of the AUV model, the unmodeled dynamics, the uncertainties and the environmental disturbances. This paper develops a robust control scheme for the dynamic positioning and way-point tracking of underactuated autonomous underwater vehicles. In order to insure the robustness of the proposed controllers, the sliding mode control technique is adopted in the design process. Simulation results are given to validate the proposed controllers. Moreover, studies are presented to evaluate the robustness of the developed controllers with model uncertainties and under different types of disturbances including unknown currents.