Control Of Autonomous Underwater Vehicle Research Articles

Imitation learning from imperfect demonstrations for AUV path tracking and obstacle avoidance

Autonomous underwater vehicle (AUV) is widely used for complex underwater tasks such as seafloor exploration. In recent years, deep reinforcement learning (DRL) has been introduced to the AUV control due to its capability to improve the autonomy of AUV. However, it is usually very difficult to design an effective reward function for the DRL methods. Generative adversarial imitation learning (GAIL) can allow AUVs to learn control policies from expert demonstrations instead of pre-defined reward functions, but suffers from the deficiency of requiring optimal expert demonstrations and not surpassing the provided demonstrations. This paper builds upon the GAIL algorithm for AUV learning control policies from expert demonstrations. We proposed an importance reweighting generative adversarial imitation learning (WGAIL) algorithm by using confidence scores to indicate the optimality of the demonstrated trajectories, which can facilitate AUVs to learn control policies from expert demonstrations of different levels. Our experimental results on a simulated AUV system modeling Sailfish 210 of our lab in the Gazebo simulation environment show that an AUV trained via WGAIL can achieve a better performance than the one trained via GAIL with different levels of expert sub-optimal demonstrations. Moreover, control policies trained via WGAIL in simple tasks can generalize better to complex tasks than those trained via GAIL, greatly extending the applicability of the AUV learning from expert demonstrations.

Three‐dimensional path following control of underactuated autonomous underwater vehicles with nonzero roll dynamics: A novel line‐of‐sight‐guided approach

AbstractMost existing path following control approaches for underactuated Autonomous underwater vehicles (AUVs) assumed that the roll angle can be passively stabilized at zero, and simplified the problem to the 5‐degree‐of‐freedom model. However, in practice, the roll motion suffers from undesirable oscillations induced by environmental disturbances, unbalanced propeller torque, etc. To preserve the stability of the system under nonzero roll dynamics, this paper proposes a novel line‐of‐sight‐ (LOS) guided path following control approach. Firstly, the spatial path following error dynamics model is derived to comprehensively reveal the negative effect of roll dynamics. Secondly, an improved LOS guidance law is designed to compensate for the nonzero roll angle. A salient feature is that the proposed guidance law can be applied to the underwater vehicle no matter it is underactuated in the roll motion or not. Thirdly, by assigning the guidance angles to the desired attitude angles, a robust attitude tracking controller is developed, in which the adaptive estimation technique is employed to handle the lumped uncertainties. The tanh‐based saturation function is incorporated such that the resultant control signals are inherently bounded. The closed‐loop path following and attitude tracking errors are proved to be globally asymptotically stable at the origin. Comparative simulation results are provided to substantiate the effectiveness and superiority of the proposed method.

Iterative learning–based model-free adaptive precise heading following of an autonomous underwater vehicle with unknown disturbances

Due to the nonlinearity, strong coupling, and uncertain parameters of autonomous underwater vehicle (AUV), it is difficult to build an accurate dynamic model, which makes precise control of AUV extremely challenging. To handle the precise heading-following problem of AUV, this paper proposes an iterative learning-based redefine model-free adaptive heading control (IL-RMFAC) method for the underactuated AUV with unknown disturbances based on data driven. The control scheme consists of a learning control algorithm, a parameter iterative update algorithm, and a parameter reset algorithm. It is designed using only the input and output (I/O) data of the controlled system and is a data-driven control method. The pseudo partial derivative (PPD) can be iteratively calculated through the parameter iterative update algorithm and reset algorithm to adjust the learning gain, solving the problem of strictly limited initial position of the traditional fixed learning gain iterative learning control (ILC). A linear combination of angle and angular velocity is introduced in the kinematic layer to avoid overshooting of the expected following target, and an iterative learning method is introduced in the dynamics to improve the accuracy. As the number of iterations increases, the steady-state error is gradually decreased. Finally, by comparing traditional proportional–integral–derivative (PID) simulations, the proposed algorithm’s effectiveness and outstanding performance for the AUV heading tracking are confirmed.

UDE-based Nonlinear Path-following Control of Autonomous Underwater Vehicles With Multiple Uncertainties and Input Saturation

UDE-based Nonlinear Path-following Control of Autonomous Underwater Vehicles With Multiple Uncertainties and Input Saturation

Singularity Avoidance Fixed-Time Adaptive Neural Control for Autonomous Underwater Vehicles Considering Unmodelled Dynamics and Disturbances

Singularity Avoidance Fixed-Time Adaptive Neural Control for Autonomous Underwater Vehicles Considering Unmodelled Dynamics and Disturbances

Backstepping method tracking control for underactuated AUV with unknown dynamics based on action-critic networks based ADP

This work investigates trajectory-tacking control problem for underactuated autonomous underwater vehicles (AUV) with unknown dynamics. Due to the unknown dynamics, an action-critic networks based adaptive dynamic programming (ADP) scheme combined with backstepping approach is designed, which can achieve high-level system stability and tracking control accuracy. Firstly, the backstepping approach is introduced into the kinematic model of underactuated AUV and produces a virtual velocity control which is taken as the desired velocity input of the dynamic model of underactuated AUV. Secondly, the error tracking system is constructed according to the dynamic model of underactuated AUV. Thirdly, the critic neural network and the action neural network are employed to transform the trajectory-tracking control problem into optimal control problem based on policy iteration algorithm. At last simulation results are given to verify the effectiveness of the proposed control scheme.

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Bearing Rigidity-Based Flocking Control of AUVs via Semi-Supervised Incremental Broad Learning.

Flocking control of autonomous underwater vehicles (AUVs) has been regarded as the basis of many sophisticated marine coordination missions. However, there is still a research gap on the flocking of AUVs in weak communication and complex marine environment. This article attempts to fill up the above research gap from graph theory and intelligent learning perspectives. We first employ the bearing rigidity graph to describe the topology relationships of AUVs, through which an iterative gradient decent-based localization estimator is provided to obtain the position information. In order to improve the localization accuracy and energy efficiency, a min-weighted bearing rigidity graph generation strategy is developed. Along with this, we adopt the semi-supervised broad learning system (BLS) to design the model-free flocking controllers for AUVs in obstacle environment. The innovations of this article are summarized as follows: 1) the min-weighted bearing rigidity-based localization strategy can balance the localization accuracy and communication consumption as compared to the neighboring rule-based solutions and 2) the semi-supervised broad learning-based flocking controller can decrease the training time and solve the label limit over the supervised learning-based controllers. Finally, simulation and experimental studies are provided to verify the effectiveness.

Adaptive Formation Motion Planning and Control of Autonomous Underwater Vehicles Using Deep Reinforcement Learning

Adaptive Formation Motion Planning and Control of Autonomous Underwater Vehicles Using Deep Reinforcement Learning

Adaptive Dual-Channel Event-Triggered Fuzzy Control for Autonomous Underwater Vehicles With Multiple Obstacles Environment

Adaptive Dual-Channel Event-Triggered Fuzzy Control for Autonomous Underwater Vehicles With Multiple Obstacles Environment

Neural Adaptive Intermittent Output Feedback Control for Autonomous Underwater Vehicles With Full-State Quantitative Designs.

In this article, a neural adaptive intermittent output feedback control is investigated for autonomous underwater vehicles (AUVs) with full-state quantitative designs (FSQDs). To achieve the prespecified tracking performance determined by quantitative indices (e.g., overshoot, convergence time, steady-state accuracy, and maximum deviation) at both kinematic and kinetic levels, FSQDs are designed by transforming constrained AUV model into an unconstrained model via one-sided hyperbolic cosecant boundaries and nonlinear mapping functions. An intermittent sampling-based neural estimator (ISNE) is devised to reconstruct the matched and mismatched lumped disturbances as well as immeasurable velocity states of transformed AUV model, where only system outputs after intermittent sampling are required. Using the estimations of ISNE and the system outputs after triggering, an intermittent output feedback control law incorporated with hybrid threshold event-triggered mechanism (HTETM) is designed to achieve ultimately uniformly bounded (UUB) results. Simulation results are provided and analyzed to validate the effectiveness of the studied control strategy with application to an omnidirectional intelligent navigator (ODIN).

Self-Adjusting Prescribed Performance H∞ Trajectory Tracking Control for Underactuated Autonomous Underwater Vehicles

Self-Adjusting Prescribed Performance H∞ Trajectory Tracking Control for Underactuated Autonomous Underwater Vehicles

Intelligent Trajectory Tracking and Formation Control of Underactuated Autonomous Underwater Vehicles: A Critical Review

Intelligent Trajectory Tracking and Formation Control of Underactuated Autonomous Underwater Vehicles: A Critical Review

Adaptive Asymptotic Tracking Control for Underactuated Autonomous Underwater Vehicles With State Constraints

Adaptive Asymptotic Tracking Control for Underactuated Autonomous Underwater Vehicles With State Constraints

Tracking control of AUV via novel soft actor-critic and suboptimal demonstrations

Tracking control for autonomous underwater vehicles (AUVs) faces multifaceted challenges, making the acquisition of optimal demonstrations a daunting task. The suboptimal demonstrations mean less tracking accuracy. To address the issue of learning from suboptimal demonstrations, this paper proposes a model-free reinforcement learning (RL) method. Our approach utilizes suboptimal demonstrations to obtain an initial controller, which is iteratively refined during training. Given the suboptimal characteristics, demonstrations will be removed from the replay buffer upon reaching capacity. Building upon the soft actor-critic (SAC), our approach integrates a Recurrent Neural Network (RNN) into the policy network to capture the relationship between states and actions. Moreover, we introduce logarithmic and cosine functions to the reward function for enhancing the training effectiveness. Finally, we validate the effectiveness of the proposed Initialize Controller from Demonstrations (ICfD) algorithm through simulations with two reference trajectories. We provide a definition for tracking success. The success rates of ICfD in the two reference trajectories are 95.60% and 94.05%, respectively, surpassing the state-of-the-art RL method SACfD (80.03% 90.55%). The average one-step distance errors of ICfD are 1.20 m and 0.76 m, respectively, significantly lower than the S-plane controller (9.725 m 8.325 m). Besides, we evaluate the generalization of the ICfD controller in different scenarios.

Rotation matrix-based finite-time trajectory tracking control of AUV with output constraints and input quantization

This study investigated a rotation-matrix-based finite-time trajectory tracking problem for autonomous underwater vehicles (AUVs) in the presence of output constraints, input quantization, and uncertainties. First, a rotation matrix-based attitude representation is introduced, which allow attitude dynamics to be globally and uniquely represented without unwinding. To satisfy the finite-time stability of AUV tracking control and the output constraints imposed by introducing the new attitude error vector, a novel finite-time command-filtered backstepping controller (FTCFBC) is proposed based on the asymmetrical time-varying barrier Lyapunov function (TVBLF). Subsequently, a second-order auxiliary dynamic system (ADS) is proposed to estimate the negative effects that of input quantization errors. Because the quantized control inputs can be switched at an appropriate earlier or later switching timing depending on the output of the ADS, the negative impact of quantization errors on the control accuracy is reduced. Moreover, an adaptive finite-time disturbance observer (AFTDO) is developed to estimate the lumped uncertainties without prior information on the bounds of the uncertainties. Finally, the results of the theoretical analysis verified that the tracking errors can converge within a finite time. Numerical simulations confirmed the effectiveness of the proposed control scheme.

Adversarial deep reinforcement learning based robust depth tracking control for underactuated autonomous underwater vehicle

In this paper, an adversarial deep reinforcement learning-based control method is proposed to address the issue of robust depth tracking of an underactuated autonomous underwater vehicle in the presence of intrinsic coupled dynamics and external disturbances. First, long-short-term-memory neural network is presented to memorize and predict the changes in the state of vehicle, and a cascaded multilayer perception projects the output into action space of vehicle. Subsequently, adversarial deep reinforcement learning scheme is applied to the training of control agent by introducing an adversary which counteracts the control behavior, whereby the agent is enabled to learn the control strategy in different distributions of state transition. For evaluation of the performance, a control agent is pre-trained in simulation environment based on the reliable digital model of a real vehicle, and the simulation environment is paced at one iteration per second to align with real-time operations to ensure the portability of training result. Furthermore, the training cost is also extremely reduced. Finally, experiments are conducted with time-varying disturbances to further prove the feasibility of the proposed learning-based control scheme on a prototype of underwater vehicle in towing tank. Moreover, comparative experiment results show the better robustness performance of the learning-based control agent than that of classic line-of-sight based proportional–integral–derivative and adaptive line-of-sight based proportional–integral–derivative controllers in different scenarios.

Average consensus time-varying formation control of underactuated autonomous underwater vehicles based on event triggering decoupling mechanism

This paper focuses on the challenge of time-varying formation control for underactuated autonomous underwater vehicles (AUVs). To tackle this challenge, an average consensus time-varying formation control based on event triggering decoupling mechanism (ETDM-ACTVF) is proposed. Firstly, the average consensus time-varying formation (ACTVF) based on the affine transformation are introduced, and the control objective considering the dynamic characteristics of underactuated AUVs is proposed. The trajectory tracking controller is designed by the back stepping method. Secondly, an event triggering decoupling mechanism (ETDM) achieves the state decoupling to constrain the uncontrollable state. Moreover, a dynamic event triggering mechanism is employed to reduce the execution frequency of actuators. Finally, the excellent performance of the proposed algorithm in addressing both the underactuated AUV time-varying formation control and decoupling control problems is demonstrated through simulation results.

Robust Path-Following Control for AUV under Multiple Uncertainties and Input Saturation

In this paper, a robust path-following control strategy is proposed to deal with the path-following problem of the underactuated autonomous underwater vehicle (AUV) with multiple uncertainties and input saturation, and the effectiveness of the proposed control strategy is verified by semi-physical simulation experiments. Firstly, the control laws are constructed based on the traditional backstepping method; the multiple uncertainties are treated as lumped uncertainties, which can be estimated and eliminated by the employed extended state observers (ESOs). In addition, the influence of input saturation can be compensated by the designed auxiliary dynamic compensators. Secondly, to simplify controller design and address the “complexity explosion”, two command filters are used to obtain the estimated value of the unknown sideslip angular velocity and the desired yaw angular acceleration, respectively. Finally, the superiority and robustness of the proposed control strategy are verified through computer simulation. A semi-physical simulation experiment platform is built based on the NI Compact cRIO-9068 and PLC S7-1200 to further demonstrate the effectiveness of the proposed control strategy.

Adaptive nonsingular fast terminal sliding mode control for the tracking control of underactuated autonomous underwater vehicles

This article proposes an adaptive nonsingular fast terminal sliding mode control scheme with piecewise fast multi-power reaching law for tracking control of underactuated autonomous underwater vehicles under model uncertainties, ocean disturbances, and measurement noise. This control approach enhances the robustness and guarantees faster convergence of state error to zero in finite time while reducing the chattering effect. Utilizing the benefit of adaption law prevents overestimating control parameters, and it eliminates the need for the upper bound value of disturbances. The overall stability of the system is analyzed using the Lyapunov criterion. The results of the proposed approach are compared with adaptive nonsingular terminal sliding mode control and adaptive sliding mode control. The performance of the proposed control approach is evaluated by using the performance indices root mean square error and chattering indicator. The simulation results confirm the efficiency of the proposed approach.