Dynamics Of Autonomous Underwater Vehicle Research Articles

Adaptive meta-reinforcement learning for AUVs 3D guidance and control under unknown ocean currents

Given the challenges posed by underwater communication constraints, Autonomous Underwater Vehicles (AUVs) must operate with enhanced autonomy while balancing operational safety and efficiency within complex marine settings. AUVs face a 3D guidance and control challenge, requiring complex adjustments to unknown ocean currents that affect their stability, maneuverability, and Guidance-Navigation-Control system efficiency. This study introduces and implements an off-policy meta-reinforcement learning framework to address AUVs’ 3D guidance and control challenges under unknown ocean currents. Our method, APE-SAC, utilizes an inference network to identify latent variables, streamlining the adaptation to unknown ocean currents by optimizing their strength and direction. Consequently, this framework redefines the issue of guidance and control under unknown current disturbances into a manageable learning task across multiple predefined scenarios. Utilizing the 3D dynamics of the REMUS AUV, we construct meta-Markov Decision Processes to detail the comprehensive state space, action space, and robust reward function tailored to the task’s specific demands. Furthermore, we develop a highly realistic simulation environment specifically designed for comprehensive experiments. Extensive results demonstrate that, compared to deep reinforcement learning baselines, APE-SAC exhibits outstanding performance in guiding and controlling AUVs under unknown ocean currents.

Research on Modeling Method of Autonomous Underwater Vehicle Based on a Physics-Informed Neural Network

Accurately modeling the system dynamics of autonomous underwater vehicles (AUVs) is imperative to facilitating the implementation of intelligent control. In this research, we introduce a physics-informed neural network (PINN) method to model the dynamics of AUVs by integrating dynamical equations with deep neural networks. This integration leverages the nonlinear expressive power of deep neural networks, alongside the robust foundation of physical prior knowledge, resulting in an AUV model proficient in long-term motion forecasting. The experimental results indicate that this method is capable of effectively extracting AUV system dynamics from datasets, exhibiting strong generalization capabilities and achieving robust long-term motion prediction. Furthermore, a model predictive control method is proposed, using the learned PINN as the predictive model to accurately track the closed-loop trajectory. This research offers novel perspectives on the dynamics modeling of AUVs and has the potential to be applied in other relevant research endeavors.

A Gaussian-Process-Based Model Predictive Control Approach for Trajectory Tracking and Obstacle Avoidance in Autonomous Underwater Vehicles

To achieve the efficient and precise control of autonomous underwater vehicles (AUVs) in dynamic ocean environments, this paper proposes an innovative Gaussian-Process-based Model Predictive Control (GP-MPC) method. This method combines the advantages of Gaussian process regression in modeling uncertainties in nonlinear systems, and MPC’s constraint optimization and real-time control abilities. To validate the effectiveness of the proposed GP-MPC method, its performance is first evaluated for trajectory tracking control tasks through numerical simulations based on a 6-degrees-of-freedom, fully actuated, AUV dynamics model. Subsequently, for 3D scenarios involving static and dynamic obstacles, an AUV horizontal plane decoupled motion model is constructed to verify the method’s obstacle avoidance capability. Extensive simulation studies demonstrate that the proposed GP-MPC method can effectively manage the nonlinear motion constraints faced by AUVs, significantly enhancing their intelligent obstacle avoidance performance in complex dynamic environments. By effectively handling model uncertainties and satisfying motion constraints, the GP-MPC method provides an innovative and efficient solution for the design of AUV control systems, substantially improving the control performance of AUVs.

Optimal tracking control of an Autonomous Underwater Vehicle: A PMP approach

This paper addresses the optimal tracking control problem of an Autonomous Underwater Vehicle (AUV) using Pontryagin’s Minimum Principle (PMP). The formulation uses a choice of Hamiltonian function using the desired control objective with AUV dynamics acting as dynamic constraints. We first develop PMP based on AUV kinematics and then extend it to the dynamics to arrive at optimal thrusts and moments. Necessary conditions for optimality are derived for both the models using PMP, which results in optimal trajectories that simultaneously minimize the tracking error as well as the control cost, thereby arriving at energy optimality. It was observed that the adjoint variables (costates) are indeed the momenta in the inertial and body-fixed frames. At the kinematic level, this forms a stable solution. The developed methodology is applied to both 2D and 3D AUV model with disturbances due to inputs and ocean currents. Numerical simulations are carried out with the derived control laws for a given trajectory tracking target. Quantitative evaluation of the performance and comparison of the controller is done using Mean Square Error(MSE) and Total Variation(TV) measures. The proposed control laws are found to achieve the desired control objectives.

Parameter identification and model prediction path following control of underactuated AUV: Methodology and experimental verification

This paper explores the integrated optimal control scheme from the identification of underactuated autonomous underwater vehicle (AUV) system dynamics model parameters to the application of Model Predictive Control (MPC) technology in underactuated AUV motion control, and presents the simulation and field test results. First, the AUV kinematics and dynamics models are introduced to represent the horizontal plane motion characteristics. Based on Computational Fluid Dynamics (CFD) numerical simulation and overset mesh technology, AUV plane motion is simulated to determine the unknown hydrodynamic coefficients in the model, and the accuracy of the mathematical model is validated by comparing the maneuverability simulation and field test results. After that, an adaptive line-of-sight (ALOS) guiding algorithm with exponentially variable forward distance is utilized in the guidance layer to reconcile the rapidity and stability of tracking. The path following problem is formulated as an MPC optimization problem that takes into account multiple realistic constraints and then converted into a standard convex quadratic programming structure, permitting for online real-time optimization. Simulation and field test results indicate that the control algorithm proposed in this paper provides a more effective control strategy than the traditional LOS+PID controller in tracking effect and robustness.

Adaptive Saturated Path Following Control of Underactuated AUV With Unmodeled Dynamics and Unknown Actuator Hysteresis

This article addresses the three-dimensional (3-D) path-following problem of underactuated autonomous underwater vehicles (AUVs) in the presence of multiple uncertainties and nonlinearities, including unmodeled dynamics, actuator hysteresis, input saturation, and oceanic disturbance. First, the AUV kinetic model is reconstructed as a second-order nonlinear system by coupling the smooth saturation function and hysteresis dynamics of actuators. Second, the 3-D tracking error is derived in a simplified manner based on the relative motion theory. Then the approach angle-based symmetrical hyperbolic-tangent line-of-sight (SHLOS) guidance and kinematic control scheme are designed to ensure the global asymptotic stability of the kinematic system. Third, two sets of kinetic control schemes are designed without prior knowledge of AUV dynamics and disturbance, where the Nussbaum gain technique is applied to deal with the unknown hysteresis parameter. The application of the reconstructed kinetic model avoids the degraded performance caused by actuator nonlinearities. The first scheme is improved by adaptively estimating the bound of the generalized disturbance-like term which lumps the unmodeled dynamics, external disturbance, and saturation approximation error. The improved scheme guarantees the global stability of the system and transient tracking performance. Finally, comparative simulation studies under dynamic oceanic environments prove the effectiveness and robustness of the proposed control schemes.

Overshoot Reduction Using Adaptive Neuro-Fuzzy Inference System for an Autonomous Underwater Vehicle

In this paper, an adaptive depth and heading control of an autonomous underwater vehicle using the concept of an adaptive neuro-fuzzy inference system (ANFIS) is designed. The autonomous underwater vehicle dynamics have six degrees of freedom, which are highly nonlinear and time-varying. It is affected by environmental effects such as ocean currents and tidal waves. Due to nonlinear dynamics designing, a stable controller in an autonomous underwater vehicle is a difficult end to achieve. Fuzzy logic and neural network control blocks make up the proposed control design to control the depth and heading angle of autonomous underwater vehicle. The neural network is trained using the back-propagation algorithm. In the presence of noise and parameter variation, the proposed adaptive controller’s performance is compared with that of the self-tuning fuzzy-PID and fuzzy logic controller. Simulations are conducted to obtain the performance of both controller models in terms of overshoot, and the rise time and the result of the proposed adaptive controller exhibit superior control performance and can eliminate the effect of uncertainty.

Time-Varying Constraint-Driven Optimal Task Execution for Multiple Autonomous Underwater Vehicles

This letter focuses on the constraint-driven optimal control for multiple Autonomous Underwater Vehicles (AUVs). The different tasks are formulated as multiple constraints on optimization. First, the task goals are formulated by the time-varying control barrier functions. Then, a novel cost-to-constraint transformation method is proposed for time-varying tasks to achieve the high-order constraint-driven task execution under AUV dynamics. The priority of each task is adjusted by the maximum allowed limit of the slack variable, thus relaxing the task constraints and avoiding potential constraint conflicts. Finally, the collision-free optimal energy-aware task execution is achieved by a minimum input norm objective that is incorporated into computationally efficient quadratic programming, which is scalable to different task behaviors. Comparative numerical simulations considering time-varying trajectory tracking and cooperative coverage tasks validate the effectiveness of the proposed method.

Reinforcement Learning for Autonomous Underwater Vehicles via Data-Informed Domain Randomization

Autonomous Underwater Vehicles (AUVs) or underwater vehicle-manipulator systems often have large model uncertainties from degenerated or damaged thrusters, varying payloads, disturbances from currents, etc. Other constraints, such as input dead zones and saturations, make the feedback controllers difficult to tune online. Model-free Reinforcement Learning (RL) has been applied to control AUVs, but most results were validated through numerical simulations. The trained controllers often perform unsatisfactorily on real AUVs; this is because the distributions of the AUV dynamics in numerical simulations and those of real AUVs are mismatched. This paper presents a model-free RL via Data-informed Domain Randomization (DDR) for controlling AUVs, where the mismatches between the trajectory data from numerical simulations and the real AUV were minimized by adjusting the parameters in the simulated AUVs. The DDR strategy extends the existing adaptive domain randomization technique by aggregating an input network to learn mappings between control signals across domains, enabling the controller to adapt to sudden changes in dynamics. The proposed RL via DDR was tested on the problems of AUV pose regulation through extensive numerical simulations and experiments in a lab tank with an underwater positioning system. These results have demonstrated the effectiveness of RL-DDR for transferring trained controllers to AUVs with different dynamics.

Markovian-Jump Reinforcement Learning for Autonomous Underwater Vehicles under Disturbances with Abrupt Changes

This paper studies the position regulation problems of an Autonomous Underwater Vehicle (AUV) subject to external disturbances that may have abrupt variations due to some events, e.g., water flow hitting nearby underwater structures. The disturbing forces may frequently exceed the actuator capacities, necessitating a constrained optimization of control inputs over a future time horizon. However, the AUV dynamics and the parameters of the disturbance models are unknown. Estimating the Markovian processes of the disturbances is challenging since it is entangled with uncertainties from AUV dynamics. As opposed to a single-Markovian description, this paper formulates the disturbed AUV as an unknown Markovian-Jump Linear System (MJLS) by augmenting the AUV state with the unknown disturbance state. Based on an observer network and an embedded solver, this paper proposes a reinforcement learning approach, Disturbance-Attenuation-net (MDA–net), for attenuating Markovian-jump disturbances and stabilizing the disturbed AUV. MDA–net is trained based on the sensitivity analysis of the optimality conditions and is able to estimate the disturbance and its transition dynamics based on observations of AUV states and control inputs online. Extensive numerical simulations of position regulation problems and preliminary experiments in a tank testbed have shown that the proposed MDA–net outperforms the existing DOB–net and a classical approach, Robust Integral of Sign of Error (RISE).

Domain-aware Control-oriented Neural Models for Autonomous Underwater Vehicles

Conventional physics-based modeling is a time-consuming bottleneck in control design for complex nonlinear systems like autonomous underwater vehicles (AUVs). In contrast, purely data-driven models require a large number of observations and lack operational guarantees for safety-critical systems. Data-driven models leveraging available partially characterized dynamics have potential to provide reliable systems models in a typical data-limited scenario for high value complex systems, thereby avoiding months of expensive expert modeling time. In this work we explore this middle-ground between expert-modeled and pure data-driven modeling. We present control-oriented parametric models with varying levels of domain-awareness that exploit known system structure and prior physics knowledge to create constrained deep neural dynamical system models. We employ universal differential equations to construct data-driven blackbox and graybox representations of the AUV dynamics. In addition, we explore a hybrid formulation that explicitly models the residual error related to imperfect graybox models. We compare the prediction performance of the learned models for different distributions of initial conditions and control inputs to assess their suitability for control.

Adaptive optimal trajectory tracking control of AUVs based on reinforcement learning

In this paper, an adaptive model-free optimal reinforcement learning (RL) neural network (NN) control scheme based on filter error is proposed for the trajectory tracking control problem of an autonomous underwater vehicle (AUV) with input saturation. Generally, the optimal control is realized by solving the Hamilton–Jacobi–Bellman (HJB) equation. However, due to its inherent nonlinearity and complexity, the HJB equation of AUV dynamics is challenging to solve. To deal with this problem, an RL strategy based on an actor–critic framework is proposed to approximate the solution of the HJB equation, where actor and critic NNs are used to perform control behavior and evaluate control performance, respectively. In addition, for the AUV system with the second-order strict-feedback dynamic model, the optimal controller design method based on filtering errors is proposed for the first time to simplify the controller design and accelerate the response speed of the system. Then, to solve the model-dependent problem, an extended state observer (ESO) is designed to estimate the unknown nonlinear dynamics, and an adaptive law is designed to estimate the unknown model parameters. To deal with the input saturation, an auxiliary variable system is utilized in the control law. The strict Lyapunov analysis guarantees that all signals of the system are semi-global uniformly ultimately bounded (SGUUB). Finally, the superiority of the proposed method is verified by comparative experiments.

Command filtered-based neuro-adaptive robust finite-time trajectory tracking control of autonomous underwater vehicles under stochastic perturbations

In this paper, the problem of finite-time trajectory tracking control is studied and addressed for a 6 degree of freedom (DOF) autonomous underwater vehicle (AUV) subjected to unknown dynamic model, stochastic perturbations, external disturbances (matched and mismatched) and saturation input nonlinearities. Based on the backstepping control approach, novel finite-time control inputs are designed and proposed. Artificial neural networks (ANNs) and finite-time adaptation laws are exploited to approximate the nonlinear dynamics of AUV, the stochastic perturbations and the upper bound of external disturbances. To handle the destructive effects of saturation input nonlinearities, finite-time auxiliary system method is utilized. To overcome the explosion of complexity problem of backstepping control strategy, compensator-based finite-time command filter approach is exploited. By utilizing the Lyapunov stability theorem, it is mathematically proven and demonstrated that the suggested nonlinear control inputs are able to guarantee the semi-global finite-time stability in probability (SGFSP) of the closed-loop AUV system. Finally, numerical simulations are carried out to illustrate and depict the effectiveness and performance of the proposed neuro-adaptive robust finite-time control scheme.

Electric-field-coupled oscillators for collective electrochemical perception in biohybrid robotics

This work explores the application of nonlinear oscillators coupled by an electric field in water, inspired by weakly electric fish. Such coupled oscillators operate in clear and colloidal (mud, bottom silt) water and represent a collective electrochemical sensor that is sensitive to global environmental parameters, the geometry of the common electric field and spatial dynamics of autonomous underwater vehicles (AUVs). Implemented in hardware and software, this approach can be used to create global awareness in a group of robots, which possess limited sensing and communication capabilities. Using oscillators from different AUVs enables extension of the range limitations related to the electric dipole of a single AUV. Applications of this technique are demonstrated for detecting the number of AUVs, distances between them, perception of dielectric objects and synchronization of behavior. Recognizing self-/nonself-generated signals by electric fish is re-embodied in a technological way through an ‘electrical mirror’ for discrimination between ‘collective self’ and ‘collective nonself’. These approaches have been implemented in several research projects with bioinspired/biohybrid systems in fresh and salt water, and electrochemical sensing in fluidic media.

Attention-Based Meta-Reinforcement Learning for Tracking Control of AUV With Time-Varying Dynamics.

Reinforcement learning (RL) is a promising technique for designing a model-free controller by interacting with the environment. Several researchers have applied RL to autonomous underwater vehicles (AUVs) for motion control, such as trajectory tracking. However, the existing RL-based controller usually assumes that the unknown AUV dynamics keep invariant during the operation period, limiting its further application in the complex underwater environment. In this article, a novel meta-RL-based control scheme is proposed for trajectory tracking control of AUV in the presence of unknown and time-varying dynamics. To this end, we divide the tracking task for AUV with time-varying dynamics into multiple specific tasks with fixed time-varying dynamics, to which we apply meta-RL for training to distill the general control policy. The obtained control policy can transfer to the testing phase with high adaptability. Inspired by the line-of-sight (LOS) tracking rule, we formulate each specific task as a Markov decision process (MDP) with a well-designed state and reward function. Furthermore, a novel policy network with an attention module is proposed to extract the hidden information of AUV dynamics. The simulation environment with time-varying dynamics is established, and the simulation results reveal the effectiveness of our proposed method.

Command-Filtered Robust Adaptive NN Control With the Prescribed Performance for the 3-D Trajectory Tracking of Underactuated AUVs.

A novel robust adaptive neural network (NN) control scheme with prescribed performance is developed for the 3-D trajectory tracking of underactuated autonomous underwater vehicles (AUVs) with uncertain dynamics and unknown disturbances using new prescribed performance functions, an additional term, the radial basis function (RBF) NN, and the command-filtered backstepping approach. Different from the traditional prescribed performance functions, the new prescribed performance functions are innovatively proposed such that the time desired for the trajectory tracking errors of AUVs to reach and stay within the prescribed error tolerance band can be preset exactly and flexibly. The additional term with the Nussbaum function is designed to deal with the underactuation problem of AUVs. By means of RBF NN, the uncertain item lumped by the uncertain dynamics of AUVs and unknown disturbances is eventually transformed into a linearly parametric form with only a single unknown parameter. The developed control scheme ensures that all signals in the AUV 3-D trajectory tracking closed-loop control system are bounded. Simulation results with comparisons show the validity and the superiority of our developed control scheme.

Distributed Path Tracking for Autonomous Underwater Vehicles Based on Pseudo Position Feedback

In this paper, we consider the distributed polynomial path tracking problem for a swarm of autonomous underwater vehicles (AUVs) modeled by second-order uncertain multi-agent systems. The application scenario of this paper has three distinguished characteristics. First, the communication network for the multi-agent system is unreliable and switching. Under the jointly connected condition, the communication network can be disconnected the entire time. Second, it is supposed that only the relative position between AUVs can be obtained for trajectory tracking control. Third, the AUV dynamics are subject to uncertain system parameters. By applying the cooperative output regulation control framework, a novel distributed robust control scheme is proposed to solve the distributed path tracking problem, which consists of three parts. First, to cope with communication network uncertainty, the distributed observer was invoked to recover the polynomial path for each AUV. Second, based on the relative position measurement between AUVs, a pseudo position estimator was adopted to generate the pseudo position for each AUV. Finally, based on the estimated polynomial path and the pseudo position, a certainty equivalent robust internal model control law was synthesized to achieve asymptotic reference trajectory tracking, where the internal model compensator aims to tackle uncertain system parameters. Numerical simulations are provided to validate the effectiveness of the proposed control scheme.

Analyzing the impact of disturbances on the AUV docking control system

We consider the problem of docking the autonomous underwater vehicle (AUV) to the seabed container. A two-level control system was previously developed using the mathematical model of the AUV dynamics. We further improve the mathematical model with the model of the AUV Computer Vision system, which is based on cameras and specialized markers installed on the container. In this paper, the studies are supplemented by considering the influence of two disturbance factors. The first one is the pixel coordinate error of the detected markers? Which is internal to the system. The second one is the external stationary ocean current. We study system performance in the vicinity of these factors and determine the acceptable docking conditions.

Robust Path-Following Control of Underactuated AUVs with Multiple Uncertainties in the Vertical Plane

The current study employs a novel nonlinear robust control approach for path-following control of underactuated autonomous underwater vehicles (AUVs) with multiple uncertainties in the vertical plane. Firstly, a nonlinear underactuated AUV model is established to characterize the dynamics of AUV and path-following error. To resolve dependence on a detailed model that appeared in previous studies, the unknown time-varying attack angular velocity in the dynamic model of the path-following error is considered as the kinematic uncertainty, while the linear superposition of the external environmental disturbances, the perturbations in the internal model parameters, and other unmodeled dynamics in the dynamic model is chosen as lumped dynamic uncertainties. Several reduced-order extended state observers (ESOs) are designed for estimating both of these uncertainties. Secondly, to reduce the impact of input saturation and avoid the “explosion of complexity” associated with traditional back-stepping method, a nonlinear track differentiator (NTD) is utilized to follow the virtual control signal and its derivative. Thirdly, the constructed reduced-order ESOs and NTD are adopted to establish an augmented back-stepping controller, where its ability to stabilize the overall system is demonstrated using the Lyapunov theorem. Finally, extensive simulations and analyses in various working conditions, including the nominal working condition without disturbances, the working condition with multiple uncertainties, and the conditions which better replicate the actual environment, are performed to demonstrate the effectiveness, superiority, and robustness of the designed controller.

Finite-time observer based tracking control of uncertain heterogeneous underwater vehicles using adaptive sliding mode approach

In this study, a finite-time consensus tracking problem is investigated for a group of autonomous underwater vehicles (AUVs) with heterogeneous uncertain dynamics. We firstly propose a two-layer distributed control strategy, which consists of an upper-layer distributed observer and a lower-layer controller, without using any global information. Based on Hölder’s inequality and the theory of finite-time stability, a distributed finite-time observer is developed for each follower to estimate the position information of a leader (i.e., an exosystem). Based on the sliding mode control method, a consensus tracking control scheme is designed for each AUV, by which all follower AUVs can track the leader in finite time. Secondly, when the parameters in the AUV dynamics are uncertain, a parameter-adaptive sliding mode control algorithm is introduced to improve the control performance. Finally, simulation results are presented to demonstrate the effectiveness of the proposed control algorithms.