Autonomous Underwater Vehicle System Research Articles

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

Abstract The autonomous navigation and obstacle avoidance capabilities of autonomous underwater vehicles (AUVs) are essential for ensuring their safe navigation and long-term, efficient operation. However, the complexity of the marine environment poses significant challenges to safe and effective obstacle avoidance. To address this issue, this study proposes an AUV obstacle avoidance control algorithm based on offline reinforcement learning. This method adopts the Conservative Q-learning (CQL) algorithm, which is based on the Soft Actor-Critic (SAC) framework. It learns from obtained historical obstacle avoidance data and ultimately achieves a favorable obstacle avoidance control strategy. In this method, PID and SAC control algorithms are utilized to generate expert obstacle avoidance data to construct a diversified offline database. Additionally, based on the line-of-sight (LOS) guidance method and artificial potential field (APF) method, information regarding the distance and orientation of targets and obstacles is incorporated into the state space, and heading and obstacle avoidance reward terms are integrated into the reward function design. The algorithm successfully guides the AUV in autonomous navigation and dynamic obstacle avoidance in three-dimensional space. Furthermore, the algorithm exhibits a certain degree of anti-interference capability against uncertain disturbances and ocean currents, enhancing the safety and robustness of the AUV system. Simulation results fully demonstrate the feasibility and effectiveness of the intelligent obstacle avoidance method based on offline reinforcement learning. This study highlights the profound significance of offline reinforcement learning in enabling robust and reliable control systems for AUVs, paving the way for enhanced operational capabilities in challenging marine environments.

Learning‐based tracking control of AUV: Mixed policy improvement and game‐based disturbance rejection

AbstractA mixed adaptive dynamic programming (ADP) scheme based on zero‐sum game theory is developed to address optimal control problems of autonomous underwater vehicle (AUV) systems subject to disturbances and safe constraints. By combining prior dynamic knowledge and actual sampled data, the proposed approach effectively mitigates the defect caused by the inaccurate dynamic model and significantly improves the training speed of the ADP algorithm. Initially, the dataset is enriched with sufficient reference data collected based on a nominal model without considering modelling bias. Also, the control object interacts with the real environment and continuously gathers adequate sampled data in the dataset. To comprehensively leverage the advantages of model‐based and model‐free methods during training, an adaptive tuning factor is introduced based on the dataset that possesses model‐referenced information and conforms to the distribution of the real‐world environment, which balances the influence of model‐based control law and data‐driven policy gradient on the direction of policy improvement. As a result, the proposed approach accelerates the learning speed compared to data‐driven methods, concurrently also enhancing the tracking performance in comparison to model‐based control methods. Moreover, the optimal control problem under disturbances is formulated as a zero‐sum game, and the actor‐critic‐disturbance framework is introduced to approximate the optimal control input, cost function, and disturbance policy, respectively. Furthermore, the convergence property of the proposed algorithm based on the value iteration method is analysed. Finally, an example of AUV path following based on the improved line‐of‐sight guidance is presented to demonstrate the effectiveness of the proposed method.

Robust H∞ Control for Autonomous Underwater Vehicle’s Time-Varying Delay Systems under Unknown Random Parameter Uncertainties and Cyber-Attacks

This paper investigates robust H∞-based control for autonomous underwater vehicle (AUV) systems under time-varying delay, model uncertainties, and cyber-attacks. Sensor and actuator cyber-attacks can cause faults in the overall AUV system. In addition, the behavior of the system can be affected by the presence of complexities, such as unknown random uncertainties that occur in system modeling. In this paper, the robustness against unpredictable random uncertainties is investigated by considering unknown but norm-bounded (UBB) random uncertainties. By constructing a proper Lyapunov–Krasovskii functional (LKF) and using linear matrix inequality (LMI) techniques, new stability criteria in the form of LMIs are derived such that the AUV system is stable. Moreover, this work is novel in addressing robust H∞ control, which considers time-varying delay, cyber-attacks, and randomly occurring uncertainties for AUV systems. Finally, the effectiveness of the proposed results is demonstrated through two examples and their computer simulations.

Research on Multiple AUVs Task Allocation with Energy Constraints in Underwater Search Environment

The allocation of tasks among multiple Autonomous Underwater Vehicles (AUVs) with energy constraints in underwater environments presents an NP-complete problem with far-reaching consequences for marine exploration, environmental monitoring, and underwater construction. This paper critically examines the contemporary methodologies and technologies in the task allocation for multiple AUVs, with a particular focus on strategies that optimize navigation time with energy consumption constraints. By conceptualizing the multiple AUVs task allocation issue as a Capacitated Vehicle Routing Problem (CVRP) and addressing it using the SCIP solver, this study seeks to identify effective task allocation strategies that enhance the operational efficiency and minimize the mission duration in energy-restricted underwater settings. The findings of this research provide valuable insights into efficient task allocation under energy constraints, providing useful theoretical implications and practical guidance for optimizing task planning and energy management in multiple AUVs systems. These contributions are demonstrated through the improved solution quality and computational efficiency.

A Novel Algorithm for Enhancing Terrain-Aided Navigation in Autonomous Underwater Vehicles

The position error in an inertial navigation system (INS) for autonomous underwater vehicles (AUVs) increases over time. Terrain-aided navigation can assist in correcting these INS position errors. To enhance the matching accuracy under large initial position errors, an improved terrain matching algorithm comprising terrain contour matching (TERCOM), particle swarm optimization (PSO), and iterative closest contour point (ICCP), named TERCOM-PSO-ICCP, is proposed. Initially, an enhanced TERCOM with an increased rotation angle is utilized to minimize heading errors and reduce the initial position error. The similarity extremum approach evaluates the initial matching outcomes, leading to an enhanced accuracy in the initial results. Next, artificial bee colony (ABC)-optimized PSO is employed for secondary matching to further reduce the initial position error and narrow the matching area. Finally, the ICCP, using the Mahalanobis distance as the objective function, is applied for the third matching, leveraging the ICCP’s fine search capabilities. The effective combination of these three algorithms significantly improves the terrain-aided navigation matching effect. Two tests show that the improved TERCOM-PSO-ICCP effectively reduces the matching error and corrects the position of the INS.

Consensus Control of Heterogeneous Uncertain Multiple Autonomous Underwater Vehicle Recovery Systems in Scenarios of Implicit Reduced Visibility

This paper investigates consensus control in heterogeneous and uncertain multiple autonomous underwater vehicle (AUV) systems under implicit reduced visibility conditions. We address challenges such as environmental uncertainties and system nonlinearity by utilizing a unified connectivity approach to model low-visibility interactions and heterogeneous multi-AUV dynamics. Our main contributions include developing a feedback linearization model for heterogeneous multi-AUV systems that accounts for uncertainties, introducing an adaptive consensus controller based on relative positioning that effectively manages implicit visual interaction limitations and validating our strategies through stability analysis and numerical simulations. Our simulations demonstrate approximately a 60% improvement in accuracy compared to previous algorithms, highlighting the practical value of our approach in AUV recovery operations. These advancements provide a robust solution for consensus control in complex underwater environments.

Adaptive fixed‐time fault‐tolerant fuzzy control of AUVs with asymmetric output constraints

AbstractThe fast and safe formation tracking problem for multiple autonomous underwater vehicle (AUV) systems (MAUVS) with asymmetric output constraints and actuator faults is studied in this article. Actuator faults are composed of loss of effectiveness and time‐varying unknown bias faults, an adaptive fixed‐time fault‐tolerant controller (AFFTC) is designed by employing fixed‐time stable theory and fuzzy logic systems. Under the designed control algorithm, the MAUVS can be practically fixed‐time stable, the tracking errors among the follower AUVs and the virtual leader AUV can converge to a small area near the origin within the fixed time, and the settling time is unaffected by the initial state of the system. To enhance the safety of MAUVS, a novel asymmetric barrier function is utilized to constrain the trajectory tracking errors within the prescribed range. Finally, the simulation results demonstrate the effectiveness of the proposed control algorithm.

Anti-disturbance cooperative formation containment control for multiple autonomous underwater vehicles with collision-free and actuator saturation constraints

This article investigates the formation containment control (FCC) problem of multiple autonomous underwater vehicles system (MAUVS) in the simultaneous presence of disturbances, collision avoidance between autonomous underwater vehicles (AUVs) and actuator saturation constraints. Firstly, for the purpose of enhancing the resisting disturbance capability of the MAUVS, the extended state observers are designed to estimate the total disturbance comprised of internal uncertainties and external interferences in real time. In order to hinder the occurrence of complexity explosion during the backstepping design, tracking differentiators are adopted to provide estimates for the derivative of the virtual control signals. Furthermore, the auxiliary system is constructed for reducing the influence of the actuator saturation on the MAUVS. Potential functions are integrated into the designed FCC strategy, so that the collisions between AUVs can be effectively avoided while the MAUVS achieving formation containment. Afterwards, via resorting to Lyapunov stability theory, it is demonstrated that the formation tracking errors and containment errors are uniformly ultimately bounded. Lastly, the comparative simulation results are illustrated to validate the feasibility of the presented anti-disturbance FCC scheme with collision-free and anti-saturation.

High-Power-Density Wireless Power Transfer System for Autonomous Underwater Vehicle Based on a Variable Ring-Shaped Magnetic Coupler

High-Power-Density Wireless Power Transfer System for Autonomous Underwater Vehicle Based on a Variable Ring-Shaped Magnetic Coupler

Research on Modeling Method of Autonomous Underwater Vehicle Based on a Physics-Informed Neural Network

Accurately modeling the system dynamics of autonomous underwater vehicles (AUVs) is imperative to facilitating the implementation of intelligent control. In this research, we introduce a physics-informed neural network (PINN) method to model the dynamics of AUVs by integrating dynamical equations with deep neural networks. This integration leverages the nonlinear expressive power of deep neural networks, alongside the robust foundation of physical prior knowledge, resulting in an AUV model proficient in long-term motion forecasting. The experimental results indicate that this method is capable of effectively extracting AUV system dynamics from datasets, exhibiting strong generalization capabilities and achieving robust long-term motion prediction. Furthermore, a model predictive control method is proposed, using the learned PINN as the predictive model to accurately track the closed-loop trajectory. This research offers novel perspectives on the dynamics modeling of AUVs and has the potential to be applied in other relevant research endeavors.

Research on model predictive control of autonomous underwater vehicle based on physics informed neural network modeling

In the rapidly evolving field of Autonomous Underwater Vehicles (AUVs), achieving precise control remains a critical endeavor. This study presents a pioneering integration of Model Predictive Control (MPC) with a Physics-Informed Neural Network (PINN), aiming to enhance control system precision and operational efficiency in AUVs. The efficacy of MPC lies in its adept handling of the intricate constraints and inherent nonlinear dynamics intrinsic to AUV systems. Concurrently, the PINN architecture incorporates the fundamental physical laws represented by Partial Differential Equations (PDEs), augmenting the predictive fidelity of the system. Firstly, this research implements the novel PINN-enhanced MPC framework for trajectory tracking and conducts a comparative evaluation against adaptive proportional-integral-derivative (PID) and Gaussian-process-based MPC controllers. This comparative analysis elucidates the advancements in control mechanisms attributable to the PINN integration. Furthermore, this study meticulously assesses the PINN-MPC's proficiency in navigating through static and dynamic obstacles within three-dimensional marine environments, a critical capability for AUV operations. Through extensive and meticulous simulations, the proposed approach demonstrates notable progress in overcoming environmental challenges and executing intricate operational tasks, such as obstacle avoidance, with heightened efficiency and dexterity. This research constitutes a substantial contribution to the theoretical advancement and elucidation of control systems in the AUV domain, bearing profound practical implications. It lays the foundation for the development of increasingly sophisticated, advanced, and reliable AUV missions, signifying a crucial advancement in the realms of underwater exploration and operational technology.

Finite-time fuzzy cooperative control for multi-AUV systems under cyber-attacks with hybrid unknown nonlinearities

In this paper, we investigate the fuzzy cooperative control and collision-free problems of discrete-time multiple autonomous underwater vehicles (AUVs) systems subject to unknown nonlinearities and constrained communication under cyber-attacks. A cooperative controller composed of three components is proposed for each AUV follower. The first component maneuvers the position and velocity for tracking desired signals by the feedback information. The repulsion part protects the AUV from collision with environmental obstacles or other vehicles. The remaining nonlinearities compensation term is employed to compensate for the system’s nonlinearities. Notably, the fuzzy logic systems (FLSs) approximate the hybrid unknown term constituted by external disturbances, unmodeled dynamics and deceptive signals. Then, the unknown terms identification and cooperative control problems are transformed into an error system stability problem. Inequality conditions for finite-time bounded error signals are presented through stability analysis. Also, it is theoretically proven that the cooperative control objective with collision avoidance can be accomplished by using suitable controller parameters. Lastly, the simulation results demonstrate the effectiveness of the complete control framework. Comparisons of various scenarios also affirm the significance of the FLSs and repulsion control for the multi-AUV cooperative control system.

Segmented hybrid event‐triggered control for underactuated autonomous underwater vehicles with an asymmetrical prescribed performance constraint

AbstractIn this technical article, the trajectory‐tracking problem of event‐based adaptive prescribed performance control for underactuated autonomous underwater vehicles (AUVs) is considered. The primary innovation of this article is the proposal of a segmented event‐triggered mechanism (SETM) that incorporates preselected convergence time. This mechanism allows for the regulation of communication frequency in the control channel (controller‐to‐actuator) according to the dynamic characteristics of the AUV system at different stages. In addition to presenting new design scheme based on SETM, an enhanced vision known as segmented hybrid event‐triggered mechanism (SHETM) is introduced. Notably, the minimum and maximum triggering intervals (MITI and MATI) for these two ETMs can be calculated from the corresponding resettable dynamic variables presented in the trigger conditions. Subsequently, by utilizing the proposed asymmetrical prescribed performance control (APPC) strategy, the system's output tracking error behaviors in both transient and steady‐state stages can be qualitatively predetermined through design parameters within the boundary function. Moreover, the minimum learning parameter (MLP) based RBFNN adaptive control law is developed to counteract the effects of model uncertainties and ocean current perturbations. Finally, rigorous theoretical analysis and simulation results confirm the viability of the proposed scheme.

Fuzzy neural network adaptive AUV control based on FTHGO

ABSTRACT A fuzzy radial basis function neural network (Fuzzy RBFNN) adaptive control scheme, based on fixed-time high gain state observer (FTHGO), is proposed to address the unpredictability of real-time state and composite interference in the trajectory tracking of the fully driven Autonomous Underwater Vehicle (AUV), ensuring fixed-time system convergence regardless of initial conditions. Firstly, a fixed-time backstepping controller is designed and a first-order fixed-time filter is introduced to tackle the differential explosion issue. Secondly, an FTHGO is developed to observe the real-time states of the AUV without assuming global known state signal. Then, the composite interference in the AUV system is effectively compensated by integrating the Fuzzy RBFNN technique. Finally, the fixed-time stability of the entire closed-loop system is proven utilising the Lyapunov stability theory. The effectiveness of the proposed algorithm is proved by the simulation experiment.

Imitation learning from imperfect demonstrations for AUV path tracking and obstacle avoidance

Autonomous underwater vehicle (AUV) is widely used for complex underwater tasks such as seafloor exploration. In recent years, deep reinforcement learning (DRL) has been introduced to the AUV control due to its capability to improve the autonomy of AUV. However, it is usually very difficult to design an effective reward function for the DRL methods. Generative adversarial imitation learning (GAIL) can allow AUVs to learn control policies from expert demonstrations instead of pre-defined reward functions, but suffers from the deficiency of requiring optimal expert demonstrations and not surpassing the provided demonstrations. This paper builds upon the GAIL algorithm for AUV learning control policies from expert demonstrations. We proposed an importance reweighting generative adversarial imitation learning (WGAIL) algorithm by using confidence scores to indicate the optimality of the demonstrated trajectories, which can facilitate AUVs to learn control policies from expert demonstrations of different levels. Our experimental results on a simulated AUV system modeling Sailfish 210 of our lab in the Gazebo simulation environment show that an AUV trained via WGAIL can achieve a better performance than the one trained via GAIL with different levels of expert sub-optimal demonstrations. Moreover, control policies trained via WGAIL in simple tasks can generalize better to complex tasks than those trained via GAIL, greatly extending the applicability of the AUV learning from expert demonstrations.

Design and extensibility analysis of a variable buoyancy system for small autonomous underwater vehicles

PurposeA switching depth controller based on a variable buoyancy system (VBS) is proposed to improve the performance of small autonomous underwater vehicles (AUVs). First, the requirements of VBS for small AUVs are analyzed. Second, a modular VBS with high extensibility and easy integration is proposed based on the concepts of generality and interchangeability. Subsequently, a depth-switching controller is proposed based on the modular VBS, which combines the best features of the linear active disturbance rejection controller and the nonlinear active disturbance rejection controller.Design/methodology/approachThe controller design and endurance of tiny AUVs are challenging because of their low environmental adaptation, limited energy resources and nonlinear dynamics. Traditional and single linear controllers cannot solve these problems efficiently. Although the VBS can improve the endurance of AUVs, the current VBS is not extensible for small AUVs in terms of the differences in individuals and operating environments.FindingsThe switching controller’s performance was examined using simulation with water flow and external disturbances, and the controller’s performance was compared in pool experiments. The results show that switching controllers have greater effectiveness, disturbance rejection capability and robustness even in the face of various disturbances.Practical implicationsA high degree of standardization and integration of VBS significantly enhances the performance of small AUVs. This will help expand the market for small AUV applications.Originality/valueThis solution improves the extensibility of the VBS, making it easier to integrate into different models of small AUVs. The device enhances the endurance and maneuverability of the small AUVs by adjusting buoyancy and center of gravity for low-power hovering and pitch angle control.

Robust model predictive control for perturbed nonlinear systems via an error differential-integral based event-triggered approach

This paper aims to establish a new event-triggered model predictive control (MPC) strategy for perturbed nonlinear systems working in a networked environment, which can effectively save the computation and communication resources while ensuring control performance. The core of the framework is a new event-triggered mechanism, which is built by combining the differential and integral (D-I) information of the error between the optimal state and the actual state in a pre-specified time horizon. Based on such a triggering mechanism, a D-I based event-triggered robust MPC algorithm is proposed to stabilise the system and reduce the computation/communication resource consumption. In addition, sufficient conditions to ensure the feasibility and stability of the proposed MPC algorithm and avoid Zeno behaviour are obtained through rigorous theoretical analysis. Finally, the proposed control framework is implemented in the nonlinear cart-damper-spring and autonomous underwater vehicle systems to verify its effectiveness.

Wavelet OFDM Wireless Communication System for Autonomous Underwater Vehicles Using Loop-Shaped Antennas in Underwater Environments

Wavelet OFDM Wireless Communication System for Autonomous Underwater Vehicles Using Loop-Shaped Antennas in Underwater Environments

Addressing Actuator Saturation during Fault Compensation in Model-Based Underwater Vehicle Control

Robust control systems are a necessity for autonomous underwater vehicle (AUV) systems due to the challenges they face during operation. Many AUV control-design methods have been developed for different actuator configurations, with robustness against model parameter uncertainties, environmental disturbances, and system faults. Actuator faults can reduce the physical capabilities of a system, which can be compensated for through control re-allocation. However, the increased control allocation to the remaining actuators may cause actuator saturation and reduce controller performance. In this work, we present a depth-pitch model-based nonlinear control law that directly considers actuator saturation, and a fault-tolerant control allocation method for a hybrid AUV actuator configuration. Two types of actuator faults are considered for an underwater vehicle with a hybrid actuator configuration. The proposed controller is implemented in a simulated system, and its trajectory tracking performance is compared with a baseline system without fault or saturation tolerance. To determine the utility of the proposed saturation and fault tolerance control methods, the tracking performance in these simulations is quantified in terms of the settling time, post-fault peak values, and root mean square of the depth and pitch errors.

Model-Based Digital Overall Integrated Design Method of AUVs

Autonomous underwater vehicles (AUVs) have the characteristics of a high performance, a complex coupling mechanism, a compact, complex system composition, as well as high requirements for design constraints, quality, and reliability. In the traditional overall design process, numerous design tools and software programs are used, which results in poor model data sharing, a lack of uniqueness and synchronization between system levels, and difficulty in process tracing. Moreover, it is challenging to meet the technical requirements for close collaboration and rapid iteration of multiple positions. To address the aforementioned limitations, this study proposes a digital overall integrated design method for the design and simulation integration of AUVs and defines a unified architecture and interface for system-level design simulation models, thus solving the interoperability and consistency problems in multiple tools and models. In addition, a model-based AUV system integration design verification method that combines different processes, specifications, and models is designed, and software similar to Cameo, which can provide technical means for system-level integrated design and achieve rapid modeling and simulation verification based on system design solutions, is developed. Finally, a practical system design is conducted by taking specific AUV equipment as a research object, and the proposed methods are compared with traditional methods to prove the improvement effect of the technical route on the equipment and development efficiency.