Autonomous Underwater Vehicle Path Research Articles

Learning‐based tracking control of AUV: Mixed policy improvement and game‐based disturbance rejection

AbstractA mixed adaptive dynamic programming (ADP) scheme based on zero‐sum game theory is developed to address optimal control problems of autonomous underwater vehicle (AUV) systems subject to disturbances and safe constraints. By combining prior dynamic knowledge and actual sampled data, the proposed approach effectively mitigates the defect caused by the inaccurate dynamic model and significantly improves the training speed of the ADP algorithm. Initially, the dataset is enriched with sufficient reference data collected based on a nominal model without considering modelling bias. Also, the control object interacts with the real environment and continuously gathers adequate sampled data in the dataset. To comprehensively leverage the advantages of model‐based and model‐free methods during training, an adaptive tuning factor is introduced based on the dataset that possesses model‐referenced information and conforms to the distribution of the real‐world environment, which balances the influence of model‐based control law and data‐driven policy gradient on the direction of policy improvement. As a result, the proposed approach accelerates the learning speed compared to data‐driven methods, concurrently also enhancing the tracking performance in comparison to model‐based control methods. Moreover, the optimal control problem under disturbances is formulated as a zero‐sum game, and the actor‐critic‐disturbance framework is introduced to approximate the optimal control input, cost function, and disturbance policy, respectively. Furthermore, the convergence property of the proposed algorithm based on the value iteration method is analysed. Finally, an example of AUV path following based on the improved line‐of‐sight guidance is presented to demonstrate the effectiveness of the proposed method.

Noisy Dueling Double Deep Q-Network algorithm for autonomous underwater vehicle path planning.

How to improve the success rate of autonomous underwater vehicle (AUV) path planning and reduce travel time as much as possible is a very challenging and crucial problem in the practical applications of AUV in the complex ocean current environment. Traditional reinforcement learning algorithms lack exploration of the environment, and the strategies learned by the agent may not generalize well to other different environments. To address these challenges, we propose a novel AUV path planning algorithm named the Noisy Dueling Double Deep Q-Network (ND3QN) algorithm by modifying the reward function and introducing a noisy network, which generalizes the traditional D3QN algorithm. Compared with the classical algorithm [e.g., Rapidly-exploring Random Trees Star (RRT*), DQN, and D3QN], with simulation experiments conducted in realistic terrain and ocean currents, the proposed ND3QN algorithm demonstrates the outstanding characteristics of a higher success rate of AUV path planning, shorter travel time, and smoother paths.

Research on Autonomous Underwater Vehicle Path Optimization Using a Field Theory-Guided A* Algorithm

Autonomous Underwater Vehicles (AUVs) have become indispensable tools in the fields of ocean exploration, resource exploitation, and environmental monitoring. Path planning and obstacle avoidance are crucial to improve the operational capabilities of AUVs. However, most algorithms focus only on macro-global or micro-local path planning and rarely address both problems simultaneously. This study extends the classical A* algorithm by integrating field theory principles. The resulting Field Theory Augmented A* (FT-A*) algorithm combines the constraints in the AUV’s dynamics and the threats posed by obstacles to ensure a safe navigation distance. The paths planned by the FT-A* algorithm were subsequently re-optimised in conjunction with Dubins curves, taking into account path smoothness and redundant node problems. Simulation experiments confirm that the improved algorithm can effectively help AUVs navigate safely around obstacles, which greatly improves navigation safety and increases the arithmetic power and navigation efficiency. The proposed FT-A* algorithm provides a robust solution for underwater path planning and demonstrates great practical value for AUV operation in complex marine environments.

A path planning method based on deep reinforcement learning for AUV in complex marine environment

The potential of autonomous underwater vehicle (AUV) on future applications is significant due to advancements in autonomy and intelligence. Path planning is a critical technology for AUV to perform operational missions in complex marine environments. To this end, this paper proposes a path planning method for AUV based on deep reinforcement learning. Initially, considering actual requirements, a complex marine environment model containing underwater terrain, sonobuoy detection, and ocean currents is established. Subsequently, the corresponding state space, action space, and reward function are formulated. Furthermore, to address the inherent limitations of existing deep reinforcement learning algorithms in terms of training efficiency, a mixed experience replay (MER) strategy is proposed. This strategy aims to enhance the efficiency of sample learning by integrating prior knowledge and exploration experience. Lastly, a novel HMER-SAC algorithm for AUV path planning is proposed by integrating the Soft Actor–Critic (SAC) algorithm with the hierarchical reinforcement learning strategy and the MER strategy. The results of the simulation and experiment demonstrate that the method is capable of efficiently planning executable paths in complex marine environments and exhibits superior training efficiency, stability, and performance.

Knowledge hierarchy-based dynamic multi-objective optimization method for AUV path planning in cooperative search missions

This study focuses on path planning for Autonomous Underwater Vehicles (AUVs) in underwater cooperative search missions. The complexities of the ocean environment, the uncertainty of target movements, and limited communication capabilities render underwater cooperative search missions dynamic multi-objective optimization challenges. Studies focusing on communication-constrained search strategies still lack reliable metrics for assessing underwater communication quality and do not consider communication dead zones. Addressing these deficiencies, this paper introduces a novel communication optimization strategy utilizing packet error rate as a metric for assessing communication efficacy, complemented by a potential field-based method for navigating out of communication dead zones. To tackle the inherently dynamic nature of cooperative search missions for mobile underwater targets, we propose a dynamic multi-objective optimization algorithm that employs a knowledge hierarchy strategy. This method enhances the NSGA-II algorithm's efficiency by extracting effective gene segments from historical Pareto solution set and generating a new initial population through recombination, hierarchy, and prediction. Distinct from other advanced dynamic multi-objective optimization approaches that are limited to theoretical problems, our approach is directly applicable to practical scenarios. The effectiveness and practicality of the proposed method are validated through a series of simulation experiments considering the impact of underwater acoustic communication. These results demonstrate that this research is not only innovative in theory but also holds significant engineering value and practical prospects in real-world applications.

A dual-stage coverage path planning method for bathymetric survey using an AUV in graph-based SLAM framework considering positioning uncertainty

Autonomous underwater vehicles (AUVs) are becoming increasingly popular for seabed coverage survey tasks but suffer from a serious problem of positioning error accumulation, which leads to low path tracking accuracy and a low coverage rate of the target area. A dual-stage AUV path planning (DSPP) approach was proposed within a simultaneous localization and mapping (SLAM) framework for coverage bathymetric surveys. In the first stage, a global coverage path is designed in a lawnmower style. Crossed paths are added to generate periodic loop closures in SLAM, which can improve the efficiency of online replanning. In the second stage, with a probabilistic neural network (PNN) used for feature-rich bathymetric sub-map identification, an entropy-based online path replanning approach was implemented using 20 templates to adjust the global path to produce accurate loop closures for positioning error correction. Two bathymetric datasets were used to test the proposed DSPP method using a self-developed experimental simulation system. Experiments verified that DSPP can significantly improve the positioning accuracy and coverage rate with acceptably longer paths. In a 6000 m× 5000 marea,the DSPP approach reduced the positioning error by 76% compared with the pure dead reckoning method, and the coverage ratio reached 98.15%.

Autonomous Underwater Vehicle Path Planning Based on Improved Salp Swarm Algorithm

Aiming at the problem of path planning for autonomous underwater vehicle (AUV) to cope with the influence of obstacles and eddies in complex marine environments, a path planning method based on an improved salp swarm algorithm (ISSA) is proposed. Firstly, the motion model of the AUV and eddy current model are constructed, including the relationship between position, velocity, attitude, and control inputs. Secondly, the improved SSA is proposed, which introduces the Levy flight strategy to enhance the algorithm’s optimization seeking ability and adds a nonlinear convergence factor to enhance the convergence ability of the algorithm. The stability and robustness of the improved algorithm are verified by test functions. Finally, the ISSA is applied to AUV path planning, which optimizes the AUV travel distance, improves the search efficiency and accuracy, and avoids the local optimum of the algorithm. The ISSA enhances the adaptive ability and robustness of the algorithm by introducing a dynamic adjustment strategy and feedback mechanism. Experimental verification is carried out using a simulated marine environment. The results show that the ISSA is better than the traditional algorithm in terms of path length as well as algorithm stability, and can effectively improve the navigation performance of AUV.

AUV path planning in a three-dimensional marine environment based on a novel multiple swarm co-evolutionary algorithm

Autonomous underwater vehicles (AUVs) are rapidly advancing in ocean exploration. High-performance path planning techniques are essential for AUVs. Path planning for AUVs is a multi-faceted challenge, necessitating careful consideration of safety, energy consumption and the influence of sea currents to ensure the development of a high-quality trajectory that satisfies all operational criteria. Existing path planning algorithms have problems such as incomplete consideration of influencing factors, computational complexity and weak applicability. Therefore, we propose a swarm intelligence optimisation algorithm based on multiple swarm co-evolution (MCO) to address the issue of AUV path planning in a three-dimensional marine environment. First, a three-dimensional marine environment model and the corresponding path evaluation mechanism are established. Second, the MCO rule is established to ensure that the MCO has balanced exploration and exploitation capabilities. A shared dynamic optimal particle between populations is introduced to ensure information exchange between populations. In addition, the cross-integration mutation strategy has been proposed for promoting the fusion of two populations of superior genes to ensure the inheritance of superior paternal genes to the offspring. Finally, four comparison experiments are designed, and the experiments compare eight commonly used intelligent search algorithms and improved versions. The results of the experiments proved that the MCO has excellent three-dimensional marine environment path planning capability, with robustness and search capabilities superior to other swarm intelligence optimisation algorithms.

Integrated Autonomous Underwater Vehicle Path Planning and Collision Avoidance Algorithms

Integrated Autonomous Underwater Vehicle Path Planning and Collision Avoidance Algorithms

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

In order to improve the autonomous navigation capability of autonomous underwater vehicles (AUV), this paper applies Deep Reinforcement Learning (DRL) to realize autonomous path planning. In response to the limited detection range and difficult tracking DRL policy in three-dimensional unknown environments, this paper proposes an improved deep reinforcement learning method. Firstly, performing AUV modeling and processing obstacle detection information. Second, under a framework of reinforcement learning with the control system, an improved environmental interaction method was designed by analyzing the motion characteristics and controller performance of the AUV. Finally, based on this method, a policy network that can generate paths that satisfy obstacle avoidance and navigation is trained in combination with a twin delayed deterministic policy gradient (TD3). The simulation results show that the proposed method can achieve AUV path planning in a three-dimensional unknown environment. The proposed framework can ensure that the planned path meets the dynamics constraints and is trackable. The improved algorithm can enhance the convergence of training and the attitude stability of AUV.

Imitation learning from imperfect demonstrations for AUV path tracking and obstacle avoidance

Autonomous underwater vehicle (AUV) is widely used for complex underwater tasks such as seafloor exploration. In recent years, deep reinforcement learning (DRL) has been introduced to the AUV control due to its capability to improve the autonomy of AUV. However, it is usually very difficult to design an effective reward function for the DRL methods. Generative adversarial imitation learning (GAIL) can allow AUVs to learn control policies from expert demonstrations instead of pre-defined reward functions, but suffers from the deficiency of requiring optimal expert demonstrations and not surpassing the provided demonstrations. This paper builds upon the GAIL algorithm for AUV learning control policies from expert demonstrations. We proposed an importance reweighting generative adversarial imitation learning (WGAIL) algorithm by using confidence scores to indicate the optimality of the demonstrated trajectories, which can facilitate AUVs to learn control policies from expert demonstrations of different levels. Our experimental results on a simulated AUV system modeling Sailfish 210 of our lab in the Gazebo simulation environment show that an AUV trained via WGAIL can achieve a better performance than the one trained via GAIL with different levels of expert sub-optimal demonstrations. Moreover, control policies trained via WGAIL in simple tasks can generalize better to complex tasks than those trained via GAIL, greatly extending the applicability of the AUV learning from expert demonstrations.

Oceanic Challenges to Technological Solutions: A Review of Autonomous Underwater Vehicle Path Technologies in Biomimicry, Control, Navigation, and Sensing

Oceanic Challenges to Technological Solutions: A Review of Autonomous Underwater Vehicle Path Technologies in Biomimicry, Control, Navigation, and Sensing

Technical Design and Data Analysis of Autonomous Underwater Vehicle-Based Side-Scan Sonar Operations

In response to the absence of standardized work practices, work safety measures, efficient work procedures, and suitable line planning methods for exploring seabed topography using autonomous underwater vehicles (AUVs) equipped with side-scan sonar systems, this paper proposes a design framework for side-scan sonar measurement technology based on AUVs. Detailed research has been conducted on line planning methods that differ significantly from those used in shipborne operations. The new line planning method was developed based on the performance characteristics of AUV equipment, the principles of motion modes, and shipborne side-scan sonar operations. The validity of the line planning method was verified through actual sea experiments using different types of AUVs in various terrains. The experimental results demonstrate that the new survey line planning method is distinct from existing AUV path planning methods and is highly applicable for conducting seabed topography exploration operations. Furthermore, the technical design scheme studied in this article is applicable in different sea areas and ensures the safety, operational efficiency, and data quality of AUVs. It holds significant importance for marine science engineering and practical applications. The article also features an analysis of successful data collected from AUVs equipped with side-scan sonar for conducting seabed topography exploration operations.

AUV Path Planning Considering Ocean Current Disturbance Based on Cloud Desktop Technology.

In the field of ocean energy detection, Autonomous Underwater Vehicles (AUVs) offer significant advantages in terms of manpower, resource, and energy efficiency. However, the unpredictable nature of the ocean environment, particularly the real-time changes in ocean currents, poses navigational risks for AUVs. Therefore, effective path planning in dynamic environments is crucial for AUVs to perform specific tasks. This paper addresses the static path planning problem and proposes a model called the noise net double DQN network with prioritized experience replay (N-DDQNP). The N-DDQNP model combines a noise network and a prioritized experience replay mechanism to address the limited exploration and slow convergence speed issues of the DQN algorithm, which are caused by the greedy strategy and uniform sampling mechanism. The proposed approach involves constructing a double DQN network with a priority experience replay and an exploration mechanism using the noise network. Second, a compound reward function is formulated to take into account ocean current, distance, and safety factors, ensuring prompt feedback during the training process. Regarding the ocean current, the reward function is designed based on the angle between the current direction and the AUV's heading direction, considering its impact on the AUV's speed. As for the distance factor, the reward is determined by the Euclidean distance between the current position and the target point. Furthermore, the safety factor considers whether the AUV may collide with obstacles. By incorporating these three factors, the compound reward function is established. To evaluate the performance of the N-DDQNP model, experiments were conducted using real ocean data in various complex ocean environments. The results demonstrate that the path planning time of the N-DDQNP model outperforms other algorithms in different ocean current scenarios and obstacle environments. Furthermore, a user console-AUV connection has been established using spice cloud desktop technology. The cloud desktop architecture enables intuitive observation of the AUV's navigation posture and the surrounding marine environment, facilitating safer and more efficient underwater exploration and marine resource detection tasks.

Energy-Efficient Multiple Autonomous Underwater Vehicle Path Planning Scheme in Underwater Sensor Networks

The issue of limited energy resources is crucial for underwater wireless sensor networks (UWSNs) because these networks operate in remote and harsh environments where access to power sources is limited. Overcoming the energy constraints is necessary to ensure the long-term functionality and sustainability of UWSN, enabling continuous data collection and communication for various applications such as environmental monitoring and surveillance. To solve the problems of limited energy and the difficulty of battery replacement in UWSN, a path planning and energy-saving scheme for charging underwater sensor nodes using AUVs (autonomous underwater vehicles) is proposed. Applying multiple AUVs to charge the sensing network nodes will maximize the size of the underwater sensing network as well as meet the transmission reliability, and the optimal path of AUVs is solved by using a genetic algorithm. Simulation results show that the AUV path planning scheme convergence is faster than that of conventional algorithms, and the lifetime of UWSN is prolonged while energy balancing according to the network size and node density. In high-density networks, the average energy consumption generated by AUVs for exploration is reduced by 15 percent for each additional AUV with our path planning.

Improved Artificial Potential Field Algorithm Assisted by Multisource Data for AUV Path Planning.

With the development of ocean exploration technology, the exploration of the ocean has become a hot research field involving the use of autonomous underwater vehicles (AUVs). In complex underwater environments, the fast, safe, and smooth arrival of target points is key for AUVs to conduct underwater exploration missions. Most path-planning algorithms combine deep reinforcement learning (DRL) and path-planning algorithms to achieve obstacle avoidance and path shortening. In this paper, we propose a method to improve the local minimum in the artificial potential field (APF) to make AUVs out of the local minimum by constructing a traction force. The improved artificial potential field (IAPF) method is combined with DRL for path planning while optimizing the reward function in the DRL algorithm and using the generated path to optimize the future path. By comparing our results with the experimental data of various algorithms, we found that the proposed method has positive effects and advantages in path planning. It is an efficient and safe path-planning method with obvious potential in underwater navigation devices.

A conflict-directed approach to chance-constrained mixed logical linear programming

Resistance to the adoption of autonomous systems comes in part from the perceived unreliability of the systems. The concerns can be addressed by deploying decision making algorithms that defines what it means to fail, and look for plans with the highest reward while limiting the probability of failure.This chance-constrained approach thus explicitly imposes a set of constraints that must be satisfied for success, and provides upper-bounds on the probability of violating such constraints. A chance-constrained mixed logical-linear program (CC-MLLP) is a natural formulation, allowing for the specification of linear and logical constraints, with probabilistic continuous variables. The formalism can be used to describe problems ranging from autonomous underwater vehicle path planning, to network routing under uncertainty.While naive encodings of CC-MLLPs can be solved with generalised solvers, the solution time may be unreasonable. In this work, we study architectures to speed up solutions by partitioning CC-MLLPs into the discrete and continuous portions. In order to provide faster solutions, we investigate methods for speeding up the solutions to the continuous chance-constrained linear programs. Further, by exploiting the new solution methods, we develop techniques for guiding the discrete decision making portion of the problem. The resulting algorithm achieves 10 times speed up over prior approaches on autonomous path planning benchmarks.

Improved AP-Clustering-Based AUV-Aided Data Collection Method for UWSNs

In underwater sensor networks, nodes are usually sparsely arranged, and autonomous underwater vehicles (AUVs) are often used to collect data by moving. However, it is a key problem for AUVs to shorten the path as much as possible to guarantee whole-network data acquisition. Aiming to solve this problem, this paper proposes an AUV underwater data collection method based on affinity propagation clustering. First, this clustering theory is used to select the nodes with large residual energy as cluster head nodes, and the clustering results are dynamically adjusted according to the communication radius of nodes to obtain the final clustering results. Finally, the cluster head nodes of the clusters are sequentially traversed using the traveling salesman problem, and an optimal path of AUV is planned to complete data collection for the whole network. A large number of simulation experiments verify that this method can effectively plan the mobile path of AUVs.

Data collection optimization of ocean observation network based on AUV path planning and communication

The Ocean Observation Network (OON) is an ocean science and engineering observation platform which integrates underwater wireless sensors, acoustic buoys, submersible buoys, and sonars. The complex underwater environment makes it challenging to place the observation nodes flexibly. In this research, firstly, a novel OON data collection optimization scheme is proposed, which is based on Autonomous Underwater Vehicle (AUV) path planning and communication. By combining the Voronoi diagram and K-means algorithm, an optimized clustering algorithm (VorK-means) is proposed to optimize the location of junction boxes and reduce the losses in the transmission energy of submarine cables in the OON simultaneously. Subsequently, a computational algorithm for AUV data collection position is proposed based on Voronoi vertices and communication circles. Moreover, an improved Grey Wolf Optimization (GWO) algorithm, based on the Levy flight and dimension (LD) search strategy, is proposed and applied for the AUV path planning, which is named LD-GWO. The accuracy and efficiency are validated by applying the proposed algorithms to the Canadian North-East Pacific Time-series Underwater Networked Experiment (NEPTUNE). Compared with the K-means algorithm, the VorK-means algorithm can reduce the total length of submarine cables by nearly 14%, and compared with the GWO algorithm, LD-GWO can obtain shorter data collection paths under identical parameter settings.

Adaptive SLAM Methodology Based on Simulated Annealing Particle Swarm Optimization for AUV Navigation

Simultaneous localization and mapping (SLAM) is crucial and challenging for autonomous underwater vehicle (AUV) autonomous navigation in complex and uncertain ocean environments. However, inaccurate time-varying observation noise parameters may lead to filtering divergence and poor mapping accuracy. In addition, particles are easily trapped in local extrema during the resampling, which may lead to inaccurate state estimation. In this paper, we propose an innovative simulated annealing particle swarm optimization-adaptive unscented FastSLAM (SAPSO-AUFastSLAM) algorithm. To cope with the unknown observation noise, the maximum a posteriori probability estimation algorithm is introduced into SLAM to recursively correct the measurement noise. Firstly, the Sage–Husa (SH) based unscented particle filter (UPF) algorithm is proposed to estimate time-varying measurement noise adaptively in AUV path estimation for improving filtering accuracy. Secondly, the SH-based unscented Kalman filter (UKF) algorithm is proposed to enhance mapping accuracy in feature estimation. Thirdly, SAPSO-based resampling is proposed to optimize posterior particles. The random judgment mechanism is used to update feasible solutions iteratively, which makes particles disengage local extreme values and achieve optimal global effects. The effectiveness and accuracy of the proposed algorithm are evaluated through simulation and sea trial data. The average AUV navigation accuracy of the presented SAPSO-AUFastSLAM method is improved by 18.0% compared to FastSLAM, 6.5% compared to UFastSLAM, and 5.9% compared to PSO-UFastSLAM.