Autonomous Underwater Vehicle Path Research Articles

A path planning method based on deep reinforcement learning for AUV in complex marine environment

The potential of autonomous underwater vehicle (AUV) on future applications is significant due to advancements in autonomy and intelligence. Path planning is a critical technology for AUV to perform operational missions in complex marine environments. To this end, this paper proposes a path planning method for AUV based on deep reinforcement learning. Initially, considering actual requirements, a complex marine environment model containing underwater terrain, sonobuoy detection, and ocean currents is established. Subsequently, the corresponding state space, action space, and reward function are formulated. Furthermore, to address the inherent limitations of existing deep reinforcement learning algorithms in terms of training efficiency, a mixed experience replay (MER) strategy is proposed. This strategy aims to enhance the efficiency of sample learning by integrating prior knowledge and exploration experience. Lastly, a novel HMER-SAC algorithm for AUV path planning is proposed by integrating the Soft Actor–Critic (SAC) algorithm with the hierarchical reinforcement learning strategy and the MER strategy. The results of the simulation and experiment demonstrate that the method is capable of efficiently planning executable paths in complex marine environments and exhibits superior training efficiency, stability, and performance.

Knowledge hierarchy-based dynamic multi-objective optimization method for AUV path planning in cooperative search missions

This study focuses on path planning for Autonomous Underwater Vehicles (AUVs) in underwater cooperative search missions. The complexities of the ocean environment, the uncertainty of target movements, and limited communication capabilities render underwater cooperative search missions dynamic multi-objective optimization challenges. Studies focusing on communication-constrained search strategies still lack reliable metrics for assessing underwater communication quality and do not consider communication dead zones. Addressing these deficiencies, this paper introduces a novel communication optimization strategy utilizing packet error rate as a metric for assessing communication efficacy, complemented by a potential field-based method for navigating out of communication dead zones. To tackle the inherently dynamic nature of cooperative search missions for mobile underwater targets, we propose a dynamic multi-objective optimization algorithm that employs a knowledge hierarchy strategy. This method enhances the NSGA-II algorithm's efficiency by extracting effective gene segments from historical Pareto solution set and generating a new initial population through recombination, hierarchy, and prediction. Distinct from other advanced dynamic multi-objective optimization approaches that are limited to theoretical problems, our approach is directly applicable to practical scenarios. The effectiveness and practicality of the proposed method are validated through a series of simulation experiments considering the impact of underwater acoustic communication. These results demonstrate that this research is not only innovative in theory but also holds significant engineering value and practical prospects in real-world applications.

A dual-stage coverage path planning method for bathymetric survey using an AUV in graph-based SLAM framework considering positioning uncertainty

Autonomous underwater vehicles (AUVs) are becoming increasingly popular for seabed coverage survey tasks but suffer from a serious problem of positioning error accumulation, which leads to low path tracking accuracy and a low coverage rate of the target area. A dual-stage AUV path planning (DSPP) approach was proposed within a simultaneous localization and mapping (SLAM) framework for coverage bathymetric surveys. In the first stage, a global coverage path is designed in a lawnmower style. Crossed paths are added to generate periodic loop closures in SLAM, which can improve the efficiency of online replanning. In the second stage, with a probabilistic neural network (PNN) used for feature-rich bathymetric sub-map identification, an entropy-based online path replanning approach was implemented using 20 templates to adjust the global path to produce accurate loop closures for positioning error correction. Two bathymetric datasets were used to test the proposed DSPP method using a self-developed experimental simulation system. Experiments verified that DSPP can significantly improve the positioning accuracy and coverage rate with acceptably longer paths. In a 6000 m× 5000 marea,the DSPP approach reduced the positioning error by 76% compared with the pure dead reckoning method, and the coverage ratio reached 98.15%.

Autonomous Underwater Vehicle Path Planning Based on Improved Salp Swarm Algorithm

Aiming at the problem of path planning for autonomous underwater vehicle (AUV) to cope with the influence of obstacles and eddies in complex marine environments, a path planning method based on an improved salp swarm algorithm (ISSA) is proposed. Firstly, the motion model of the AUV and eddy current model are constructed, including the relationship between position, velocity, attitude, and control inputs. Secondly, the improved SSA is proposed, which introduces the Levy flight strategy to enhance the algorithm’s optimization seeking ability and adds a nonlinear convergence factor to enhance the convergence ability of the algorithm. The stability and robustness of the improved algorithm are verified by test functions. Finally, the ISSA is applied to AUV path planning, which optimizes the AUV travel distance, improves the search efficiency and accuracy, and avoids the local optimum of the algorithm. The ISSA enhances the adaptive ability and robustness of the algorithm by introducing a dynamic adjustment strategy and feedback mechanism. Experimental verification is carried out using a simulated marine environment. The results show that the ISSA is better than the traditional algorithm in terms of path length as well as algorithm stability, and can effectively improve the navigation performance of AUV.

AUV path planning in a three-dimensional marine environment based on a novel multiple swarm co-evolutionary algorithm

Autonomous underwater vehicles (AUVs) are rapidly advancing in ocean exploration. High-performance path planning techniques are essential for AUVs. Path planning for AUVs is a multi-faceted challenge, necessitating careful consideration of safety, energy consumption and the influence of sea currents to ensure the development of a high-quality trajectory that satisfies all operational criteria. Existing path planning algorithms have problems such as incomplete consideration of influencing factors, computational complexity and weak applicability. Therefore, we propose a swarm intelligence optimisation algorithm based on multiple swarm co-evolution (MCO) to address the issue of AUV path planning in a three-dimensional marine environment. First, a three-dimensional marine environment model and the corresponding path evaluation mechanism are established. Second, the MCO rule is established to ensure that the MCO has balanced exploration and exploitation capabilities. A shared dynamic optimal particle between populations is introduced to ensure information exchange between populations. In addition, the cross-integration mutation strategy has been proposed for promoting the fusion of two populations of superior genes to ensure the inheritance of superior paternal genes to the offspring. Finally, four comparison experiments are designed, and the experiments compare eight commonly used intelligent search algorithms and improved versions. The results of the experiments proved that the MCO has excellent three-dimensional marine environment path planning capability, with robustness and search capabilities superior to other swarm intelligence optimisation algorithms.

Integrated Autonomous Underwater Vehicle Path Planning and Collision Avoidance Algorithms

Integrated Autonomous Underwater Vehicle Path Planning and Collision Avoidance Algorithms

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

In order to improve the autonomous navigation capability of autonomous underwater vehicles (AUV), this paper applies Deep Reinforcement Learning (DRL) to realize autonomous path planning. In response to the limited detection range and difficult tracking DRL policy in three-dimensional unknown environments, this paper proposes an improved deep reinforcement learning method. Firstly, performing AUV modeling and processing obstacle detection information. Second, under a framework of reinforcement learning with the control system, an improved environmental interaction method was designed by analyzing the motion characteristics and controller performance of the AUV. Finally, based on this method, a policy network that can generate paths that satisfy obstacle avoidance and navigation is trained in combination with a twin delayed deterministic policy gradient (TD3). The simulation results show that the proposed method can achieve AUV path planning in a three-dimensional unknown environment. The proposed framework can ensure that the planned path meets the dynamics constraints and is trackable. The improved algorithm can enhance the convergence of training and the attitude stability of AUV.

Imitation learning from imperfect demonstrations for AUV path tracking and obstacle avoidance

Autonomous underwater vehicle (AUV) is widely used for complex underwater tasks such as seafloor exploration. In recent years, deep reinforcement learning (DRL) has been introduced to the AUV control due to its capability to improve the autonomy of AUV. However, it is usually very difficult to design an effective reward function for the DRL methods. Generative adversarial imitation learning (GAIL) can allow AUVs to learn control policies from expert demonstrations instead of pre-defined reward functions, but suffers from the deficiency of requiring optimal expert demonstrations and not surpassing the provided demonstrations. This paper builds upon the GAIL algorithm for AUV learning control policies from expert demonstrations. We proposed an importance reweighting generative adversarial imitation learning (WGAIL) algorithm by using confidence scores to indicate the optimality of the demonstrated trajectories, which can facilitate AUVs to learn control policies from expert demonstrations of different levels. Our experimental results on a simulated AUV system modeling Sailfish 210 of our lab in the Gazebo simulation environment show that an AUV trained via WGAIL can achieve a better performance than the one trained via GAIL with different levels of expert sub-optimal demonstrations. Moreover, control policies trained via WGAIL in simple tasks can generalize better to complex tasks than those trained via GAIL, greatly extending the applicability of the AUV learning from expert demonstrations.

Oceanic Challenges to Technological Solutions: A Review of Autonomous Underwater Vehicle Path Technologies in Biomimicry, Control, Navigation, and Sensing

Oceanic Challenges to Technological Solutions: A Review of Autonomous Underwater Vehicle Path Technologies in Biomimicry, Control, Navigation, and Sensing

Technical Design and Data Analysis of Autonomous Underwater Vehicle-Based Side-Scan Sonar Operations

In response to the absence of standardized work practices, work safety measures, efficient work procedures, and suitable line planning methods for exploring seabed topography using autonomous underwater vehicles (AUVs) equipped with side-scan sonar systems, this paper proposes a design framework for side-scan sonar measurement technology based on AUVs. Detailed research has been conducted on line planning methods that differ significantly from those used in shipborne operations. The new line planning method was developed based on the performance characteristics of AUV equipment, the principles of motion modes, and shipborne side-scan sonar operations. The validity of the line planning method was verified through actual sea experiments using different types of AUVs in various terrains. The experimental results demonstrate that the new survey line planning method is distinct from existing AUV path planning methods and is highly applicable for conducting seabed topography exploration operations. Furthermore, the technical design scheme studied in this article is applicable in different sea areas and ensures the safety, operational efficiency, and data quality of AUVs. It holds significant importance for marine science engineering and practical applications. The article also features an analysis of successful data collected from AUVs equipped with side-scan sonar for conducting seabed topography exploration operations.

AUV Path Planning Considering Ocean Current Disturbance Based on Cloud Desktop Technology.

In the field of ocean energy detection, Autonomous Underwater Vehicles (AUVs) offer significant advantages in terms of manpower, resource, and energy efficiency. However, the unpredictable nature of the ocean environment, particularly the real-time changes in ocean currents, poses navigational risks for AUVs. Therefore, effective path planning in dynamic environments is crucial for AUVs to perform specific tasks. This paper addresses the static path planning problem and proposes a model called the noise net double DQN network with prioritized experience replay (N-DDQNP). The N-DDQNP model combines a noise network and a prioritized experience replay mechanism to address the limited exploration and slow convergence speed issues of the DQN algorithm, which are caused by the greedy strategy and uniform sampling mechanism. The proposed approach involves constructing a double DQN network with a priority experience replay and an exploration mechanism using the noise network. Second, a compound reward function is formulated to take into account ocean current, distance, and safety factors, ensuring prompt feedback during the training process. Regarding the ocean current, the reward function is designed based on the angle between the current direction and the AUV's heading direction, considering its impact on the AUV's speed. As for the distance factor, the reward is determined by the Euclidean distance between the current position and the target point. Furthermore, the safety factor considers whether the AUV may collide with obstacles. By incorporating these three factors, the compound reward function is established. To evaluate the performance of the N-DDQNP model, experiments were conducted using real ocean data in various complex ocean environments. The results demonstrate that the path planning time of the N-DDQNP model outperforms other algorithms in different ocean current scenarios and obstacle environments. Furthermore, a user console-AUV connection has been established using spice cloud desktop technology. The cloud desktop architecture enables intuitive observation of the AUV's navigation posture and the surrounding marine environment, facilitating safer and more efficient underwater exploration and marine resource detection tasks.

Energy-Efficient Multiple Autonomous Underwater Vehicle Path Planning Scheme in Underwater Sensor Networks

The issue of limited energy resources is crucial for underwater wireless sensor networks (UWSNs) because these networks operate in remote and harsh environments where access to power sources is limited. Overcoming the energy constraints is necessary to ensure the long-term functionality and sustainability of UWSN, enabling continuous data collection and communication for various applications such as environmental monitoring and surveillance. To solve the problems of limited energy and the difficulty of battery replacement in UWSN, a path planning and energy-saving scheme for charging underwater sensor nodes using AUVs (autonomous underwater vehicles) is proposed. Applying multiple AUVs to charge the sensing network nodes will maximize the size of the underwater sensing network as well as meet the transmission reliability, and the optimal path of AUVs is solved by using a genetic algorithm. Simulation results show that the AUV path planning scheme convergence is faster than that of conventional algorithms, and the lifetime of UWSN is prolonged while energy balancing according to the network size and node density. In high-density networks, the average energy consumption generated by AUVs for exploration is reduced by 15 percent for each additional AUV with our path planning.

Improved Artificial Potential Field Algorithm Assisted by Multisource Data for AUV Path Planning.

With the development of ocean exploration technology, the exploration of the ocean has become a hot research field involving the use of autonomous underwater vehicles (AUVs). In complex underwater environments, the fast, safe, and smooth arrival of target points is key for AUVs to conduct underwater exploration missions. Most path-planning algorithms combine deep reinforcement learning (DRL) and path-planning algorithms to achieve obstacle avoidance and path shortening. In this paper, we propose a method to improve the local minimum in the artificial potential field (APF) to make AUVs out of the local minimum by constructing a traction force. The improved artificial potential field (IAPF) method is combined with DRL for path planning while optimizing the reward function in the DRL algorithm and using the generated path to optimize the future path. By comparing our results with the experimental data of various algorithms, we found that the proposed method has positive effects and advantages in path planning. It is an efficient and safe path-planning method with obvious potential in underwater navigation devices.

A conflict-directed approach to chance-constrained mixed logical linear programming

Resistance to the adoption of autonomous systems comes in part from the perceived unreliability of the systems. The concerns can be addressed by deploying decision making algorithms that defines what it means to fail, and look for plans with the highest reward while limiting the probability of failure.This chance-constrained approach thus explicitly imposes a set of constraints that must be satisfied for success, and provides upper-bounds on the probability of violating such constraints. A chance-constrained mixed logical-linear program (CC-MLLP) is a natural formulation, allowing for the specification of linear and logical constraints, with probabilistic continuous variables. The formalism can be used to describe problems ranging from autonomous underwater vehicle path planning, to network routing under uncertainty.While naive encodings of CC-MLLPs can be solved with generalised solvers, the solution time may be unreasonable. In this work, we study architectures to speed up solutions by partitioning CC-MLLPs into the discrete and continuous portions. In order to provide faster solutions, we investigate methods for speeding up the solutions to the continuous chance-constrained linear programs. Further, by exploiting the new solution methods, we develop techniques for guiding the discrete decision making portion of the problem. The resulting algorithm achieves 10 times speed up over prior approaches on autonomous path planning benchmarks.

Improved AP-Clustering-Based AUV-Aided Data Collection Method for UWSNs

In underwater sensor networks, nodes are usually sparsely arranged, and autonomous underwater vehicles (AUVs) are often used to collect data by moving. However, it is a key problem for AUVs to shorten the path as much as possible to guarantee whole-network data acquisition. Aiming to solve this problem, this paper proposes an AUV underwater data collection method based on affinity propagation clustering. First, this clustering theory is used to select the nodes with large residual energy as cluster head nodes, and the clustering results are dynamically adjusted according to the communication radius of nodes to obtain the final clustering results. Finally, the cluster head nodes of the clusters are sequentially traversed using the traveling salesman problem, and an optimal path of AUV is planned to complete data collection for the whole network. A large number of simulation experiments verify that this method can effectively plan the mobile path of AUVs.

Data collection optimization of ocean observation network based on AUV path planning and communication

The Ocean Observation Network (OON) is an ocean science and engineering observation platform which integrates underwater wireless sensors, acoustic buoys, submersible buoys, and sonars. The complex underwater environment makes it challenging to place the observation nodes flexibly. In this research, firstly, a novel OON data collection optimization scheme is proposed, which is based on Autonomous Underwater Vehicle (AUV) path planning and communication. By combining the Voronoi diagram and K-means algorithm, an optimized clustering algorithm (VorK-means) is proposed to optimize the location of junction boxes and reduce the losses in the transmission energy of submarine cables in the OON simultaneously. Subsequently, a computational algorithm for AUV data collection position is proposed based on Voronoi vertices and communication circles. Moreover, an improved Grey Wolf Optimization (GWO) algorithm, based on the Levy flight and dimension (LD) search strategy, is proposed and applied for the AUV path planning, which is named LD-GWO. The accuracy and efficiency are validated by applying the proposed algorithms to the Canadian North-East Pacific Time-series Underwater Networked Experiment (NEPTUNE). Compared with the K-means algorithm, the VorK-means algorithm can reduce the total length of submarine cables by nearly 14%, and compared with the GWO algorithm, LD-GWO can obtain shorter data collection paths under identical parameter settings.

Adaptive SLAM Methodology Based on Simulated Annealing Particle Swarm Optimization for AUV Navigation

Simultaneous localization and mapping (SLAM) is crucial and challenging for autonomous underwater vehicle (AUV) autonomous navigation in complex and uncertain ocean environments. However, inaccurate time-varying observation noise parameters may lead to filtering divergence and poor mapping accuracy. In addition, particles are easily trapped in local extrema during the resampling, which may lead to inaccurate state estimation. In this paper, we propose an innovative simulated annealing particle swarm optimization-adaptive unscented FastSLAM (SAPSO-AUFastSLAM) algorithm. To cope with the unknown observation noise, the maximum a posteriori probability estimation algorithm is introduced into SLAM to recursively correct the measurement noise. Firstly, the Sage–Husa (SH) based unscented particle filter (UPF) algorithm is proposed to estimate time-varying measurement noise adaptively in AUV path estimation for improving filtering accuracy. Secondly, the SH-based unscented Kalman filter (UKF) algorithm is proposed to enhance mapping accuracy in feature estimation. Thirdly, SAPSO-based resampling is proposed to optimize posterior particles. The random judgment mechanism is used to update feasible solutions iteratively, which makes particles disengage local extreme values and achieve optimal global effects. The effectiveness and accuracy of the proposed algorithm are evaluated through simulation and sea trial data. The average AUV navigation accuracy of the presented SAPSO-AUFastSLAM method is improved by 18.0% compared to FastSLAM, 6.5% compared to UFastSLAM, and 5.9% compared to PSO-UFastSLAM.

Obstacle avoidance based on double closed loop control of autonomous underwater vehicle for submarine cable laying

When the autonomous underwater vehicle (AUV) lays the submarine cable in complex environment, it should balance the laying quality and its own safety. In this paper it designs an engineering modeling method under the condition of irregular obstacles and an obstacle avoidance method based on double closed loop control for AUV to lay the submarine cable. The model of irregular obstacle of bulge terrain and depression terrain are established respectively. Then the outer-loop controller based on model predictive control is designed to find the optimal path of AUV in order to avoid the unexpected obstacle. The segmented progressive optimization technique is used to keep the suspension height with the seabed. It could keep the minimal change of suspension height and obtain the shortest path. The best cabling path is designed by detecting the contour feature point and distinguishing the different work situations. The inner-loop controller based on Gaussian process is designed to reduce the influence of current interference. It compensates the control input with the mean value of Gaussian process by the outer-loop controller. Also this controller could reduce the outer-loop following error efficiently. In the simulation it compares double closed loop control with sphere model predictive control and double model predictive control respectively with the different obstacles. The result shows that double closed loop control improves the quality of the submarine cable laying and AUV's safety in the submarine complex environment effectively.

An Improved Quantum-Behaved Particle Swarm Optimization Algorithm Combined with Reinforcement Learning for AUV Path Planning

In order to solve the problem of fast path planning and effective obstacle avoidance for autonomous underwater vehicles (AUVs) in two-dimensional underwater environment, a path planning algorithm based on deep Q-network and Quantum particle swarm optimization (DQN-QPSO) was proposed. Five actions are defined first: normal, exploration, particle explode, random mutation, and fine-tuning operation. After that, the five actions are selected by DQN decision thinking, and the position information of particles is dynamically updated in each iteration according to the selected actions. Finally, considering the complexity of underwater environment, the fitness function is designed, and the route length, deflection angle, and the influence of ocean current are considered comprehensively, so that the algorithm can find the solution path with the shortest energy consumption in underwater environment. Experimental results show that DQN-QPSO algorithm is an effective algorithm, and its performance is better than traditional methods.

Hybrid Path Planning Using a Bionic-Inspired Optimization Algorithm for Autonomous Underwater Vehicles

This research presents a hybrid approach for path planning of autonomous underwater vehicles (AUVs). During path planning, static obstacles affect the desired path and path distance which result in collision penalties. In this study, the merits of grey wolf optimization (GWO) and genetic algorithm (GA) of bionic-inspired algorithms are integrated to implement a hybrid grey wolf optimization (HGWO) algorithm which allows AUVs to reach their destination safely in an obstacle rich environment. The proposed hybrid path planner is employed for path planning of a single AUV based on collision avoidance. It uses the GA as an initialization generator to overcome the random initialization problem of GWO. In this research, the total cost is considered to be a function of path distance and collision penalties. Further, the application of the proposed hybrid path planner is extended for cooperative path planning of AUVs while avoiding collision using communication consensus. Simulation results are obtained for both a single AUV and multiple AUV path planning in a 3D obstacle rich environment using a proportional-derivative controller. The Kruskal–Wallis test is employed for a non-parametric statistical analysis, where the independence of the results given by the algorithms is demonstrated.