Autonomous Underwater Vehicle Path Research Articles

Integrated Autonomous Underwater Vehicle Path Planning and Collision Avoidance Algorithms

Integrated Autonomous Underwater Vehicle Path Planning and Collision Avoidance Algorithms

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

In order to improve the autonomous navigation capability of autonomous underwater vehicles (AUV), this paper applies Deep Reinforcement Learning (DRL) to realize autonomous path planning. In response to the limited detection range and difficult tracking DRL policy in three-dimensional unknown environments, this paper proposes an improved deep reinforcement learning method. Firstly, performing AUV modeling and processing obstacle detection information. Second, under a framework of reinforcement learning with the control system, an improved environmental interaction method was designed by analyzing the motion characteristics and controller performance of the AUV. Finally, based on this method, a policy network that can generate paths that satisfy obstacle avoidance and navigation is trained in combination with a twin delayed deterministic policy gradient (TD3). The simulation results show that the proposed method can achieve AUV path planning in a three-dimensional unknown environment. The proposed framework can ensure that the planned path meets the dynamics constraints and is trackable. The improved algorithm can enhance the convergence of training and the attitude stability of AUV.

Imitation learning from imperfect demonstrations for AUV path tracking and obstacle avoidance

Autonomous underwater vehicle (AUV) is widely used for complex underwater tasks such as seafloor exploration. In recent years, deep reinforcement learning (DRL) has been introduced to the AUV control due to its capability to improve the autonomy of AUV. However, it is usually very difficult to design an effective reward function for the DRL methods. Generative adversarial imitation learning (GAIL) can allow AUVs to learn control policies from expert demonstrations instead of pre-defined reward functions, but suffers from the deficiency of requiring optimal expert demonstrations and not surpassing the provided demonstrations. This paper builds upon the GAIL algorithm for AUV learning control policies from expert demonstrations. We proposed an importance reweighting generative adversarial imitation learning (WGAIL) algorithm by using confidence scores to indicate the optimality of the demonstrated trajectories, which can facilitate AUVs to learn control policies from expert demonstrations of different levels. Our experimental results on a simulated AUV system modeling Sailfish 210 of our lab in the Gazebo simulation environment show that an AUV trained via WGAIL can achieve a better performance than the one trained via GAIL with different levels of expert sub-optimal demonstrations. Moreover, control policies trained via WGAIL in simple tasks can generalize better to complex tasks than those trained via GAIL, greatly extending the applicability of the AUV learning from expert demonstrations.

Oceanic Challenges to Technological Solutions: A Review of Autonomous Underwater Vehicle Path Technologies in Biomimicry, Control, Navigation, and Sensing

Oceanic Challenges to Technological Solutions: A Review of Autonomous Underwater Vehicle Path Technologies in Biomimicry, Control, Navigation, and Sensing

Technical Design and Data Analysis of Autonomous Underwater Vehicle-Based Side-Scan Sonar Operations

In response to the absence of standardized work practices, work safety measures, efficient work procedures, and suitable line planning methods for exploring seabed topography using autonomous underwater vehicles (AUVs) equipped with side-scan sonar systems, this paper proposes a design framework for side-scan sonar measurement technology based on AUVs. Detailed research has been conducted on line planning methods that differ significantly from those used in shipborne operations. The new line planning method was developed based on the performance characteristics of AUV equipment, the principles of motion modes, and shipborne side-scan sonar operations. The validity of the line planning method was verified through actual sea experiments using different types of AUVs in various terrains. The experimental results demonstrate that the new survey line planning method is distinct from existing AUV path planning methods and is highly applicable for conducting seabed topography exploration operations. Furthermore, the technical design scheme studied in this article is applicable in different sea areas and ensures the safety, operational efficiency, and data quality of AUVs. It holds significant importance for marine science engineering and practical applications. The article also features an analysis of successful data collected from AUVs equipped with side-scan sonar for conducting seabed topography exploration operations.

AUV Path Planning Considering Ocean Current Disturbance Based on Cloud Desktop Technology.

In the field of ocean energy detection, Autonomous Underwater Vehicles (AUVs) offer significant advantages in terms of manpower, resource, and energy efficiency. However, the unpredictable nature of the ocean environment, particularly the real-time changes in ocean currents, poses navigational risks for AUVs. Therefore, effective path planning in dynamic environments is crucial for AUVs to perform specific tasks. This paper addresses the static path planning problem and proposes a model called the noise net double DQN network with prioritized experience replay (N-DDQNP). The N-DDQNP model combines a noise network and a prioritized experience replay mechanism to address the limited exploration and slow convergence speed issues of the DQN algorithm, which are caused by the greedy strategy and uniform sampling mechanism. The proposed approach involves constructing a double DQN network with a priority experience replay and an exploration mechanism using the noise network. Second, a compound reward function is formulated to take into account ocean current, distance, and safety factors, ensuring prompt feedback during the training process. Regarding the ocean current, the reward function is designed based on the angle between the current direction and the AUV's heading direction, considering its impact on the AUV's speed. As for the distance factor, the reward is determined by the Euclidean distance between the current position and the target point. Furthermore, the safety factor considers whether the AUV may collide with obstacles. By incorporating these three factors, the compound reward function is established. To evaluate the performance of the N-DDQNP model, experiments were conducted using real ocean data in various complex ocean environments. The results demonstrate that the path planning time of the N-DDQNP model outperforms other algorithms in different ocean current scenarios and obstacle environments. Furthermore, a user console-AUV connection has been established using spice cloud desktop technology. The cloud desktop architecture enables intuitive observation of the AUV's navigation posture and the surrounding marine environment, facilitating safer and more efficient underwater exploration and marine resource detection tasks.

Energy-Efficient Multiple Autonomous Underwater Vehicle Path Planning Scheme in Underwater Sensor Networks

The issue of limited energy resources is crucial for underwater wireless sensor networks (UWSNs) because these networks operate in remote and harsh environments where access to power sources is limited. Overcoming the energy constraints is necessary to ensure the long-term functionality and sustainability of UWSN, enabling continuous data collection and communication for various applications such as environmental monitoring and surveillance. To solve the problems of limited energy and the difficulty of battery replacement in UWSN, a path planning and energy-saving scheme for charging underwater sensor nodes using AUVs (autonomous underwater vehicles) is proposed. Applying multiple AUVs to charge the sensing network nodes will maximize the size of the underwater sensing network as well as meet the transmission reliability, and the optimal path of AUVs is solved by using a genetic algorithm. Simulation results show that the AUV path planning scheme convergence is faster than that of conventional algorithms, and the lifetime of UWSN is prolonged while energy balancing according to the network size and node density. In high-density networks, the average energy consumption generated by AUVs for exploration is reduced by 15 percent for each additional AUV with our path planning.

Improved Artificial Potential Field Algorithm Assisted by Multisource Data for AUV Path Planning.

With the development of ocean exploration technology, the exploration of the ocean has become a hot research field involving the use of autonomous underwater vehicles (AUVs). In complex underwater environments, the fast, safe, and smooth arrival of target points is key for AUVs to conduct underwater exploration missions. Most path-planning algorithms combine deep reinforcement learning (DRL) and path-planning algorithms to achieve obstacle avoidance and path shortening. In this paper, we propose a method to improve the local minimum in the artificial potential field (APF) to make AUVs out of the local minimum by constructing a traction force. The improved artificial potential field (IAPF) method is combined with DRL for path planning while optimizing the reward function in the DRL algorithm and using the generated path to optimize the future path. By comparing our results with the experimental data of various algorithms, we found that the proposed method has positive effects and advantages in path planning. It is an efficient and safe path-planning method with obvious potential in underwater navigation devices.

A conflict-directed approach to chance-constrained mixed logical linear programming

Resistance to the adoption of autonomous systems comes in part from the perceived unreliability of the systems. The concerns can be addressed by deploying decision making algorithms that defines what it means to fail, and look for plans with the highest reward while limiting the probability of failure.This chance-constrained approach thus explicitly imposes a set of constraints that must be satisfied for success, and provides upper-bounds on the probability of violating such constraints. A chance-constrained mixed logical-linear program (CC-MLLP) is a natural formulation, allowing for the specification of linear and logical constraints, with probabilistic continuous variables. The formalism can be used to describe problems ranging from autonomous underwater vehicle path planning, to network routing under uncertainty.While naive encodings of CC-MLLPs can be solved with generalised solvers, the solution time may be unreasonable. In this work, we study architectures to speed up solutions by partitioning CC-MLLPs into the discrete and continuous portions. In order to provide faster solutions, we investigate methods for speeding up the solutions to the continuous chance-constrained linear programs. Further, by exploiting the new solution methods, we develop techniques for guiding the discrete decision making portion of the problem. The resulting algorithm achieves 10 times speed up over prior approaches on autonomous path planning benchmarks.

Improved AP-Clustering-Based AUV-Aided Data Collection Method for UWSNs

In underwater sensor networks, nodes are usually sparsely arranged, and autonomous underwater vehicles (AUVs) are often used to collect data by moving. However, it is a key problem for AUVs to shorten the path as much as possible to guarantee whole-network data acquisition. Aiming to solve this problem, this paper proposes an AUV underwater data collection method based on affinity propagation clustering. First, this clustering theory is used to select the nodes with large residual energy as cluster head nodes, and the clustering results are dynamically adjusted according to the communication radius of nodes to obtain the final clustering results. Finally, the cluster head nodes of the clusters are sequentially traversed using the traveling salesman problem, and an optimal path of AUV is planned to complete data collection for the whole network. A large number of simulation experiments verify that this method can effectively plan the mobile path of AUVs.

Data collection optimization of ocean observation network based on AUV path planning and communication

The Ocean Observation Network (OON) is an ocean science and engineering observation platform which integrates underwater wireless sensors, acoustic buoys, submersible buoys, and sonars. The complex underwater environment makes it challenging to place the observation nodes flexibly. In this research, firstly, a novel OON data collection optimization scheme is proposed, which is based on Autonomous Underwater Vehicle (AUV) path planning and communication. By combining the Voronoi diagram and K-means algorithm, an optimized clustering algorithm (VorK-means) is proposed to optimize the location of junction boxes and reduce the losses in the transmission energy of submarine cables in the OON simultaneously. Subsequently, a computational algorithm for AUV data collection position is proposed based on Voronoi vertices and communication circles. Moreover, an improved Grey Wolf Optimization (GWO) algorithm, based on the Levy flight and dimension (LD) search strategy, is proposed and applied for the AUV path planning, which is named LD-GWO. The accuracy and efficiency are validated by applying the proposed algorithms to the Canadian North-East Pacific Time-series Underwater Networked Experiment (NEPTUNE). Compared with the K-means algorithm, the VorK-means algorithm can reduce the total length of submarine cables by nearly 14%, and compared with the GWO algorithm, LD-GWO can obtain shorter data collection paths under identical parameter settings.

Adaptive SLAM Methodology Based on Simulated Annealing Particle Swarm Optimization for AUV Navigation

Simultaneous localization and mapping (SLAM) is crucial and challenging for autonomous underwater vehicle (AUV) autonomous navigation in complex and uncertain ocean environments. However, inaccurate time-varying observation noise parameters may lead to filtering divergence and poor mapping accuracy. In addition, particles are easily trapped in local extrema during the resampling, which may lead to inaccurate state estimation. In this paper, we propose an innovative simulated annealing particle swarm optimization-adaptive unscented FastSLAM (SAPSO-AUFastSLAM) algorithm. To cope with the unknown observation noise, the maximum a posteriori probability estimation algorithm is introduced into SLAM to recursively correct the measurement noise. Firstly, the Sage–Husa (SH) based unscented particle filter (UPF) algorithm is proposed to estimate time-varying measurement noise adaptively in AUV path estimation for improving filtering accuracy. Secondly, the SH-based unscented Kalman filter (UKF) algorithm is proposed to enhance mapping accuracy in feature estimation. Thirdly, SAPSO-based resampling is proposed to optimize posterior particles. The random judgment mechanism is used to update feasible solutions iteratively, which makes particles disengage local extreme values and achieve optimal global effects. The effectiveness and accuracy of the proposed algorithm are evaluated through simulation and sea trial data. The average AUV navigation accuracy of the presented SAPSO-AUFastSLAM method is improved by 18.0% compared to FastSLAM, 6.5% compared to UFastSLAM, and 5.9% compared to PSO-UFastSLAM.

Obstacle avoidance based on double closed loop control of autonomous underwater vehicle for submarine cable laying

When the autonomous underwater vehicle (AUV) lays the submarine cable in complex environment, it should balance the laying quality and its own safety. In this paper it designs an engineering modeling method under the condition of irregular obstacles and an obstacle avoidance method based on double closed loop control for AUV to lay the submarine cable. The model of irregular obstacle of bulge terrain and depression terrain are established respectively. Then the outer-loop controller based on model predictive control is designed to find the optimal path of AUV in order to avoid the unexpected obstacle. The segmented progressive optimization technique is used to keep the suspension height with the seabed. It could keep the minimal change of suspension height and obtain the shortest path. The best cabling path is designed by detecting the contour feature point and distinguishing the different work situations. The inner-loop controller based on Gaussian process is designed to reduce the influence of current interference. It compensates the control input with the mean value of Gaussian process by the outer-loop controller. Also this controller could reduce the outer-loop following error efficiently. In the simulation it compares double closed loop control with sphere model predictive control and double model predictive control respectively with the different obstacles. The result shows that double closed loop control improves the quality of the submarine cable laying and AUV's safety in the submarine complex environment effectively.

An Improved Quantum-Behaved Particle Swarm Optimization Algorithm Combined with Reinforcement Learning for AUV Path Planning

In order to solve the problem of fast path planning and effective obstacle avoidance for autonomous underwater vehicles (AUVs) in two-dimensional underwater environment, a path planning algorithm based on deep Q-network and Quantum particle swarm optimization (DQN-QPSO) was proposed. Five actions are defined first: normal, exploration, particle explode, random mutation, and fine-tuning operation. After that, the five actions are selected by DQN decision thinking, and the position information of particles is dynamically updated in each iteration according to the selected actions. Finally, considering the complexity of underwater environment, the fitness function is designed, and the route length, deflection angle, and the influence of ocean current are considered comprehensively, so that the algorithm can find the solution path with the shortest energy consumption in underwater environment. Experimental results show that DQN-QPSO algorithm is an effective algorithm, and its performance is better than traditional methods.

Hybrid Path Planning Using a Bionic-Inspired Optimization Algorithm for Autonomous Underwater Vehicles

This research presents a hybrid approach for path planning of autonomous underwater vehicles (AUVs). During path planning, static obstacles affect the desired path and path distance which result in collision penalties. In this study, the merits of grey wolf optimization (GWO) and genetic algorithm (GA) of bionic-inspired algorithms are integrated to implement a hybrid grey wolf optimization (HGWO) algorithm which allows AUVs to reach their destination safely in an obstacle rich environment. The proposed hybrid path planner is employed for path planning of a single AUV based on collision avoidance. It uses the GA as an initialization generator to overcome the random initialization problem of GWO. In this research, the total cost is considered to be a function of path distance and collision penalties. Further, the application of the proposed hybrid path planner is extended for cooperative path planning of AUVs while avoiding collision using communication consensus. Simulation results are obtained for both a single AUV and multiple AUV path planning in a 3D obstacle rich environment using a proportional-derivative controller. The Kruskal–Wallis test is employed for a non-parametric statistical analysis, where the independence of the results given by the algorithms is demonstrated.

An Improved S-Plane Controller for High-Speed Multi-Purpose AUVs with Situational Static Loads

The classic S-plane control method combines PD structure with fuzzy control theory, with the advantages of a simple control structure and fewer parameters to be adjusted. It has been proved as a practical method in an autonomous underwater vehicle (AUV) motion control at low and medium speeds, but it takes no account of the situational static load and varying hydrodynamic forces which influence the control quality and even result in a “dolphin effect” at the time of high-speed movement. For this reason, an improved S-plane controller is designed based on the sliding mode variable structure, sliding mode surface, and control items in order to respond to the situational static load and high-speed movement. The improved S-plane controller is verified by Lyapunov stability analysis. The thrust allocation strategies are also discussed with constraints introduced in accordance with task requirements. In order to verify the practicability and effectiveness of the improved S-plane controller, both simulation experiments and field trials of AUV motion control, long-range cruise, and path point following were carried out. The results have demonstrated the superiority of the improved S-plane controller over the classic S-plane controller.

Three-Dimensional Underwater Path Planning Based on Modified Potential Field Algorithm in Time-Varying Current

Abstract The article addresses the three-dimensional (3D) underwater path planning problem of an autonomous underwater vehicle (AUV) in a time-varying current. A modified artificial potential field algorithm combining the velocity vector synthesis method is proposed to search for the optimal path. The modified potential field (MPF) algorithm is designed to dynamically plan the non-collision path. Meanwhile, this modified method is also proved to be an effective solution to the goals not reachable with obstacles nearby (GNRON), U-shaped trap, and rotation unreachable problems. To offset the influence of time-varying current, the velocity synthesis approach is designed to adjust the AUV movement direction. Besides, considering path planning in the complex underwater environment, the multi-beam forward-looking sonar (FLS) model is used. Finally, simulation studies substantiate that the designed algorithm can implement the AUV path planning effectively and successfully in a 3D underwater environment.

Autonomous Underwater Vehicle Path Tracking Based on the Optimal Fuzzy Controller with Multiple Performance Indexes

Autonomous underwater vehicles (AUVs) are increasingly being used in missions involving submarine cable detection, underwater archaeology, pipeline inspection, military reconnaissance, and so on. It is very important to realize AUV path tracking to accomplish these missions. In this paper, a fuzzy controller based on the established kinematic and dynamic models of AUV systems is presented to solve the AUV path-tracking problem. In order to design the fuzzy controller to exhibit good performance, we select the path length, smoothness, and cross-track position error as the multiple optimization performance indexes for the fuzzy controller. We propose the particle swarm optimization (PSO) algorithm to determine the parameters of the membership functions. Different scenarios are presented to test the performance of the proposed algorithm, including the straight line, sine curve, half-moon shape, Archimedean spiral, and practical paths. The results are given to illustrate the effectiveness and feasibility of the fuzzy controller with the optimization of multiple performance indexes.

A novel reinforcement learning based tuna swarm optimization algorithm for autonomous underwater vehicle path planning

Path planning technology is an important guarantee for the safe navigation of autonomous underwater vehicle (AUV) in water, and it is also an important indicator of the intelligence of autonomous underwater vehicle. Aiming at the path planning problem of AUV in complex environments, this article presents a reinforcement learning-based tuna swarm optimization algorithm called the QLTSO. In this algorithm, individuals are independent of each other, and the choice of each individual strategy is decided by reinforcement learning. Four strategies are set for each individual in the algorithm: spiral foraging, parabolic foraging, optimization adjustment and ESOS strategy. Finally, the cubic B-spline curve is used to smooth the path so that the autonomous underwater vehicle can better track the path. To verify the superiority of the QLTSO algorithm, the algorithm is compared with other advanced optimization algorithms. The simulation results show that the QLTSO algorithm can plan safe and effective AUV navigation paths in a variety of two-dimensional and three-dimensional complex environments with better convergence and robustness, and the planning success rate is up to 100%, which is an effective AUV path planning algorithm.

A Time-Saving Path Planning Scheme for Autonomous Underwater Vehicles With Complex Underwater Conditions

Autonomous underwater vehicle (AUV) shows great potential in the Internet of Underwater Things (IoUT) system, in which the path planning algorithm plays a fundamental role. However, the complex underwater environment brings greater challenges to AUV path planning, especially the ocean current, which has a profound impact on time and energy consumption. This article focuses on the complex ocean current condition and proposes an underwater path planning method based on proximal policy optimization (UP4O). In this novel method, a deep reinforcement network is constructed to serve as a decision control to plan the moving direction of AUV. An information encoding module is developed to extract the features of the local obstacles. Furthermore, UP4O integrates the obstacle features with the current state information, including relative position, ocean current, and velocity, enabling the AUV to focus on the global direction and local obstacles at the same time. Additionally, to further adapt to the ocean current and shorten the time cost, UP4O expands the action space of AUV, realizing a fine and flexible action adjustment. The wide applicability of UP4O has been proved by numerous experiments. The proposed algorithm can always plan the time-saving and collision-free paths in complex underwater environments with various terrains and ocean current.