Underwater 3D Research Articles

Multi-agent cooperative operation based on cross-domain zero-shot learning

Multi-agent systems (MAS) based on reinforcement learning have shown impressive performance in complex tasks. However, they are still weak when it comes to addressing tasks without any samples. In this paper, we design a 2D ground and a 3D underwater cooperative task, which serve as the source and target tasks, respectively. The MAS requires learning prior knowledge that can be generalized to the target task from the source task. To solve this problem, we propose cross-domain zero-shot learning (CDZSL) for MAS cooperative operation. Specifically, we first construct a state transition graph by decomposing the source tasks into multiple state phases. Subsequently, we describe the task description document for MAS and construct the task-related semantic embedding space, which effectively combats the semantic gap. Then, we propose zero-shot learning based on feature synthesis mapping (FSM), which synthesizes the feature centers of the unseen classes by the features of seen classes. During knowledge transfer, FSM is employed to assist MAS's decisions-making in the target task. Finally, extensive experiments show that CDZSL, compared with MAX-Q, improves the adaptability of MAS, where the cost of physical time for the first time to complete the new task is reduced by eight times.

Precision 3D Underwater Electric Field Measurement Method for Accurate Marine Localization

Precision 3D Underwater Electric Field Measurement Method for Accurate Marine Localization

Research on ASV/ROV Cooperative System with AI for Seagrass Distribution Survey

The authors developed the ASV/ROV cooperative system. The ASV has four thrusters located in a rhombus arrangement which enables omnidirectional movement. The developed ASV is equipped with GNSS, with such functions as dynamic positioning and automatic navigation by setting up route coordinate in advance. Moreover, it has a cable control device for linkage with ROV. A cable wound by drum is connected with a PC on the ASV through a slipring. In addition, a cage for ROV storage is set up between floating units of the ASV. The cage height is adjustable for the ROV to limit the hydrodynamic resistance during ASV cruising. It can take 3D underwater pictures and transmit images to land in real time. The ASV determines its movement from the relative position and azimuth of the ROV, as well as the cable length. The relative position of the ROV to the ASV is determined by an acoustic positioning system. Also, the ROV has AI system to catch images and grasp an image by DPS against current disturbance. The developed ASV/ROV cooperative mobility vehicle was successfully tested in the ocean for monitoring seaweeds and useful for CO2 measurement in the sea.

Virtual-Target based Path-Following in the 3D Underwater Environment

This work focuses on the design of a 3D path-following strategy for under-actuated underwater vehicles. The modeling of the problem relies on the Serret-Frenet apparatus, leading to the definition of the position error dynamics, with respect to a generic smooth curve in the operational 3D space. On the basis of such a model, a virtual-target based non-linear guidance law, derived by the exploitation of Lyapunov methodology, is designed. The theoretical proof and simulation results prove the functionality and reliability of the proposed approach.

3D concrete printing in air and under water: a comparative study on the buildability and interlayer adhesion

3D concrete printing (3DCP) technology is still in the early stages of research in the construction industry, and the development of underwater 3D concrete printing (U3DCP) based on this technology is still in its infancy. To promote the widespread application and adoption of U3DCP, a comprehensive understanding of the performance of 3D concrete printing in air and under water is crucial. In this study, the buildability and interlayer bonding of fresh concrete in different environments were evaluated and compared. The advantages of using water buoyancy to partially counterbalance the weight of concrete and enhance buildability, and the drawbacks of weakened interlayer bonding in a water environment were highlighted. Predictive formulas for the printable height of concrete printed in air and under water considering the impact of time intervals on the characteristics of the concrete, were proposed based on the relationship between the structural build-up coefficient and the height of the printed specimens. The interfacial adhesion of concrete printed in air was higher than that printed under water. However, the interlayer adhesion over time of concrete printed in air decreased faster than that printed under water. Permeability measurements on fresh concrete confirmed that water penetration is mainly responsible for the differences observed in interface adhesion.

Bioinspired self-assembled colloidal collectives drifting in three dimensions underwater.

Active matter systems feature a series of unique behaviors, including the emergence of collective self-assembly structures and collective migration. However, realizing collective entities formed by synthetic active matter in spaces without wall-bounded support makes it challenging to perform three-dimensional (3D) locomotion without dispersion. Inspired by the migration mechanism of plankton, we propose a bimodal actuation strategy in the artificial colloidal systems, i.e., combining magnetic and optical fields. The magnetic field triggers the self-assembly of magnetic colloidal particles to form a colloidal collective, maintaining numerous colloids as a dynamically stable entity. The optical field allows the colloidal collectives to generate convective flow through the photothermal effect, enabling them to use fluidic currents for 3D drifting. The collectives can perform 3D locomotion underwater, transit between the water-air interface, and have a controlled motion on the water surface. Our study provides insights into designing smart devices and materials, offering strategies for developing synthetic active matter capable of controllable collective movement in 3D space.

Development of a Methodology for Low-Cost 3D Underwater Motion Capture: Application to the Biomechanics of Horse Swimming.

Hydrotherapy has been utilized in horse rehabilitation programs for over four decades. However, a comprehensive description of the swimming cycle of horses is still lacking. One of the challenges in studying this motion is 3D underwater motion capture, which holds potential not only for understanding equine locomotion but also for enhancing human swimming performance. In this study, a marker-based system that combines underwater cameras and markers drawn on horses is developed. This system enables the reconstruction of the 3D motion of the front and hind limbs of six horses throughout an entire swimming cycle, with a total of twelve recordings. The procedures for pre- and post-processing the videos are described in detail, along with an assessment of the estimated error. This study estimates the reconstruction error on a checkerboard and computes an estimated error of less than 10 mm for segments of tens of centimeters and less than 1 degree for angles of tens of degrees. This study computes the 3D joint angles of the front limbs (shoulder, elbow, carpus, and front fetlock) and hind limbs (hip, stifle, tarsus, and hind fetlock) during a complete swimming cycle for the six horses. The ranges of motion observed are as follows: shoulder: 17 ± 3°; elbow: 76 ± 11°; carpus: 99 ± 10°; front fetlock: 68 ± 12°; hip: 39 ± 3°; stifle: 68 ± 7°; tarsus: 99 ± 6°; hind fetlock: 94 ± 8°. By comparing the joint angles during a swimming cycle to those observed during classical gaits, this study reveals a greater range of motion (ROM) for most joints during swimming, except for the front and hind fetlocks. This larger ROM is usually achieved through a larger maximal flexion angle (smaller minimal angle of the joints). Finally, the versatility of the system allows us to imagine applications outside the scope of horses, including other large animals and even humans.

PCD reconstruction, object classification and pose estimation for underwater vehicles using orthogonal multibeam forward looking sonar fusion

Orthogonal Multibeam Sonar Fusion (OMSF) is a recent fusion based method capable of producing accurate underwater 3D Point Cloud Data (PCD) from Multibeam Forward Looking Sonars (MFLS), enabling accurate seabed mapping and object scanning. This article provides methodical testing of OMSF reconstruction for MFLS accounting for operating frequency effects and object shapes. The article then proposes novel perception based applications for OMSF, consisting of classification technique integrating OMSF PCDs with a PCD based Convolutional Neural Network (CNN), and pose estimation method combining Orthogonal Feature Matching (OFM) bounding box regression with a pose regression CNN. Reconstruction test results show that OMSF is more accurate and efficient using higher frequency MFLS and achieves up to 53% higher accuracy on solid surfaces compared to hollow frames. Application tests based on an underwater garage dock show classification using OMSF PCDs can achieve 25% and 37% higher success rate and confidence while being more efficient, compared to using raw 3D sonar data. OFM bounding box regression produces 4.28% higher mean Intersection over Union (IoU), and 10% increase in ¿25% IoU metric compared to methods based on more traditional MFLS filtering. Similarly, end-to-end pose estimation achieves 6.25% higher success rate with OFM bounding box samples compared to ones obtained using traditional MFLS filtering.

Underwater single photon 3D imaging with millimeter depth accuracy and reduced blind range.

Mono-static system benefits from its more flexible field of view and simplified structure, however, the backreflection photons from mono-static system lead to count loss for target detection. Counting loss engender range-blind, impeding the accurate acquisition of target depth. In this paper, count loss is reduced by introducing a polarization-based underwater mono-static single-photon imaging method, and hence reduced blind range. The proposed method exploits the polarization characteristic of light to effectively reduce the count loss of the target, thus improving the target detection efficiency. Experiments demonstrate that the target profile can be visually identified under our method, while the unpolarization system can not. Moreover, the ranging precision of system reaches millimeter-level. Finally, the target profile is reconstructed using non-local pixel correlations algorithm.

Influential factors on mechanical properties and microscopic characteristics of underwater 3D printing concrete

Underwater 3D Printing Concrete (U3DPC) technology can be a promising approach to address the challenges of renewable marine energy and coastal protection. Conducting extensive research on the mechanical properties of U3DPC is crucial for its successful implementation. This study investigates various factors, including printing environment, construction methods (mold-cast and printing), curing age, and material composition, regarding their significance on the mechanical properties of U3DPC. The compressive strength, anisotropy, interlayer bonding, and interface microstructure are evaluated. The research findings reveal that compared to casting specimens in air, the printing method and water environmental factors result in approximately 20% and 15.1% reduction in the compressive strength of U3DPC, respectively. U3DPC exhibits different anisotropic variations compared to specimens printed in air, attributed to weak interlayer bonding interfaces caused by external water. The influence of material components on the interlayer bonding strength of U3DPC has a critical threshold that should not be exceeded; otherwise, it weakens the interlayer bonding. The variation in fine aggregate types and fibers leads to changes in interlayer interface roughness, where excessive roughness captures more ambient water due to the “dog-tooth overlapping” structure, thereby reducing the adhesive capacity. The pore distribution reveals the underlying mechanism of how interface roughness affects interlayer bonding.

3D underwater acoustic Luneburg lens based on gradient face-centered-cubic phononic crystals

A Luneburg lens is a gradient refractive index lens that can focus plane waves on a point at the perimeter without aberration. Three-dimensional (3D) Luneburg lens for airborne sound has been well investigated in recent years. However, constructing a 3D Luneburg lens for underwater sound is a challenging task due to the difficulties in the designing and fabricating of the desired isotropic underwater acoustic materials. This work presents the practical implementation of a 3D Luneburg lens for underwater sound. Such a 3D Luneburg lens is designed based on 3D gradient face-centered-cubic phononic crystals, which have quasi-isotropic refractive index patterns and can be fabricated with photosensitive resin by 3D printing. The experimental results show that the lens can realize the omnidirectional imaging of underwater sound from 30 to 38 kHz. This 3D underwater acoustic Luneburg lens may prompt the potential applications in underwater acoustic wide-angle retroreflectors, sonars, and biomedical imaging devices.

Three-Dimensional Prescribed Performance Tracking Control of UUV via PMPC and RBFNN-FTTSMC

To address the search-and-docking problem in multi-stage prescribed performance switching (MPPS) scenarios, this paper presents a novel compound control method for three-dimensional (3D) underwater trajectory tracking control of unmanned underwater vehicles (UUVs) subjected to unknown disturbances. The proposed control framework can be divided into two parts: kinematics control and dynamics control. In the kinematics control loop, a novel parallel model predictive control (PMPC) law is proposed, which is composed of a soft-constrained model predictive controller (SMPC) and hard-constrained model predictive controller (HMPC), and utilizes a weight allocator to enable switching between soft and hard constraints based on task goals, thus achieving global optimal control in MPPS scenarios. In the dynamics control loop, a finite-time terminal sliding mode control (FTTSMC) method combining a finite-time radial basis function neural network adaptive disturbance observer (RBFNN-FTTSMC) is proposed to achieve disturbance estimation and fast convergence of velocity tracking errors. The simulation results demonstrate that the proposed PMPC-FTTSMC approach achieved an average improvement of 33% and 80% in the number of iterations compared with MPC with sliding mode control (MPC-SMC) and traditional MPC methods, respectively. Furthermore, the approach improved the speed of response by 35% and 44%, respectively, while accurately achieving disturbance observation and enhancing the system robustness.

State Super Sampling Soft Actor–Critic Algorithm for Multi-AUV Hunting in 3D Underwater Environment

Reinforcement learning (RL) is known for its efficiency and practicality in single-agent planning, but it faces numerous challenges when applied to multi-agent scenarios. In this paper, a Super Sampling Info-GAN (SSIG) algorithm based on Generative Adversarial Networks (GANs) is proposed to address the problem of state instability in Multi-Agent Reinforcement Learning (MARL). The SSIG model allows a pair of GAN networks to analyze the previous state of dynamic system and predict the future state of consecutive state pairs. A multi-agent system (MAS) can deduce the complete state of all collaborating agents through SSIG. The proposed model has the potential to be employed in multi-autonomous underwater vehicle (multi-AUV) planning scenarios by combining it with the Soft Actor–Critic (SAC) algorithm. Hence, this paper presents State Super Sampling Soft Actor–Critic (S4AC), which is a new algorithm that combines the advantages of SSIG and SAC and can be applied to Multi-AUV hunting tasks. The simulation results demonstrate that the proposed algorithm has strong learning ability and adaptability and has a considerable success rate in hunting the evading target in multiple testing scenarios.

A low‐cost, long‐running, open‐source stereo camera for tracking aquatic species and their behaviours

Abstract Ecologists are now widely utilising video data to quantify the behaviours and interactions of animals in the wild. This process can be facilitated by collecting videos in stereo, which can provide information about animals' positions, movements and behaviours in three‐dimensions (3D). However, there are no published designs that can collect underwater 3D stereo data at high spatial and temporal resolutions for extended periods (days). Here, we present complete hardware and software solutions for a long‐running, open‐source, underwater stereo camera rig, costing £1337. This stereo camera can continuously record aquatic species and their behaviours/interactions in high resolution (1080 p and 30 fps) and in 3D, over multiple days. We provide full design guides for the cameras and a travel‐friendly rig, and include guidance and open‐source code for calibrating the cameras in space and time. We also show how these cameras could be used to track animals' body parts and positions, and how their size, posture and behaviour can be inferred. This stereo camera will facilitate the collection of high‐resolution ecological and behavioural data, such as affiliative, agonistic or trophic interactions between species, which can inform us about the health and structure of ecosystems. These data will assist ecologists and conservationists in monitoring and understanding the impacts of current environmental pressures on ecosystem functioning.

Optimum energy efficient multi‐hop protocol for homogenous and heterogeneous 3D underwater acoustic sensor networks

SummaryInformation transmission is extremely challenging in underwater acoustic sensor networks (UASNs) because, in the acoustic channel, the packet loss is high compared to other channels due to low bandwidth and a long propagation delay. So, designing an efficient energy algorithm is a big challenge in the acoustic channel. In this paper, first, a cluster head (CH) election fitness function based on a hybrid particle swarm optimization and gravitational search algorithm (PSO‐GSA) is proposed for a three‐dimensional (3D) underwater sensor network (UASN). The proposed algorithm includes CH election, CH load, and neighbor CH distance for multi‐hop transmission (MH‐PSOGSA). Then, based on the proposed algorithm, a relay‐based traffic‐aware energy‐efficient routing protocol (R‐TAEERP) with a weight function is proposed to offload the heavy consumption of the CHs. MATLAB simulations were accomplished to compare the performance of MH‐PSOGSA and R‐TAEERP with the existing representative protocols. The heterogeneity in energy and traffic has been considered. The simulation results show better performance of the proposed algorithm than other existing representative protocols in terms of energy consumption, the performance of the stable region, and the lifetime of the network.

Overview of Underwater 3D Reconstruction Technology Based on Optical Images

At present, 3D reconstruction technology is being gradually applied to underwater scenes and has become a hot research direction that is vital to human ocean exploration and development. Due to the rapid development of computer vision in recent years, optical image 3D reconstruction has become the mainstream method. Therefore, this paper focuses on optical image 3D reconstruction methods in the underwater environment. However, due to the wide application of sonar in underwater 3D reconstruction, this paper also introduces and summarizes the underwater 3D reconstruction based on acoustic image and optical–acoustic image fusion methods. First, this paper uses the Citespace software to visually analyze the existing literature of underwater images and intuitively analyze the hotspots and key research directions in this field. Second, the particularity of underwater environments compared with conventional systems is introduced. Two scientific problems are emphasized by engineering problems encountered in optical image reconstruction: underwater image degradation and the calibration of underwater cameras. Then, in the main part of this paper, we focus on the underwater 3D reconstruction methods based on optical images, acoustic images and optical–acoustic image fusion, reviewing the literature and classifying the existing solutions. Finally, potential advancements in this field in the future are considered.

Three-Dimensional Underwater Path Planning Based on Modified Potential Field Algorithm in Time-Varying Current

Abstract The article addresses the three-dimensional (3D) underwater path planning problem of an autonomous underwater vehicle (AUV) in a time-varying current. A modified artificial potential field algorithm combining the velocity vector synthesis method is proposed to search for the optimal path. The modified potential field (MPF) algorithm is designed to dynamically plan the non-collision path. Meanwhile, this modified method is also proved to be an effective solution to the goals not reachable with obstacles nearby (GNRON), U-shaped trap, and rotation unreachable problems. To offset the influence of time-varying current, the velocity synthesis approach is designed to adjust the AUV movement direction. Besides, considering path planning in the complex underwater environment, the multi-beam forward-looking sonar (FLS) model is used. Finally, simulation studies substantiate that the designed algorithm can implement the AUV path planning effectively and successfully in a 3D underwater environment.

A Combined Physical and Mathematical Calibration Method for Low-Cost Cameras in the Air and Underwater Environment.

Low-cost camera calibration is vital in air and underwater photogrammetric applications. However, various lens distortions and the underwater environment influence are difficult to be covered by a universal distortion compensation model, and the residual distortions may still remain after conventional calibration. In this paper, we propose a combined physical and mathematical camera calibration method for low-cost cameras, which can adapt to both in-air and underwater environments. The commonly used physical distortion models are integrated to describe the image distortions. The combination is a high-order polynomial, which can be considered as basis functions to successively approximate the image deformation from the point of view of mathematical approximation. The calibration process is repeated until certain criteria are met and the distortions are reduced to a minimum. At the end, several sets of distortion parameters and stable camera interior orientation (IO) parameters act as the final camera calibration results. The Canon and GoPro in-air calibration experiments show that GoPro owns distortions seven times larger than Canon. Most Canon distortions have been described with the Australis model, while most decentering distortions for GoPro still exist. Using the proposed method, all the Canon and GoPro distortions are decreased to close to 0 after four calibrations. Meanwhile, the stable camera IO parameters are obtained. The GoPro Hero 5 Black underwater calibration indicates that four sets of distortion parameters and stable camera IO parameters are obtained after four calibrations. The camera calibration results show a difference between the underwater environment and air owing to the refractive and asymmetric environment effects. In summary, the proposed method improves the accuracy compared with the conventional method, which could be a flexible way to calibrate low-cost cameras for high accurate in-air and underwater measurement and 3D modeling applications.

Optimal topology design of multi-target AUVs for 3D cooperative localization formation based on angle of arrival measurement

With the expansion and deepening of ocean research, cooperative localization technology of multi-AUVs is being explored more and more. However, the underwater cooperative localization based on Angle of Arrival (AOA) as well as cooperative localization formation are still relatively less explored research areas. Moreover, the cooperative localization formation has significant impact on the localization accuracy of cooperative localization system. In this paper, firstly, a novel optimization model of multi-target AUVs for 3D cooperative localization formation based on AOA measurement is formulated. Secondly, AOA-based evaluation functions of single and multiple targets are derived for underwater 3D cooperative localization system using the D-optimality criterion. Finally, considering the practical constraints of the underwater environment and the computational complexity of the multi-target system, a Sparrow Search Algorithm based on Simulated Annealing (SA-SSA) is proposed to obtain the optimal formation of the cooperative localization system. The algorithm ensured that the global optimal solution for the cooperative localization system is independent of the initial positions of the leader AUVs. Various simulation examples verify the effectiveness of the results and algorithms. Comparison of proposed algorithm with the traditional iterative algorithm proves that the positioning accuracy and system stability are effectively improved under proposed SA-SSA algorithm.

Underwater 3D reconstruction using a photometric stereo with illuminance estimation.

Underwater image sensing is directly affected by the change in illuminance towards a camera caused by the refraction of light in different media. In this study, the convergence of a near-field point light source in water is analyzed using a light propagation model. The photometric stereo (PS) formula is determined based on an accurate estimation of the illuminance entering the camera. An underwater PS system is designed to verify the proposed method's feasibility. The experimental results demonstrate improved accuracy in normal calculation. This helps achieve accurate underwater 3D reconstruction of objects that is suitable for underwater surface microdefect detection.