Underwater 3D Reconstruction Research Articles

Method of 3D reconstruction of underwater concrete by laser line scanning

High-precision 3D (three-dimensional) reconstruction of underwater concrete is helpful for the detection and analysis of underwater concrete diseases. A prevalent method involves mounting optical cameras in waterproof housings to enable the 3D reconstruction of underwater concrete. Achieving millimetric accuracy in underwater 3D reconstruction poses a significant challenge in current research. The refraction of light on surfaces of different media and its reflection underwater complicate the process. In this paper, a 3D reconstruction method of underwater concrete based on laser line scanning is proposed. Firstly, the underwater imaging model is established by the ray tracing method, and the refraction conversion matrix is derived. Then, based on this matrix, the conversion relationship between the measured 3D size and the real 3D size of the underwater object is obtained. Finally, image restoration technology is introduced in the process of laser stripe extraction to eliminate the influence of reflected light on the extraction of laser stripe center, to improve the reconstruction accuracy. We used the independently developed laser line 3D scanning equipment to carry out experimental analysis. The results showed that the error rate caused by direct reconstruction of underwater objects was 9.75 %. Additionally, the error rate obtained from 3D reconstruction of underwater objects using the conversion model was 2.89 %. This study provides technical support for the 3D detection of underwater concrete.

Online 3D behavioral tracking of aquatic model organism with a dual-camera system

Behavioral tracking system of aquatic model organism is crucial for applications in aquaculture, environment and biomedicine, as it facilitates human to monitor subject states by automatically recognizing individual identities, and quantify their movement trajectories. Previous research has been devoted to this topic, but they are still not simple and effective enough. Therefore, this work introduces a novel online monitoring system implemented by dual-camera equipment and software modules consisting of an object detector and a multi-view multi-target tracker. The tracker provides the abilities of cross-view matching, underwater 3D reconstruction, and 3D target tracking. Specifically, our solution adopts a new paradigm, called tracking by early-reconstruction, which prioritizes the 3D reconstruction of targets’ coordinates on a frame-by-frame basis and then tracks them directly in 3D space rather than in a 2D image plane. This paradigm simplifies the complex multi-view tracking problem into a series of local association procedures, allowing us to achieve an online resolution through the iterative approach. To verify the effectiveness of the system, we employ zebrafish as the research subject, and evaluate the accuracy and robustness of the system on tracking benchmark, behavioral tasks and simulated data. Finally, we conducted extensive experiments and demonstrated the efficiency and effectiveness of the proposed system.

Underwater 3D reconstruction based on double N-step orthogonal polarization state phase shift strategy

The nonlinear response of projection equipment and the uneven reflectance of HDR (high dynamic range) object surface pose challenges in underwater 3D reconstruction. As a result, the phase-shifting fringe pattern collected by the camera does not have good sinusoidality, leading to phase errors. Therefore, this paper proposes a double N-step orthogonal polarization state phase-shift strategy (DOPS). This technique uses the degree of linear polarization (DOLP) of polarized light to design a double N-step phase-shifting fringe pattern. The original phase shift fringes are encoded as horizontal polarization fringes, and the additional phase shift fringes are encoded as vertical polarization fringes. Then, the camera captures and extracts the phase information of the two groups of DOLP patterns, which are separately subjected to phase unwrapping. The resulting unwrapped phases are fused to reduce phase error. The experimental results indicate that this method reduces the measurement error by 57 % when dealing with 3D shape measurement of underwater HDR objects, compared to the polarization-encoded phase shift algorithm (PEPS). Additionally, this method is more accurate in measurement accuracy and improves measurement efficiency by 50 % compared to the double N-step phase shift method (DNPS).

Three-dimentional reconstruction of underwater side-scan sonar images based on shape-from-shading and monocular depth fusion

Modern marine research requires high-precision three-dimensional (3D) underwater data. Underwater environments experience severe visible light attenuation, which leads to inferior imaging compared with air. In contrast, sound waves are less affected underwater; hence side-scan sonar is used for underwater 3D reconstruction. Typically, the shape-from-shading algorithm (SfS) is widely used to reconstruct surface normal or heights from side-scan sonar images. However, this approach has challenges because of global information loss and noise. To address these issues, this study introduces a surface-normal fusion method. Specifically, we propose a frequency separation SfS algorithm using a discrete cosine transform, which provides a surface-normal map with less noise. We then fuse the surface-normal map with a novel depth estimation network to achieve high-precision 3D reconstruction of underwater side-scan sonar images. We conducted experiments on synthetic, NYU-depth-v2, and real side-scan sonar datasets to demonstrate the effectiveness of the proposed method.

Overview of Underwater 3D Reconstruction Technology Based on Optical Images

At present, 3D reconstruction technology is being gradually applied to underwater scenes and has become a hot research direction that is vital to human ocean exploration and development. Due to the rapid development of computer vision in recent years, optical image 3D reconstruction has become the mainstream method. Therefore, this paper focuses on optical image 3D reconstruction methods in the underwater environment. However, due to the wide application of sonar in underwater 3D reconstruction, this paper also introduces and summarizes the underwater 3D reconstruction based on acoustic image and optical–acoustic image fusion methods. First, this paper uses the Citespace software to visually analyze the existing literature of underwater images and intuitively analyze the hotspots and key research directions in this field. Second, the particularity of underwater environments compared with conventional systems is introduced. Two scientific problems are emphasized by engineering problems encountered in optical image reconstruction: underwater image degradation and the calibration of underwater cameras. Then, in the main part of this paper, we focus on the underwater 3D reconstruction methods based on optical images, acoustic images and optical–acoustic image fusion, reviewing the literature and classifying the existing solutions. Finally, potential advancements in this field in the future are considered.

Underwater 3D reconstruction using a photometric stereo with illuminance estimation.

Underwater image sensing is directly affected by the change in illuminance towards a camera caused by the refraction of light in different media. In this study, the convergence of a near-field point light source in water is analyzed using a light propagation model. The photometric stereo (PS) formula is determined based on an accurate estimation of the illuminance entering the camera. An underwater PS system is designed to verify the proposed method's feasibility. The experimental results demonstrate improved accuracy in normal calculation. This helps achieve accurate underwater 3D reconstruction of objects that is suitable for underwater surface microdefect detection.

Correction model of linear structured light sensor in underwater environment

Underwater 3D reconstruction using linear structured light sensor is crucial for many applications. This paper presents a correction model of a linear structured light sensor and a calibration method for such a model. The model takes into account the distortion introduced to the system by the flat sealed glass and the water medium. This model takes the linear structured light sensor as a whole to analyze the influence of underwater multi-medium refraction on it, instead of considering the influence of underwater refraction on camera imaging alone. The correction model can be modeled knowing the thickness of the flat window, the refractive index of the air, the refractive index of the flat widow, the refractive index of the water, the coordinates of the camera pinhole center, and the distance between the sensor coordinate system and the flat glass interface. The present study reports results obtained using the real sensor in a water tank to reconstruct objects to assess the sensor's accuracy.

Adaptive depth constraint-based underwater binocular image feature matching

Objective In this paper, addressing sparse feature points and unique epipolar constraint, an adaptive depth constraint-based underwater feature matching (ADC-UFM) scheme is proposed. Methods By combining the features from accelerated segment test (FAST) operator with scale invariant feature transform (SIFT) descriptors such that the matching accuracy can be dramatically improved. By introducing the underwater refractive factor, the matching constraint model (MCM) can be effectively established, and thereby contributing to eliminating mismatched points. The adaptive threshold choosing (ATC) module is finely devised to extremely preserve image feature information in changeable underwater environments. Results Comprehensive experiments show that the proposed ADC-UFM scheme can outperform typical matching schemes including SIFT, speeded-up robust features (SURF) and SIFT feature matching based on underwater curve constraint (UCC-SIFT), which not only achieves 85.2% matching accuracy but also meets real-time requirements. Conclusion The results of this study can provide a reliable guarantee for the subsequent underwater 3D reconstruction based on binocular vision system.

Estimation of the Scale of Artificial Reef Sets on the Basis of Underwater 3D Reconstruction

Estimation of the Scale of Artificial Reef Sets on the Basis of Underwater 3D Reconstruction

Underwater Imaging System Based on Light Field Technology

The image quality might be degraded severely due to water scatterings, especially near-shore water scatterings. In addition, the degraded image reduces the accuracy of underwater 3D reconstruction. We propose an underwater imaging system based on light field camera array to obtain enhanced high-quality images and accurate three-dimensional reconstruction from harsh underwater environments. Both 2D-imaging and 3D-reconstruction using light-field imaging were studied by experiments. By refocusing and analyzing the light field images of the target through turbid water, we obtained accurate three-dimensional estimations in the underwater scene. These experiments confirm the reliability of our method, showing that the SSIM (structural similarity) value of the system in turbid water is 0.378, compared to 0.197, the SSIM value of conventional single-camera imaging in the experiments. Therefore, we verify that the light field imaging system is less affected by forward scattering of water and has a larger detective range than conventional single-camera imaging with any specific resolution.

Underwater High-Precision 3D Reconstruction System Based on Rotating Scanning.

This paper presents an underwater high-precision line laser three-dimensional (3D) scanning (LLS) system with rotary scanning mode, which is composed of a low illumination underwater camera and a green line laser projector. The underwater 3D data acquisition can be realized in the range of field of view of 50° (vertical) × 360° (horizontal). We compensate the refraction of the 3D reconstruction system to reduce the angle error caused by the refraction of light on different media surfaces and reduce the impact of refraction on the image quality. In order to verify the reconstruction effect of the 3D reconstruction system and the effectiveness of the refraction compensation algorithm, we conducted error experiments on a standard sphere. The results show that the system’s underwater reconstruction error is less than 0.6 mm within the working distance of 140 mm~2500 mm, which meets the design requirements. It can provide reference for the development of low-cost underwater 3D laser scanning system.

3D RECONSTRUCTION OF UNSTABLE UNDERWATER ENVIRONMENT WITH SFM USING SLAM

Abstract. The underwater environment has substantial properties for underwater research such as marine archaeology, monitoring coral reefs, and shipwrecks. SfM, as a major step of photogrammetry, has been widely used in the field. For a high 3D construction quality, images must have a clear visual sight environment and known orientations of the images. However, underwater images have various types of visual disturbances, but also GPS/INS, commonly used on the ground, are not accepted. Finding more feature points or using more images for SfM are solutions to the problems. However, these methods take high computational costs. An alternative to this problem is to provide the known orientations of the images. For a solution to provide known orientations of images, the presented method in this study uses visual SLAM that processes the localization of a vehicle system and mapping of surroundings. The experiment aims to verify whether SLAM improves the quality of underwater 3D reconstruction and the computation efficiency of SfM. We examine the two aqualoc datasets with the results of the number of cloud points, SfM processing time, and the number of matched images/total images and mean reprojection errors. The outcome shows SLAM-determined orientations improved the quality of 3D reconstruction and the computation efficiency of SfM with results of the increased number of point clouds and the decreased processing time.

Underwater Photogrammetry Application for Coral Reef Mapping and Monitoring

Underwater 3D reconstruction techniques such as underwater photogrammetry are the latest trend in low cost survey techniques. This article explains how the data acquisition process is up to the process of producing an underwater photo map. Automatic data processing for coral reef objects from the initial data acquisition stage to building 3D models from seabed objects to rendering in virtual reality. In this paper, underwater photogrammetry was applied with SfM (Structure from Motion) algorithm in performing 3D reconstruction of coral reefs that will be produce orthophoto map. Geodetic measurements also performed to the model for control point. The analysis will be divided into t-test and F-test. From the processing, coral reef modeling has obtained the result of a linear error which t-test produce <50% data, both at 95% and at 99 % confidence level. The F-test gain a positive value where the point value used in the linear error test on marker L and K, the value at α = 0.05. The 3D coral reef model in the resulting orthophoto still has weaknesses, for example the color appearance that depends on the level of clarity of the water as well as the camera filter used. However, in 2D display is sufficient and can be an alternative to other methods, for example, the LIT method.

Underwater image conversion method based on multi-layer plane refraction model with light field processing method

In order to adopt the conventional visual measurement technology to underwater application, this paper proposes an underwater image conversion method based on a multi-layer plane refraction model, which converts the underwater image into an air image. First, under the multi-layer plane refraction model, the imaging process of the underwater camera is modelled in the form of a four-dimensional light field parameterization, and the direction vector of the air image is calculated using the light field direction information. Then, the perspective projection transformation is used to obtain the corresponding pixel coordinates of the direction vector and the image plane. Finally, the transformed air image is obtained by interpolation method, using the mapping relation of the corresponding image point coordinates before and after the transformation. Simulation and experimental results show that the average error of the corrected image is 0.56 pixel, and the high-precision air image provides strong support for subsequent underwater 3D reconstruction.

Underwater Target Detection and 3D Reconstruction System Based on Binocular Vision.

To better solve the problem of target detection in marine environment and to deal with the difficulty of 3D reconstruction of underwater target, a binocular vision-based underwater target detection and 3D reconstruction system is proposed in this paper. Two optical sensors are used as the vision of the system. Firstly, denoising and color restoration are performed on the image sequence acquired by the vision of the system and the underwater target is segmented and extracted according to the image saliency using the super-pixel segmentation method. Secondly, aiming to reduce mismatch, we improve the semi-global stereo matching method by strictly constraining the matching in the valid target area and then optimizing the basic disparity map within each super-pixel area using the least squares fitting interpolation method. Finally, based on the optimized disparity map, triangulation principle is used to calculate the three-dimensional data of the target and the 3D structure and color information of the target can be given by MeshLab. The experimental results show that for a specific size underwater target, the system can achieve higher measurement accuracy and better 3D reconstruction effect within a suitable distance.

Two-view underwater 3D reconstruction for cameras with unknown poses under flat refractive interfaces

In an underwater imaging system, a refractive interface is introduced when a camera looks into the water-based environment, resulting in distorted images due to the refraction of light. Simply ignoring the refraction effect or using the lens radial distortion model causes erroneous 3D reconstruction. This paper deals with a general underwater imaging setup using two cameras, of which each camera is placed in a separate waterproof housing with a flat glass window. In order to handle refraction properly, a simplified refractive camera model is used in this paper. Based on two new concepts, namely the Ellipse of Refrax (EoR) and the Refractive Depth (RD) of a scene point, we derive two new formulations of the underwater known rotation structure and motion (SaM) problem. One gives a globally optimal solution and the other is robust to outliers. The constraint of known rotation is further relaxed by incorporating the robust known rotation SaM into a new hybrid optimization framework. Our method is able to simultaneously perform underwater camera calibration and 3D reconstruction automatically without using any calibration object or additional calibration device. In order to evaluate the performance and practical applicability of our method, extensive experiments using synthetic data, synthetically rendered images and real underwater images were carried out. The experimental results demonstrate that the proposed method can significantly improve the accuracy of the reconstructed 3D structure (within 0.78 mm for an object of dimension over 200 mm compared with the ground truth model captured by a land-based system) and of the system parameters for underwater applications. Compared with bundle adjustment using the refractive camera model initialized with traditional 3D reconstruction methods, our proposed optimization method has significantly better completeness and accuracy and lower 3D errors in the reconstructed models.

A Laser Line Auto-Scanning System for Underwater 3D Reconstruction.

In this study, a laser line auto-scanning system was designed to perform underwater close-range 3D reconstructions with high accuracy and resolution. The system changes the laser plane direction with a galvanometer to perform automatic scanning and obtain continuous laser strips for underwater 3D reconstruction. The system parameters were calibrated with the homography constraints between the target plane and image plane. A cost function was defined to optimize the galvanometer’s rotating axis equation. Compensation was carried out for the refraction of the incident and emitted light at the interface. The accuracy and the spatial measurement capability of the system were tested and analyzed with standard balls under laboratory underwater conditions, and the 3D surface reconstruction for a sealing cover of an underwater instrument was proved to be satisfactory.

Underwater 3D Reconstruction for Underwater Construction Robot Based on 2D Multibeam Imaging Sonar

This paper presents an underwater structure 3D reconstruction method using a 2D multibeam imaging sonar. Compared with other underwater environmental recognition sensors, the 2D multibeam imaging sonar offers high resolution images in water with a high turbidity level by showing the reflection intensity data in real-time. With such advantages, almost all underwater applications, including ROVs, have applied this 2D multibeam imaging sonar. However, the elevation data are missing in sonar images, which causes difficulties with correctly understanding the underwater topography. To solve this problem, this paper concentrates on the physical relationship between the sonar image and the scene topography to find the elevation information. First, the modeling of the sonar reflection intensity data is studied using the distances and angles of the sonar beams and underwater objects. Second, the elevation data are determined based on parameters like the reflection intensity and shadow length. Then, the elevation information is applied to the 3D underwater reconstruction. This paper evaluates the presented real-time 3D reconstruction method using real underwater environments. Experimental results are shown to appraise the performance of the method. Additionally, with the utilization of ROVs, the contour and texture image mapping results from the obtained 3D reconstruction results are presented as applications.

A structured light method for underwater surface reconstruction

A new structured-light method for 3D imaging has been developed which can simultaneously estimate both the geometric shape of the water surface and the geometric shape of underwater objects. The method requires only a single image and thus can be applied to dynamic as well as static scenes. Experimental results show the utility of this method in non-invasive underwater 3D reconstruction applications. The performance of the new method is studied through a sensitivity analysis for different parameters of the suggested method.

3D underwater scene reconstruction through descattering and colour correction

This paper describes a novel method for ocean scene 3D reconstruction. While light is travelling through water, light rays are distorted depending on the wavelength. That is, absorption, scattering and colour distortion are three major distortion issues for underwater optical imaging. Scattering is caused by large suspended particles, as in turbid water that contains abundant particles, which causes the degradation of the captured image. Colour distortion corresponds to the varying degrees of attenuation encountered by light travelling in water at different wavelengths, causing ambient underwater environments to be dominated by a bluish tone. Our key contributions proposed here include a novel deep-sea imaging model to compensate for the attenuation discrepancy along the propagation path and an effective underwater scene 3D reconstruction method. The recovered 3D images are characterised by a reduced noise level, better exposure of the dark regions, and improved global contrast where the finest details and edges are enhanced significantly.