Underwater Imaging System Research Articles

Underwater Imaging System Based on Light Field Technology

The image quality might be degraded severely due to water scatterings, especially near-shore water scatterings. In addition, the degraded image reduces the accuracy of underwater 3D reconstruction. We propose an underwater imaging system based on light field camera array to obtain enhanced high-quality images and accurate three-dimensional reconstruction from harsh underwater environments. Both 2D-imaging and 3D-reconstruction using light-field imaging were studied by experiments. By refocusing and analyzing the light field images of the target through turbid water, we obtained accurate three-dimensional estimations in the underwater scene. These experiments confirm the reliability of our method, showing that the SSIM (structural similarity) value of the system in turbid water is 0.378, compared to 0.197, the SSIM value of conventional single-camera imaging in the experiments. Therefore, we verify that the light field imaging system is less affected by forward scattering of water and has a larger detective range than conventional single-camera imaging with any specific resolution.

Underwater hyperspectral imaging system using a prism-grating-prism structure.

An underwater hyperspectral imager (UHI) detection system is designed and developed, and a push-broom scan imaging mode was used. The telescope system adopts an image-side telecentric design with 22 mm focal length and 11∘×11∘ field of view. A prism-grating-prism structure was used as the spectral unit of spectrometer system to reduce spectral line bending. The UHI system reaches 5 nm spectral resolution in spectral range from 400 nm to 800 nm, and its modulation transfer function is greater than 0.62 at characteristic frequency in spectral range. An underwater hyperspectral detection experiment was carried out with an electric turntable, and the UHI system is of good imaging quality and spectral resolution, being able to distinguish various underwater targets.

Classical and state-of-the-art approaches for underwater image defogging: a comprehensive survey

In underwater scenes, the quality of the video and image acquired by the underwater imaging system suffers from severe degradation, influencing target detection and recognition. Thus, restoring real scenes from blurred videos and images is of great significance. Owing to the light absorption and scattering by suspended particles, the images acquired often have poor visibility, including color shift, low contrast, noise, and blurring issues. This paper aims to classify and compare some of the significant technologies in underwater image defogging, presenting a comprehensive picture of the current research landscape for researchers. First we analyze the reasons for degradation of underwater images and the underwater optical imaging model. Then we classify the underwater image defogging technologies into three categories, including image restoration approaches, image enhancement approaches, and deep learning approaches. Afterward, we present the objective evaluation metrics and analyze the state-of-the-art approaches. Finally, we summarize the shortcomings of the defogging approaches for underwater images and propose seven research directions.

Energy-Efficient Image Recognition System for Marine Life

This article focuses on designing an energy-efficient image recognition system for marine monitoring. One of the main challenges of an underwater imaging system is the strict power consumption constraints due to the limited on-site resources. Considering the need for continuous operation in different water turbidity levels and background illumination conditions, an energy-efficient approach is needed for the effective utilization of the resources. In this work, we propose a recognition framework that will adaptively adjust the system parameters, such as camera frame rate and LED illumination level, based on the environmental conditions to optimize the energy consumption while ensuring a high recognition accuracy. The first part of the proposed decision system contains the convolutional neural network (CNN)-based animal recognition block which is used for obtaining the confidence level for a single frame. The second part is the adaptive decision block that dynamically changes the system parameters and combines the results of the recognition block for multiple frames based on the environmental conditions. In our experiments, we have used nearly 8000 underwater images for training and testing the single frame recognition block and used nearly 200 different video sequences for training and testing the adaptive decision block. Based on measurements of a hardware framework composed of a Raspberry Pi 3 Model B, a Pi NoIR Camera v2.1, and 850 nm LEDs, the proposed system achieves up to 92.7% energy savings with a comparable recognition performance by dynamically changing the frame rate and emitted light intensity based on water turbidity and background illumination level.

Calibration of underwater fisheye camera with multi-frequency heterodyne technique based on light field

In this paper, a novel approach is proposed to calibrate an underwater fisheye camera with the multi-frequency heterodyne technique based on the light field. First, an underwater fisheye camera imaging model based on the light field is established to demonstrate the multiple refractions of light in underwater propagation and the extraordinary imaging characteristics of the fisheye lens, which shows better conciseness and organization. Then, the calibration method applied in the underwater fisheye camera imaging system is proposed to determine the mapping of the light field and the pose of the underwater fisheye camera. The light field mapping of the fisheye lens is obtained with the assistance of the multi-frequency heterodyne technique. Finally, an underwater fisheye camera is developed and the experimental results are obtained to validate the correctness and feasibility of the proposed method. Our underwater imaging system proves capable of achieving higher accuracy in the context of underwater 3D measurement.

Automated Activity Estimation of the Cold-Water Coral Lophelia pertusa by Multispectral Imaging and Computational Pixel Classification

AbstractThe cold-water coral Lophelia pertusa builds up bioherms that sustain high biodiversity in the deep ocean worldwide. Photographic monitoring of the polyp activity represents a helpful tool to characterize the health status of the corals and to assess anthropogenic impacts on the microhabitat. Discriminating active polyps from skeletons of white Lophelia pertusa is usually time consuming and error prone due to their similarity in color in common red–green–blue (RGB) camera footage. Acquisition of finer-resolved spectral information might increase the contrast between the segments of polyps and skeletons, and therefore could support automated classification and accurate activity estimation of polyps. For recording the needed footage, underwater multispectral imaging systems can be used, but they are often expensive and bulky. Here we present results of a new, lightweight, compact, and low-cost deep-sea tunable LED-based underwater multispectral imaging system (TuLUMIS) with eight spectral channels. A branch of healthy white Lophelia pertusa was observed under controlled conditions in a laboratory tank. Spectral reflectance signatures were extracted from pixels of polyps and skeletons of the observed coral. Results showed that the polyps can be better distinguished from the skeleton by analysis of the eight-dimensional spectral reflectance signatures compared to three-channel RGB data. During a 72-h monitoring of the coral with a half-hour temporal resolution in the laboratory, the polyp activity was estimated based on the results of the multispectral pixel classification using a support vector machine (SVM) approach. The computational estimated polyp activity was consistent with that of the manual annotation, which yielded a correlation coefficient of 0.957.

Three-dimensional cross-shelf zooplankton distributions off the Central Oregon Coast during anomalous oceanographic conditions

The Northern California Current (NCC) is a complex, dynamic system experiencing distinctly different levels of upwelling and downwelling, ranging from intermittent upwelling in summer to downwelling in winter. In recent years, warm water anomalies along the Oregon coast have had significant effects on coastal plankton assemblages. To resolve some of the fine-scale responses to these conditions, we used a towed, undulating underwater imaging system to investigate fine-scale (1 m vertical) zoo- and ichthyoplankton distributions along a 57-km section parallel to the Newport Hydrographic Line encompassing the shelf, shelf break, and slope off the central coast of Oregon. A sparse Convolutional Neural Network was used to automate the identification of 52 million plankton images of 64 plankton taxa, ranging from protists to copepods, larval fishes, and gelatinous organisms. Taxa distributions were interpolated over the whole transect, providing unprecedented insight into their horizontal and vertical distributions and revealing seven broad patterns of distribution. Additional fine-scale distribution data enable examination of some of the physical and biological processes underlying these fine-scale distribution patterns, building upon the historical time series data that exists for this region and advancing our knowledge of planktonic processes in this productive region of the NCC.

Software for tracking aquatic environment geographic information systems for further integration with advanced search capabilities

A close inspection of existing software for relatively accessible underwater imaging and data-gathering systems reveals its inadequacy for conducting search operations and subsequent map making. In order to broaden the application of existing recreational and low-cost echo sounders and side scan sonar (SSS), the authors of this paper propose a program, which is part of a specialized software and hardware complex under implementation. Being the initial stage of the project, the proposed program is capable of visualizing routes on a graph and displaying the resulting trajectory using a GNSS receiver, further synchronizing it with hydrographic data from a sonar device.

Underwater Hyperspectral Target Detection with Band Selection

Compared to multi-spectral imagery, hyperspectral imagery has very high spectral resolution with abundant spectral information. In underwater target detection, hyperspectral technology can be advantageous in the sense of a poor underwater imaging environment, complex background, or protective mechanism of aquatic organisms. Due to high data redundancy, slow imaging speed, and long processing of hyperspectral imagery, a direct use of hyperspectral images in detecting targets cannot meet the needs of rapid detection of underwater targets. To resolve this issue, a fast, hyperspectral underwater target detection approach using band selection (BS) is proposed. It first develops a constrained-target optimal index factor (OIF) band selection (CTOIFBS) to select a band subset with spectral wavelengths specifically responding to the targets of interest. Then, an underwater spectral imaging system integrated with the best-selected band subset is constructed for underwater target image acquisition. Finally, a constrained energy minimization (CEM) target detection algorithm is used to detect the desired underwater targets. Experimental results demonstrate that the band subset selected by CTOIFBS is more effective in detecting underwater targets compared to the other three existing BS methods, uniform band selection (UBS), minimum variance band priority (MinV-BP), and minimum variance band priority with OIF (MinV-BP-OIF). In addition, the results also show that the acquisition and detection speed of the designed underwater spectral acquisition system using CTOIFBS can be significantly improved over the original underwater hyperspectral image system without BS.

An Extrinsic Calibration Method With Closed-Form Solution for Underwater Opti-Acoustic Imaging System

Forward-looking sonar (FLS) and camera are widely used sensors in multimodal systems for underwater environments. The extrinsic calibration is a precondition for this kind of multimodal systems to fuse optical and acoustic information. Due to the very different imaging formation mechanisms and models of 3-D-to-2-D projection for imaging sonar and optical camera, the multimodal correspondence and the sonar-pose recovery from a 2-D sonar image are two main issues in extrinsic calibration work. In this article, we propose an extrinsic calibration method for the opti-acoustic imaging system combining a camera and an FLS in underwater environment. The proposed method includes an estimation process yielding closed-form expressions to determine the sonar pose with respect to the calibration board, which enables the estimation of extrinsic parameters to be directly computed from sensor measurements and the knowledge of calibration-board structure. A calibration board with two sets of feature arrays is used to generate two 2-D-to-3-D correspondences, which circumvents the 2-D-to-2-D multimodal correspondence problem. An iterative optimization process is provided to further improve the performance, while the extra computation time is negligible. Both simulated and actual experiments show that the method outperforms the existing iterative method based on bundle adjustment.

Prey and predator overlap at the edge of a mesoscale eddy: fine-scale, in-situ distributions to inform our understanding of oceanographic processes

Eddies can enhance primary as well as secondary production, creating a diverse meso- and sub-mesoscale seascape at the eddy front which can affect the aggregation of plankton and particles. Due to the coarse resolution provided by sampling with plankton nets, our knowledge of plankton distributions at these edges is limited. We used a towed, undulating underwater imaging system to investigate the physical and biological drivers of zoo- and ichthyoplankton aggregations at the edge of a decaying mesoscale eddy (ME) in the Straits of Florida. Using a sparse Convolutional Neural Network we identified 132 million images of plankton. Larval fish and Oithona spp. copepod concentrations were significantly higher in the eddy water mass, compared to the Florida Current water mass, only four days before the ME's dissipation. Larval fish and Oithona distributions were tightly coupled, indicating potential predator-prey interactions. Larval fishes are known predators of Oithona, however, Random Forests models showed that Oithona spp. and larval fish concentrations were primarily driven by variables signifying the physical footprint of the ME, such as current speed and direction. These results suggest that eddy-related advection leads to largely passive overlap between predator and prey, a positive, energy-efficient outcome for predators at the expense of prey.

An Airborne Sonar System for Underwater Remote Sensing and Imaging

High-resolution imaging and mapping of the ocean and its floor has been limited to less than 5% of the global waters due to technological barriers. Whereas sonar is the primary contributor to existing underwater imagery, the water-based system is limited in spatial coverage due to its low imaging throughput. On the other hand, aerial synthetic aperture radar systems have provided high-resolution imaging of the entire earth's landscapes but are incapable of deep penetration into water. In this work, we present a proof-of-concept system which bridges the gap between electromagnetic imaging in air and sonar imaging in water through the laser-induced photoacoustic effect and high-sensitivity airborne ultrasonic detection. Here, we use air-coupled capacitive micromachined ultrasonic transducers (CMUTs) which is a critical differentiator from previous works and has enabled the acquisition of an underwater image from a fully airborne acoustic imaging system - a task that has yet to be accomplished in the literature. With the entire imaging system located on an airborne platform, there is much promise for the scalability of our system to one which could perform high-throughput imaging of underwater in large-scale deployment. Non-contact acoustic-based imaging modalities are also of much interest to the medical imaging and non-destructive testing communities. Incorporating air-coupled transducers, for example CMUTs, or other resonant sensors in these applications could be aided by the analysis presented throughout this work.

基于LCTF的水下光谱成像系统研制

Underwater spectral imaging technology plays an important role in underwater object recognition and ocean ecological monitoring. An underwater spectral imaging system using liquid crystal tunable filter (LCTF) was designed based on the actual engineering environment. The system obtained spectral information by using LCTF as a filter structure. Under the illumination of a wide-spectrum LED, a pool experiment was performed to obtain 31 channel spectral images of the target between 400 and 700 nm. The spectral information of objects with similar colors under water was discussed and analyzed. The results show that the system is helpful for underwater target recognition and classification. In-situ observation of corals in the sea trial successfully obtained underwater spectral images of coral reefs. The system is expected to be applied to ocean remote sensing, ocean ecological environment monitoring and other fields.

PelagiCam: a novel underwater imaging system with computer vision for semi-automated monitoring of mobile marine fauna at offshore structures.

Engineered structures in the open ocean are becoming more frequent with the expansion of the marine renewable energy industry and offshore marine aquaculture. Floating engineered structures function as artificial patch reefs providing novel and relatively stable habitat structure not otherwise available in the pelagic water column. The enhanced physical structure can increase local biodiversity and benefit fisheries yet can also facilitate the spread of invasive species. Clear evidence of any ecological consequences will inform the design and placement of structures to either minimise negative impacts or enhance ecosystem restoration. The development of rapid, cost-effective and reliable remote underwater monitoring methods is crucial to supporting evidence-based decision-making by planning authorities and developers when assessing environmental risks and benefits of offshore structures. A novel, un-baited midwater video system, PelagiCam, with motion-detection software (MotionMeerkat) for semi-automated monitoring of mobile marine fauna, was developed and tested on the UK's largest offshore rope-cultured mussel farm in Lyme Bay, southwest England. PelagiCam recorded Atlantic horse mackerel (Trachurus trachurus), garfish (Belone belone) and two species of jellyfish (Chrysaora hysoscella and Rhizostoma pulmo) in open water close to the floating farm structure. The software successfully distinguished video frames where fishes were present versus absent. The PelagiCam system provides a cost-effective remote monitoring tool to streamline biological data acquisition in impact assessments of offshore floating structures. With the rise of sophisticated artificial intelligence for object recognition, the integration of computer vision techniques should receive more attention in marine ecology and has great potential to revolutionise marine biological monitoring.

Refractive Two-View Reconstruction for Underwater 3D Vision

Recovering 3D geometry from cameras in underwater applications involves the Refractive Structure-from-Motion problem where the non-linear distortion of light induced by a change of medium density invalidates the single viewpoint assumption. The pinhole-plus-distortion camera projection model suffers from a systematic geometric bias since refractive distortion depends on object distance. This leads to inaccurate camera pose and 3D shape estimation. To account for refraction, it is possible to use the axial camera model or to explicitly consider one or multiple parallel refractive interfaces whose orientations and positions with respect to the camera can be calibrated. Although it has been demonstrated that the refractive camera model is well-suited for underwater imaging, Refractive Structure-from-Motion remains particularly difficult to use in practice when considering the seldom studied case of a camera with a flat refractive interface. Our method applies to the case of underwater imaging systems whose entrance lens is in direct contact with the external medium. By adopting the refractive camera model, we provide a succinct derivation and expression for the refractive fundamental matrix and use this as the basis for a novel two-view reconstruction method for underwater imaging. For validation we use synthetic data to show the numerical properties of our method and we provide results on real data to demonstrate its practical application within laboratory settings and for medical applications in fluid-immersed endoscopy. We demonstrate our approach outperforms classic two-view Structure-from-Motion method relying on the pinhole-plus-distortion camera model.

2D Ultrasonic Antenna System for Imaging in Liquid Sodium.

Ultrasonic techniques are developed at CEA (French Alternative Energies and Nuclear Energy Commission) for in-service inspection of sodium-cooled reactors (SFRs). Among them, an ultrasound imaging system made up of two orthogonal antennas and originally based on an underwater imaging system is studied for long-distance vision in the liquid sodium of the reactor’s primary circuit. After a description of the imaging principle of this system, some results of a simulation study performed with the software CIVA in order to optimize the antenna parameters are presented. Then, experimental measurements carried out in a water tank illustrate the system capabilities. Finally, the limitations of the imaging performances and the ongoing search of solutions to address them are discussed.

Design of Low-Complexity 3-D Underwater Imaging System With Sparse Planar Arrays

The development of a real-time 3-D underwater imaging system with a planar array is restricted by the huge hardware cost and computational burden associated with the large number of elements required. In this paper, a new design of low-complexity 3-D underwater imaging system is proposed, involving two key parts: a special sparse array with low hardware cost and an efficient beamforming with low computational burden. Especially, the simulated annealing algorithm is optimized based on distributed and parallel subarray (DPS) beamforming and results in fewer active elements than existing methods. To improve the computational efficiency especially for sparse arrays, an optimized DPS beamforming is presented with the pruning fast Fourier transform technology. To test the validity of the design, a real system based on the proposed methods was devised and employed in the lake and sea trials. The experiment results demonstrate that the low-complexity 3-D underwater imaging system can provide a satisfactory imaging quality.

Multi-adaptive Feature Extraction Method of Marine Plant Information in Internet of Things

Xing, J.; Wang, X., and Dong, J., 2019. Multi-adaptive feature extraction method of marine plant information in Internet of Things. In: Guido-Aldana, P.A. and Mulahasan, S. (eds.), Advances in Water Resources and Exploration. Journal of Coastal Research, Special Issue No. 93, pp. 495–500. Coconut Creek (Florida), ISSN 0749-0208.In ocean monitoring and development, underwater image processing and recognition has gradually become a hot topic in ocean research. Because of the complex shooting environment, the underwater image is often seriously distorted. The absorption of water will lead to the attenuation of light in the transmission process, and the scattering of water will make the whole image appear atomization effect. According to the characteristics of underwater light propagation and underwater imaging system, the characteristics of underwater noise are analyzed. Butterworth high pass filter is used to smooth the backscatter noise of low frequency underwater image. On the basis of the above, SIFT feature points are extracted from the improved gray image, and K-nearest neighbor algorithm is used to match the feature points extracted from the standard image data set to obtain the information features of marine plants. The experimental results show that this method can obtain more feature points of marine plant information, and at the same time guarantee the accuracy of feature extraction time.

A joint adaptive evolutionary model towards optical image contrast enhancement and geometrical reconstruction approach in underwater remote sensing

Nowadays, the underwater optical imaging is used by a wide range of sciences including marine scientists for studying underwater structures and organisms, oil and gas companies, telecommunications to monitor fuel transmission lines and cables, and military centers to detect the sea mines and submarines. Images with non-uniform brightness are visually limited due to low ambient light during imaging or excessive exposure, fogging, hazing, or a combination of these factors. As a result, a process of imaging improvement is highly complex with some conditions. Particularly, this issue is harmful for the underwater environments as well as its highly complexity. In this paper, we improve the underwater images from different environments by exploiting image processing techniques including elimination of fogging in the image, and improvement of contrast with combining super resolution methods which result in significant enhancement of the image quality. To remove fogging in the image enhancement method, a variational-based fusion method was proposed without increasing or decreasing image resolution to increase the adaptability of non-uniform backlight images. This approach is optimized by the bee colony algorithm. Additionally, weighting coefficient selection method is used for super resolution of image after adjusting contrast. The main reason for using hybrid contrast techniques includes the adaptive enhancement and color correction. The bee colony algorithm and weighting coefficient in super resolution due to the lack of hybrid methods in visually improved water images has been the subject of recent researches. The hybrid set has provided more effect on different images at various conditions. Having applied the algorithm set for low-quality visual image set of underwater, some analyses such as PSNR, MSE and SSIM are tested. Compared to similar solutions, the quality is significantly improved and highlighted the exhaustive operation of the algorithm. Applying the presented algorithm in the software part of underwater imaging systems is widely effective for the accurate analysis of underwater images.

Design of an Ultrasonic Phased Array Launch System Based on Gigabit Ethernet Transmission

To better use ultrasonic phased array detection technology for underwater object detection and real-time imaging, an ultrasonic phased array launch system based on Gigabit Ethernet transmission was proposed in this work. The principle of phased focusing emission and the UDP communication protocol are analysed. Due to abundant I/O pin resources and internal logic resources, FPGA is used to excite the 32-channel ultrasonic phased array transducer, and the communication protocol is Gigabit Ethernet UDP. Generally, the delay parameters are sent to the FPGA in real time and high-speed, and then transmitted to the pulse signals. By using an integrated chip Max4940, the pulse signals are amplified and delivered to the phased array transducers. According to experiment, it is conformed that this system can transmit a large amount of data in real time at high speed; realize phase-controlled focus transmission, and the delay precision reaches 2.5 ns. This study provides a good hardware foundation for underwater imaging systems.