Underwater Image Processing Research Articles

Computer vision and deep learning for fish classification in underwater habitats: A survey

AbstractMarine scientists use remote underwater image and video recording to survey fish species in their natural habitats. This helps them get a step closer towards understanding and predicting how fish respond to climate change, habitat degradation and fishing pressure. This information is essential for developing sustainable fisheries for human consumption, and for preserving the environment. However, the enormous volume of collected videos makes extracting useful information a daunting and time‐consuming task for a human being. A promising method to address this problem is the cutting‐edge deep learning (DL) technology. DL can help marine scientists parse large volumes of video promptly and efficiently, unlocking niche information that cannot be obtained using conventional manual monitoring methods. In this paper, we first provide a survey of computer visions (CVs) and DL studies conducted between 2003 and 2021 on fish classification in underwater habitats. We then give an overview of the key concepts of DL, while analysing and synthesizing DL studies. We also discuss the main challenges faced when developing DL for underwater image processing and propose approaches to address them. Finally, we provide insights into the marine habitat monitoring research domain and shed light on what the future of DL for underwater image processing may hold. This paper aims to inform marine scientists who would like to gain a high‐level understanding of essential DL concepts and survey state‐of‐the‐art DL‐based fish classification in their underwater habitat.

Light Attenuation Prior based Underwater Image Enhancement

Images captured underwater usually suffer from color distortion, detail blurring, low contrast, and a bluish or greenish tone due to light scattering and absorption in the underwater medium, which in turn affects the visibility adversely. Underwater image processing schemes are broadly categorized into two groups, restoration methods and enhancement methods. The approach of restoration methods is to assume the effects of underwater environment as degradation but in enhancement, this environment is assumed to be natural and tries to enhance the visual information to next extent. Restoration schemes are proved to give better performance than enhancement schemes. These restoration schemes are further classified as optical imaging methods, polarization methods and prior knowledge methods. The key problems faced by these schemes are excessive optimization parameters, difficulty of recognizing artificial lighting, adapting to multi-scatter scenario, red artifacts and over exposure. A large number of schemes are proposed in the literature under these categories and most of them suffer from one or more of the above-mentioned issues. In this paper, a rapid and effective scene depth estimation model will be proposed based on underwater light attenuation prior for underwater images and train the model coefficients with learning-based supervised linear regression. With the correct depth map, the background light and transmission maps for R-G-B light are easily estimated to recover the true scene radiance under the water.

Severely Degraded Underwater Image Enhancement with a Wavelet-based Network

Underwater images are important in marine science and ocean engineering fields owing to giving color information, low cost, and compact. Yet obtained underwater images are often degraded and restoring and enhancing wavelength selective signal attenuation of underwater images depending on complex underwater physical process is essential in practical application. While recently developed deep learning is a promising choice, constructing sufficiently large dataset covering whole real images is challenging, peculiar to underwater image processing. In order to supplement relatively small dataset, previous studies alternatively construct an artificial underwater image dataset based on a physical model or Generative Adversarial Network. Also, incorporating traditional signal processing methods into the network architecture has shown promising success, though enhancement of severely degraded underwater images remains to be a big issue. In this paper, we tackle underwater image enhancement based on an encoder-decoder based deep learning model incorporating discrete wavelet transform and whitening and coloring transform. We also construct a severely degraded real underwater image dataset. The presented model shows excellent results both qualitatively and quantitatively in the artificial and real image dataset. Constructed dataset is available at https://github.com/tkswalk/2022-IJACSA.

Analysis of Underwater Image Processing Methods for Annotation in Deep Learning Based Fish Detection

With the advent of deep-learning (DL) techniques, image annotation has become a fundamental part of the research process. In the case of underwater image annotation, the human in charge of the task is faced whith the inherent quality problems of this kind of images. A large number of underwater image enhancement (UIE) methods have been developed aimed at improving the colors and contrast of these images. However, no attention has been paid to the specific problem of image annotation. In this case the global image quality of the processed image is less important than the fact that the objects to be annotated stand out and that their contours are easy to delineate. In this paper we evaluate seven state-of-the-art UIE techniques and rank them, through a subjective approach, according to their utility for the annotation process. The conclusion of our study is that, in general, the model-free Multiscale Retinex algorithm is preferred over more complex techniques that try to model the formation of the underwater images.

Fractal Dimension as an Effective Feature for Characterizing Hard Marine Growth Roughness from Underwater Image Processing in Controlled and Uncontrolled Image Environments

Hard marine growth is an important process that affects the design and maintenance of floating offshore wind turbines. A key parameter of hard biofouling is roughness since it considerably changes the level of drag forces. Assessment of roughness from on-site inspection is required to improve updating of hydrodynamic forces. Image processing is rapidly developing as a cost effective and easy to implement tool for observing the evolution of biofouling and related hydrodynamic effects over time. Despite such popularity; there is a paucity in literature to address robust features and methods of image processing. There also remains a significant difference between synthetic images of hard biofouling and their idealized laboratory approximations in scaled wave basin testing against those observed in real sites. Consequently; there is a need for such a feature and imaging protocol to be linked to both applications to cater to the lifetime demands of performance of these structures against the hydrodynamic effects of marine growth. This paper proposes the fractal dimension as a robust feature and demonstrates it in the context of a stereoscopic imaging protocol; in terms of lighting and distance to the subject. This is tested for synthetic images; laboratory tests; and real site conditions. Performance robustness is characterized through receiver operating characteristics; while the comparison provides a basis with which a common measure and protocol can be used consistently for a wide range of conditions. The work can be used for design stage as well as for lifetime monitoring and decisions for marine structures, especially in the context of offshore wind turbines.

Denoising, Edge Aware Restoration and Enhancement of Single Shallow Coastal Water Image

The dissolved particles and their resultant scattering are the underlying cause of low contrast and blur thereby producing poor quality underwater images. Single-shot shallow coastal underwater images are much in need of the preprocessing steps viz. image enhancement and restoration. The underwater image processing operations like classification, object detection and computer vision require an enhancement/restoration preprocessing. The paper aims to restore the visibility of objects in turbid water images with its effective edge aware restoration cum enhancement model. The restricted rolling guidance filter on the DCP-based restoration method produces better edge aware restoration and denoised output image. The model-based (MB) dark channel prior (DCP) along with an edge emphasis and contrast enhancement achieves the essential dehazing and improvement in contrast for the heavily blurred underwater images. The subjective and objective projections are an evidence for the same. This edge preserving and denoising nature of the model is also exhibited with comparisons made with promising algorithms over the decade.

Underwater image processing method based on red channel prior and Retinex algorithm

Due to light being absorbed by water and scattered by suspended particles, color distortion, low image contrast, and detail loss are the common problems found in underwater images. We present an underwater image processing method, which based on an underwater imaging model, makes initial image restoration with the improved red channel prior combining transmission reduction, background light correction, and color compensation to acquire clearer underwater images, and then combines multiscale retinex (MSR) enhancement and hue saturation value to enhance image color contrast and saturation. Results indicate that, compared with the original images, the method proposed in this paper is capable of significantly enhancing image definition, saturation, contrast, and luminance. In comparison with dark channel prior, single-scale retinex (SSR), MSR, and multiscale retinex with color restoration (MSRCR), this method greatly improved the average gradient of images while keeping the information entropy of images unchanged. The average gradient calculated with the proposed method increased by 239%, 199%, 200%, and 220% for the three given images, respectively, compared with the other methods above.

Underwater optical image processing based on double threshold judgements and optimized red dark channel prior method

Underwater optical image processing based on double threshold judgements and optimized red dark channel prior method

Underwater Image Enhancement With Optimal Histogram Using Hybridized Particle Swarm and Dragonfly

Abstract Typically, underwater image processing is mainly concerned with balancing the color change distortion or light scattering. Various researches have been processed under these issues. This proposed model incorporates two phases, namely, contrast correction and color correction. Moreover, two processes are involved within the contrast correction model, namely: (i) global contrast correction and (ii) local contrast correction. For the image enhancement, the main target is on the histogram evaluation, and therefore, the optimal selection of histogram limit is very essential. For this optimization purpose, a new hybrid algorithm is introduced namely, swarm updated Dragonfly Algorithm, which is the hybridization of Particle Swarm Optimization (PSO) and Dragonfly Algorithm (DA). Further, this paper mainly focused on Integrated Global and Local Contrast Correction (IGLCC). The proposed model is finally distinguished over the other conventional models like Contrast Limited Adaptive Histogram, IGLCC, dynamic stretching IGLCC-Genetic Algorithm, IGLCC-PSO, IGLCC- Firefly and IGLCC-Cuckoo Search, IGLCC-Distance-Oriented Cuckoo Search and DA, and the superiority of the proposed work is proved.

Edge Aware Turbidity Restoration of Single Shallow Coastal Water Image

The blur and low contrast of underwater images are indicative of the high noise, intense scattering and ultimately the low quality of such images. Image Enhancement and Restoration are imperative pre processing steps for single images from shallow coastal areas. Pre processing of underwater images is hence a prerequisite to processes like classification, object detection and computer vision. The paper presents an effective edge aware restoration models that goals turbid water images. The dark channel based restoration method with a rolling guidance filter gives a more edge aware restored and denoised version of the heavily blurred, low contrast shallow coastal image. This is evident from the subjective and objective projections. Moreover the edge preserving nature projects 13.05% and 5.53% more than UHP (which is in close) in terms of number of edges and UCIQE scores in comparisons with promising algorithms over the decade.

Survey Paper on Visibility Restoration of Underwater Optical Images and Enhancement Techniques

We have investigated the problem of underwater hazy image enhancement and restoration in this paper studied. Underwater image processing has several applications in the field of oceanic research work and scientific applications such as archaeology, geology, underwater environmental assessment, laying of long distance gas pipelines and communication links across the continents which demand geo-referential surveying of the oceanic bed and prospection of ancient shipwreck. There are many difficulties for undersea optical imaging. To submerging a camera in underwater enough space is required. The maneuvering of the camera with the help from remote place or in person at the site is likewise a complex task. However, the major challenge is imposed by underwater medium properties. Underwater haze image enhancement has gained widespread importance with the rapid development of modern imaging equipment. Though, the contrast enhancement of single underwater hazy image is a cumbersome task for scientific exploration and computational application. At extreme depth, because of attenuation in light propagation, the underwater images are susceptible to inferior visibility.

Underwater Image Enhancement Method Combining color and Luminance

Background:: Aiming at the problems of color distortion, low clarity and poor visibility of underwater image caused by complex underwater environment, a wavelet fusion method UIPWF for underwater image enhancement is proposed. Methods:: First of all, an improved NCB color balance method is designed to identify and cut the abnormal pixels, and balance the color of R, G and B channels by affine transformation. Then, the color correction map is converted to CIELab color space, and the L component is equalized with contrast limited adaptive histogram to obtain the brightness enhancement map. Finally, different fusion rules are designed for low-frequency and high-frequency components, the pixel level wavelet fusion of color balance image and brightness enhancement image is realized to improve the edge detail contrast on the basis of protecting the underwater image contour. Results:: The experiments demonstrate that compared with the existing underwater image processing methods, UIPWF is highly effective in the underwater image enhancement task, improves the objective indicators greatly, and produces visually pleasing enhancement images with clear edges and reasonable color information. Conclusion:: The UIPWF method can effectively mitigate the color distortion, improve the clarity and contrast, which is applicable for underwater image enhancement in different environments.

Color correction and restoration based on multi-scale recursive network for underwater optical image

Underwater image processing has played an important role in various fields such as submarine terrain scanning, submarine communication cable laying, underwater vehicles, underwater search and rescue. However, there are many difficulties in the process of acquiring underwater images. Specifically, the water body will selectively absorb part of the light when light travels through the water, resulting in color degradation of underwater images. At the same time, due to the influence of floating substances in the water, the light has a certain degree of scattering, which will bring serious problems such as blurred details and low contrast to underwater images. Therefore, using image processing technology to restore the real appearance of underwater images has a high practical value. In order to solve the above problems, we combine the color correction method with the deblurring network to improve the quality of underwater images in this paper. Firstly, aiming at the problem of insufficient number and diversity of underwater image samples, a network combined with depth image reconstruction and underwater image generation is proposed to simulate underwater images based on the style transfer method. Secondly, for the problem of color distortion, we propose a dynamic threshold color correction method based on image global information combined with the loss law of light propagation in water. Finally, in order to solve the problem of image blurring caused by scattering and further improve the overall image clarity, the color-corrected image is reconstructed by a multi-scale recursive convolutional neural network. Experiment results show that we can obtain images closer to underwater style with shorter training time. Compared with several latest underwater image processing methods, the proposed method has obvious advantages in multiple underwater scenes. Simultaneously, we can restore the color information, remove blurring and boost detail for underwater images.

Multi-stream fish detection in unconstrained underwater videos by the fusion of two convolutional neural network detectors

Recently, marine biologists have begun using underwater videos to study species diversity and fish abundance. These techniques generate a large amount of visual data. Automatic analysis using image processing is therefore necessary, since manual processing is time-consuming and labor-intensive. However, there are numerous challenges to implementing the automatic processing of underwater images: for example, high luminosity variation, limited visibility, complex background, free movement of fish, and high diversity of fish species. In this paper, we propose two new fusion approaches that exploit two convolutional neural network (CNN) streams to merge both appearance and motion information for automatic fish detection. These approaches consist of two Faster R-CNN models that share either the same region proposal network or the same classifier. We significantly improve the fish detection performances on the LifeClef 2015 Fish benchmark dataset not only compared with the classic Faster R-CNN but also with all the state-of-the-art approaches. The best F-score and mAP measures are 83.16% and 73.69%, respectively.

A Fast Underwater Image Stitching Algorithm with Adaptive Adjustment of Attitude Angle

In underwater image acquisition process, due to the impact of water currents and other disturbances, the movement posture of the underwater machine will be unstable, which could lead to unusual problems such as twisting of underwater image capture. These factors will increase the error rate of feature point matching and lead to the failure of panoramic image mosaic. In this regard, we propose a new, highly applicable underwater image stitching algorithm. Firstly, the posture angle adjustment link is added to the underwater image processing, and the angle deflection problem of the underwater image is effectively improved by using the posture angle information. Secondly, the feature points of underwater images are extracted based on the accelerated robust feature (SURF) algorithm. Then, the reference image is matched with the feature points of the image to be registered, and effective feature point pairs are obtained by screening. Finally, the images are stitched based on OpenCV to obtain a good panoramic image. Afterexperimental analysis and comparison, our method can increase the number of matching feature point pairs between images. In addition, the Euclidean distance is significantly shortened during the matching process, which further makes the matching of feature points more accurate. Our method satisfactorily overcomes the adverse effects of actual underwater operations and has a better application prospect.

Underwater Image Restoration and Enhancement Based on a Fusion Algorithm With Color Balance, Contrast Optimization, and Histogram Stretching

A fusion algorithm is proposed for the restoration and enhancement of underwater images. Color balance, contrast optimization and histogram stretching are carried out. To alleviate the effect of color shift in an underwater image, the scalar values of R, G, B channels are renewed so that the distributions of the three channels in histogram are similar. Instead of refining the transmittance in dark channel prior based restoration, an optimized contrast algorithm is employed by which the optimal transmittance is determined. To further improve the brightness and contrast of underwater images, a histogram stretching algorithm based on the red channel is given. To verify the effectiveness of the proposed fusion algorithm, experimental underwater images are treated. Results show that the quality of underwater images is improved significantly, both in term of subjective visual effect and objective evaluation. The proposed underwater image processing strategy is also compared with some popular techniques. Comparison results indicate the advantage of the proposed strategy over others.

Underwater Pipeline Oil Spill Detection Based on Structure of Root and Branch Cells

The existing oil spill detection methods mainly rely on physical sensors or numerical models cannot locate the spill position accurately and in time. To solve this problem, combining with underwater image processing technology, an unsupervised detection algorithm for oil spill in underwater pipelines is proposed for the first time. First, the oil spill region to be detected is regarded as the moving target, and the foreground detection algorithm is applied to the processed images. Then, the HSV (Hue, Saturation, Value) color space of the image is used to screen the oil spill region meeting the threshold requirements. Next, the bitwise of foreground mask and HSV mask into cells are divided. Finally, according to the characteristics of the oil spill image, false detection is eliminated by classifying cells three times. After qualitative and quantitative analysis, it is proved that the proposed algorithm can detect oil spill region accurately.

Underwater image processing and analysis: A review

With increasing attentions being drawn to the underwater observation and utilization of marine resources in recent years, underwater image processing and analysis have become an active research hotspot. Different from the general images, marine environment is usually faced with some complicated situations such as underwater turbulence and diffusion, severe absorption and scattering of water body, various noises, low contrast, uniform illumination, monotonous color, complex underwater-background. In response to these typical challenges, a large body of works in underwater image processing has been exploited in recent years. This survey introduces a review of existing relatively mature and representative underwater image processing models, which are classified into seven categories including enhancement, fog removal, noise reduction, segmentation, salient object detection, color constancy and restoration. We then objectively evaluate the current situations and future development trend of underwater image processing, and provide some insights into the prospective research directions to promote the development of underwater vision and beyond.

A multi-scale features-based method to detect Oplegnathus

It is of great significance to use underwater video and image processing technology to detect and analyze fish behaviors. In this paper, an Oplegnathus image dataset for fish behaviors study by deep learning algorithm is constructed, and the data is captured from two cameras (one above water and another below water); and then an improved Neural Network model based on multi-scale features is proposed for fish behaviors learning automatically. To overcome the occlusion and blur problems of the images, the lightweight neural network MobileNet-SSD is improved by adding a dilate convolution, and SE blocks are added to the feature maps at different scales to establish a self-attention mechanism; the Focal Loss function is used to calculate the classification loss and to balance the proportion of background and target samples. The results of the experiments show that the average behaviors detection accuracy of our method reach 90.94% and 88.36% in both overwater and underwater datasets.

Application of Different Techniques for Underwater Image Processing- A Systematic Review

The area of underwater image processing has gained huge attention among the researchers because of its wide application. Image processing is the approach of developing the input image quality so that it would be understood easily by users in future. Image processing develops the data content of image and changes the images visual impact on the observer. Image processing intensifies the image features and accentuates the features of image like corners, contrast to enhance the photographs display which is much helpful for study and examination. Several methods and techniques have been used by researchers worldwide in order to resolve the underwater image processing issues. This paper reviews different techniques of underwater image processing and also compares the techniques among each other. This paper discusses about the filtering method, histogram-based equalization method and particle swarm optimization techniques and compares them with their advantages.