Image Enhancement Tasks Research Articles

An Underwater Image Enhancement Algorithm Based on Generative Adversarial Network and Natural Image Quality Evaluation Index

When underwater vehicles work, underwater images are often absorbed by light and scattered and diffused by floating objects, which leads to the degradation of underwater images. The generative adversarial network (GAN) is widely used in underwater image enhancement tasks because it can complete image-style conversions with high efficiency and high quality. Although the GAN converts low-quality underwater images into high-quality underwater images (truth images), the dataset of truth images also affects high-quality underwater images. However, an underwater truth image lacks underwater image enhancement, which leads to a poor effect of the generated image. Thus, this paper proposes to add the natural image quality evaluation (NIQE) index to the GAN to provide generated images with higher contrast and make them more in line with the perception of the human eye, and at the same time, grant generated images a better effect than the truth images set by the existing dataset. In this paper, several groups of experiments are compared, and through the subjective evaluation and objective evaluation indicators, it is verified that the enhanced image of this algorithm is better than the truth image set by the existing dataset.

PatchNet: a tiny low-light image enhancement net

Underexposed images are usually low in brightness and contrast, which degrade the performance of many computer vision algorithms. To solve the problem of overexposing areas that tend to be normal while recovering dark areas in low-light image enhancement tasks, we propose an image-to-patch enhancement model and design a lightweight convolutional neural network called PatchNet. Specifically, the new enhancement model indirectly enhances the network by introducing a patch image, which preserves the incremental information from the low-light image to the normal image. The incremental information is fused with the input image to recover the dark areas while protecting the normal areas of the image. Extensive experiments on real datasets demonstrate the advantages of our method over state-of-the-art methods in subjective feeling and objective evaluation. Our method has achieved better results in restoring details and the adjustment of brightness. By comparing to other methods, our method is more efficient.

Deep residual neural network based image enhancement algorithm for low dose CT images

Current deep learning based image enhancement algorithms attempt to learn the mapping relationship between degraded images and clear images directly. These algorithms often ignore the fidelity constraint of the observational model. In order to improve the image enhancement performance, an improved deep residual neural network based image enhancement algorithm (DRNN-IE) for low dose CT images is proposed in this paper. DRNN-IE embeds the image enhancement task into a deep neural network, and achieves data consistency using multiple enhancement modules and back-projection modules. The enhancement modules in DRNN-IE produce new features through fusing low-level and high-level features. In order to improve the algorithm’s generalization ability, a dual-parameter loss function is adopted to train and optimize the neural network. Experiments on real CT images show that the proposed algorithm has excellent enhancement performance and retains detailed information of low-dose CT images.

CT-Net: Cascade T-shape deep fusion networks for document binarization

Document binarization is a key step in most document analysis tasks. However, historical-document images usually suffer from various degradations, making this a very challenging processing stage. The performance of document image binarization has improved dramatically in recent years by the use of Convolutional Neural Networks (CNNs). In this paper, a dual-task, T-shaped neural network is proposed that has the main task of binarization and an auxiliary task of image enhancement. The neural network for enhancement learns the degradations in document images and the specific CNN-kernel features can be adapted towards the binarization task in the training process. In addition, the enhancement image can be considered as an improved version of the input image, which can be fed into the network for fine-tuning, making it possible to design a chained-cascade network (CT-Net). Experimental results on document binarization competition datasets (DIBCO datasets) and MCS dataset show that our proposed method outperforms competing state-of-the-art methods in most cases.

Simulation of sports action monitoring based on feature similarity model

Sports action monitoring needs to overcome the influence of multiple interference factors and match the monitored movements with standard movements. In order to improve the effect of sports action monitoring, based on the feature similarity algorithm, this paper constructs a sports monitoring model based on the feature similarity model, and proposes a new optimization-driven deep learning framework to solve a series of image enhancement tasks. Moreover, this paper introduces a data-driven deep network to provide a fast descent direction for accelerating the solution, and introduces an error control mechanism inferred by optimization theory to judge the correctness of the network output result and correct it to ensure that the desired target solution can be obtained in the end. In addition, by adopting the idea of optimization-driven deep learning, a task-specific optimization-driven deep learning framework is constructed for complex tasks. Finally, this paper designs experiments to verify the performance of the model. The comparison results show that the model constructed in this paper has certain practical effects.

Underwater Image Enhancement Method Combining color and Luminance

Background:: Aiming at the problems of color distortion, low clarity and poor visibility of underwater image caused by complex underwater environment, a wavelet fusion method UIPWF for underwater image enhancement is proposed. Methods:: First of all, an improved NCB color balance method is designed to identify and cut the abnormal pixels, and balance the color of R, G and B channels by affine transformation. Then, the color correction map is converted to CIELab color space, and the L component is equalized with contrast limited adaptive histogram to obtain the brightness enhancement map. Finally, different fusion rules are designed for low-frequency and high-frequency components, the pixel level wavelet fusion of color balance image and brightness enhancement image is realized to improve the edge detail contrast on the basis of protecting the underwater image contour. Results:: The experiments demonstrate that compared with the existing underwater image processing methods, UIPWF is highly effective in the underwater image enhancement task, improves the objective indicators greatly, and produces visually pleasing enhancement images with clear edges and reasonable color information. Conclusion:: The UIPWF method can effectively mitigate the color distortion, improve the clarity and contrast, which is applicable for underwater image enhancement in different environments.

A lowlight image enhancement method learning from both paired and unpaired data by adversarial training

In recent years, deep learning has been widely used in the field of lowlight image enhancement. In this paper, we propose a novel deep-learning-based lowlight image enhancement method, which could learn from both paired and unpaired data, and perform end-to-end contrast enhancement and noise reduction simultaneously. Concretely, we first employ a multi-level content loss on paired synthesized data for training enhancer to recover detail better. Then, we propose a GAN-based domain adaptation mechanism, which applies an adversarial training strategy and a novel gamma-correction-based self-supervised content loss on abundant unpaired real data, for training the enhancer to perform better on real lowlight images. Furthermore, a Patch-GAN-based noise reduction mechanism is proposed to adversarially train the enhancer to better reduce noise in real lowlight images. Finally, we improve the enhancer by introduce attention mechanism and global feature to original U-net, make it more suitable for lowlight image enhancement task. We conduct experiments on several common datasets and the results show that our method outperforms other state-of-the-arts under a variety of image quality assessment metrics. And when applied as pre-processing module, our method can improve the classification accuracy on lowlight dataset by 1.7%, outperforming other methods too.

Local-CycleGAN: a general end-to-end network for visual enhancement in complex deep-water environment

Underwater image analysis is crucial for many applications such as seafloor survey, biological and environment monitoring, underwater vehicle navigation, inspection and maintenance of underwater infrastructure etc. However, due to light absorption and scattering, the images acquired underwater are always blurry and distorted in color. Most existing image enhancement algorithms typically focus on a few features of the imaging environments, and enhanced results depend on the characteristics of original images. In this study, a local cycle-consistent generative adversarial network is proposed to enhance images acquired in a complex deep-water environment. The proposed network uses a combination of a local discriminator and a global discriminator. Additionally, quality-monitor loss is adopted to evaluate the effect of the generated images. Experimental results show that the local cycle-consistent generative adversarial network is robust and can be generalized for many different image enhancement tasks in different types of complex deep-water environment with varied turbidity.

Multimodal imaging and machine learning to enhance microscope images of shale

A machine learning based image processing workflow is presented to enhance shale source rock microscopic images obtained using diverse imaging platforms. Images were acquired from a 30μm diameter cylindrical Vaca Muerta shale sample using both nondestructive Transmission X-ray Microscopy (TXM, alternately referred to as nano computed tomography) and destructive Focused Ion Beam-Scanning Electron Microscopy (FIB-SEM). Output cross-sectional images from each modality were aligned using a combination of manual and automated registration techniques to create a registered image dataset. We then apply this dataset for two image processing tasks: prediction of image cross sections with SEM-like resolution from nondestructive TXM data and repair of charged region artifacts (localized accumulation of electrons) within SEM images. The image processing algorithms for both tasks use deep learning models, specifically image-to-image Convolutional Neural Networks (CNNs) and conditional Generative Adversarial Networks (cGANs). In the image enhancement tasks, we are able to achieve significant qualitative and quantitative improvement in TXM images. The best model reaches an average Peak Signal to Noise Ratio (PSNR) of 15.8 dB. Conditioning on TXM data is also shown to reduce artifacts from SEM charging, achieving an average PSNR of 25.8 dB. Our results suggest that properly trained and validated networks are capable of significant enhancement of images obtained using nondestructive techniques, thereby improving interpretation of two- and three-dimensional images while preserving samples for future use.

Unpaired Image Enhancement Featuring Reinforcement-Learning-Controlled Image Editing Software

This paper tackles unpaired image enhancement, a task of learning a mapping function which transforms input images into enhanced images in the absence of input-output image pairs. Our method is based on generative adversarial networks (GANs), but instead of simply generating images with a neural network, we enhance images utilizing image editing software such as Adobe® Photoshop® for the following three benefits: enhanced images have no artifacts, the same enhancement can be applied to larger images, and the enhancement is interpretable. To incorporate image editing software into a GAN, we propose a reinforcement learning framework where the generator works as the agent that selects the software's parameters and is rewarded when it fools the discriminator. Our framework can use high-quality non-differentiable filters present in image editing software, which enables image enhancement with high performance. We apply the proposed method to two unpaired image enhancement tasks: photo enhancement and face beautification. Our experimental results demonstrate that the proposed method achieves better performance, compared to the performances of the state-of-the-art methods based on unpaired learning.

Extreme Low-Light Image Enhancement for Surveillance Cameras Using Attention U-Net.

Low-light image enhancement is one of the most challenging tasks in computer vision, and it is actively researched and used to solve various problems. Most of the time, image processing achieves significant performance under normal lighting conditions. However, under low-light conditions, an image turns out to be noisy and dark, which makes subsequent computer vision tasks difficult. To make buried details more visible, and reduce blur and noise in a low-light captured image, a low-light image enhancement task is necessary. A lot of research has been applied to many different techniques. However, most of these approaches require much effort or expensive equipment to perform low-light image enhancement. For example, the image has to be captured in a raw camera file in order to be processed, and the addressing method does not perform well under extreme low-light conditions. In this paper, we propose a new convolutional network, Attention U-net (the integration of an attention gate and a U-net network), which is able to work on common file types (.PNG, .JPEG, .JPG, etc.) with primary support from deep learning to solve the problem of surveillance camera security in smart city inducements without requiring the raw image file from the camera, and it can perform under the most extreme low-light conditions.

A Biological Vision Inspired Framework for Image Enhancement in Poor Visibility Conditions.

Image enhancement is an important pre-processing step for many computer vision applications especially regarding the scenes in poor visibility conditions. In this work, we develop a unified two-pathway model inspired by the biological vision, especially the early visual mechanisms, which contributes to image enhancement tasks including low dynamic range (LDR) image enhancement and high dynamic range (HDR) image tone mapping. Firstly, the input image is separated and sent into two visual pathways: structure-pathway and detail-pathway, corresponding to the M-and P-pathway in the early visual system, which code the low-and high-frequency visual information, respectively. In the structure-pathway, an extended biological normalization model is used to integrate the global and local luminance adaptation, which can handle the visual scenes with varying illuminations. On the other hand, the detail enhancement and local noise suppression are achieved in the detail-pathway based on local energy weighting. Finally, the outputs of structure-and detail-pathway are integrated to achieve the low-light image enhancement. In addition, the proposed model can also be used for tone mapping of HDR images with some fine-tuning steps. Extensive experiments on three datasets (two LDR image datasets and one HDR scene dataset) show that the proposed model can handle the visual enhancement tasks mentioned above efficiently and outperform the related state-of-the-art methods.

Underwater Image Enhancement With a Deep Residual Framework

Owing to refraction, absorption, and scattering of light by suspended particles in water, raw underwater images have low contrast, blurred details, and color distortion. These characteristics can significantly interfere with visual tasks, such as segmentation and tracking. This paper proposes an underwater image enhancement solution through a deep residual framework. First, the cycle-consistent adversarial networks (CycleGAN) is employed to generate synthetic underwater images as training data for convolution neural network models. Second, the very-deep super-resolution reconstruction model (VDSR) is introduced to underwater resolution applications; with it, the Underwater Resnet model is proposed, which is a residual learning model for underwater image enhancement tasks. Furthermore, the loss function and training mode are improved. A multi-term loss function is formed with mean squared error loss and a proposed edge difference loss. An asynchronous training mode is also proposed to improve the performance of the multi-term loss function. Finally, the impact of batch normalization is discussed. According to the underwater image enhancement experiments and a comparative analysis, the color correction and detail enhancement performance of the proposed methods are superior to that of previous deep learning models and traditional methods.

Single infrared image enhancement using a deep convolutional neural network

In this paper, we propose a deep learning method for single infrared image enhancement. A fully convolutional neural network (CNN) is used to produce images with enhanced contrast and details. The conditional generative adversarial networks are incorporated into the optimization framework to avoid the background noise being amplified and further enhance the contrast and details. The existing convolutional neural network architectures, such as residual architectures and encoder–decoder architectures, fail to achieve the best results both in terms of network performance and application scope for infrared image enhancement task. To address this problem, we specifically design a new refined convolutional neural architecture that produces visually very appealing results with higher contrast and sharper details compared to other network architectures. Visible images are used for training since there are fewer infrared images. Proper training samples are generated to ensure that the network trained on visible images can be well applied to infrared images. Experiments demonstrate that our approach outperforms existing image enhancement algorithms in terms of contrast and detail enhancement. Code is available at https://github.com/Kuangxd/IE-CGAN.

Iterative Adaptive Unsymmetric Trimmed Shock Filter for High-Density Salt-and-Pepper Noise Removal

In this paper, an iterative adaptive unsymmetric trimmed shock filter based on partial differential equations (PDE) is proposed to remove high-density salt-and-pepper noise by preserving the edge details in the images. This algorithm consists of two steps: identification of pixels affected by the salt-and-pepper noise and recovery of these noisy pixels using adaptive unsymmetric trimmed shock filter. Shock filter has been used in image processing literature for image restoration and enhancement task. However, its use was limited to unwanted noise removal, enhancement, despeckling, etc. In the proposed work, a modified form of shock filter termed as adaptive unsymmetric trimmed shock filter is designed to remove the salt-and-pepper noise. The proposed algorithm is tested with different test images. Three performance evaluation measures: PSNR, CR and MSSIM are used to validate the proposed scheme. The performance of the proposed algorithm claims better results than the other considered eight existing state-of-the-art techniques.

Learning Converged Propagations with Deep Prior Ensemble for Image Enhancement.

Enhancing visual qualities of images plays very important roles in various vision and learning applications. In the past few years, both knowledge-driven maximum a posterior (MAP) with prior modelings and fully data-dependent convolutional neural network (CNN) techniques have been investigated to address specific enhancement tasks. In this paper, by exploiting the advantages of these two types of mechanisms within a complementary propagation perspective, we propose a unified framework, named deep prior ensemble (DPE), for solving various image enhancement tasks. Specifically, we first establish the basic propagation scheme based on the fundamental image modeling cues and then introduce residual CNNs to help predicting the propagation direction at each stage. By designing prior projections to perform feedback control, we theoretically prove that even with experience-inspired CNNs, DPE is definitely converged and the output will always satisfy our fundamental task constraints. The main advantage against conventional optimization-based MAP approaches is that our descent directions are learned from collected training data, thus are much more robust to unwanted local minimums. While, compared with existing CNN type networks, which are often designed in heuristic manners without theoretical guarantees, DPE is able to gain advantages from rich task cues investigated on the bases of domain knowledges. Therefore, DPE actually provides a generic ensemble methodology to integrate both knowledge and data-based cues for different image enhancement tasks. More importantly, our theoretical investigations verify that the feedforward propagations of DPE are properly controlled toward our desired solution. Experimental results demonstrate that the proposed DPE outperforms state-of-the-arts on a variety of image enhancement tasks in terms of both quantitative measure and visual perception quality.

Automated outdoor depth-map generation and alignment

Image enhancement tasks can highly benefit from depth information, but the direct estimation of outdoor depth maps is difficult due to vast object distances. This paper presents a fully automatic framework for model-based generation of outdoor depth maps and its applications to image enhancements. We leverage 3D terrain models and camera pose estimation techniques to render approximate depth maps without resorting to manual alignment. Potential local misalignments, resulting from insufficient model details and rough registrations, are eliminated with our novel free-form warping. We first align synthetic depth edges with photo edges using the as-rigid-as-possible image registration and further refine the shape of the edges using the tight trimap-based alpha matting. The resulting synthetic depth maps are accurate, calibrated in the absolute distance. We demonstrate their benefit in image enhancement techniques including reblurring, depth-of-field simulation, haze removal, and guided texture synthesis.

Fast adaptive switching technique of impulsive noise removal in color images

In the paper, a family of switching filters designed for the impulsive noise removal in color images is analyzed. The framework of the proposed denoising techniques is based on the concept of cumulated distances between the processed pixel and its neighbors. To increase the filtering efficiency, a robust scheme, in which the sum of distances to only the most similar pixels of the neighborhood serves as a measure of impulsiveness, was elaborated. As this trimmed measure is dependent on the image local structure, an adaptive mechanism was also incorporated. Additionally, a very fast design, which enables image denoising in practical applications, is proposed and the choice of the filter output, which is used to replace the noisy pixels, is discussed. The described family of filters was evaluated on a large set of natural test images and compared with the state-of-the-art restoration methods. The analysis of the achieved results shows that the novel filters outperform the existing techniques in terms of both denoising accuracy and computational complexity. In this way, the proposed techniques can be recommended for the application in various image and video enhancement tasks.

When can feedback-seeking behavior result in a better performance rating? Investigating the moderating role of political skill

ABSTRACTFeedback-seeking behavior is generally used to acquire job-related information that enables performance improvement. However, it can also be a form of impression management that improves an employee’s reputation. In this study, we show that employees who hold an image enhancement motive are more likely to seek feedback from their supervisors. Further, this feedback-seeking behavior mediates the relationship between the image enhancement motive and task performance ratings, consistent with the self-concept enhancing tactician (SCENT) model. However, we additionally incorporate socioanalytic theory to demonstrate that this indirect effect is moderated: the image enhancement motive translates into greater feedback-seeking behavior and task performance ratings only for employees with high political skill, who are capable of acting on their image enhancement motive successfully.

Enhancement of mammogram images in SMRT domain using nonlinear mapping functions

The task of medical image enhancement is to accentuate image features that are clinically relevant and difficult to visualise under normal viewing conditions. Enhancement can be accomplished by increasing image contrast. This paper presents a simple contrast enhancement algorithm in the Sequency based Mapped Real Transform (SMRT) domain. Brightness and contrast of the image are modified by varying DC and AC SMRT coefficients separately. DC SMRT coefficient is changed to bring the image mean to the mid histogram range. Nonlinear mapping functions are used to modify AC SMRT coefficients so as to improve the contrast. Enhancement is quantitatively assessed using second derivative like measurement (SDME) and Image Enhancement Metric (IEM).