Image Enhancement Tasks Research Articles

Deep learning tomographic reconstruction through hierarchical decomposition of domain transforms.

Deep learning (DL) has shown unprecedented performance for many image analysis and image enhancement tasks. Yet, solving large-scale inverse problems like tomographic reconstruction remains challenging for DL. These problems involve non-local and space-variant integral transforms between the input and output domains, for which no efficient neural network models are readily available. A prior attempt to solve tomographic reconstruction problems with supervised learning relied on a brute-force fully connected network and only allowed reconstruction with a 1284 system matrix size. This cannot practically scale to realistic data sizes such as 5124 and 5126 for three-dimensional datasets. Here we present a novel framework to solve such problems with DL by casting the original problem as a continuum of intermediate representations between the input and output domains. The original problem is broken down into a sequence of simpler transformations that can be well mapped onto an efficient hierarchical network architecture, with exponentially fewer parameters than a fully connected network would need. We applied the approach to computed tomography (CT) image reconstruction for a 5124 system matrix size. This work introduces a new kind of data-driven DL solver for full-size CT reconstruction without relying on the structure of direct (analytical) or iterative (numerical) inversion techniques. This work presents a feasibility demonstration of full-scale learnt reconstruction, whereas more developments will be needed to demonstrate superiority relative to traditional reconstruction approaches. The proposed approach is also extendable to other imaging problems such as emission and magnetic resonance reconstruction. More broadly, hierarchical DL opens the door to a new class of solvers for general inverse problems, which could potentially lead to improved signal-to-noise ratio, spatial resolution and computational efficiency in various areas.

Multi-Attention DenseNet: A Scattering Medium Imaging Optimization Framework for Visual Data Pre-Processing of Autonomous Driving Systems

The vision system is important for almost all kinds of autonomous driving systems. However, visual data interfered by scattering media, such as smoke, haze, water, and other non-uniform media will be degraded seriously, showing the characteristics of detail loss, poor contrast, low visibility, or color distortion. These characteristics can significantly interfere with the reliability of autonomous driving systems. In real environments the image degradation mechanism is complex, and the estimation of degradation parameters is difficult. This issue remains to be solved. In this study, we employed dense blocks as the framework and introduced the attention mechanism to our model from four dimensions: Multi-scale Attention, Channel Attention, Structure Attention, and ROI (region of interest) Attention. With the help of the training data provided by the weakly supervised model, the proposed method achieved excellent performance in the task of scattering medium imaging optimization in different scenes. Comparative experiments show that the proposed method is robust, and is superior to other state-of-the-art methods in image dehazing, and underwater image enhancement tasks. It is of great significance to improve the reliability of autonomous driving systems in underwater and severe weather environments.

LACN: A lightweight attention-guided ConvNeXt network for low-light image enhancement

Images captured under low-light conditions usually have poor visual quality, and hence greatly reduce the accuracy of subsequent tasks such as image segmentation and detection. In the low-light image enhancement task, noises in the dark areas are generally amplified while the images’ brightness is enhanced. It should be pointed out that many deep learning methods cannot effectively suppress the noise at this stage and capture important feature information. To address the above problem, this paper proposes a Lightweight Attention-guided ConvNeXt Network (LACN) for low-light image enhancement. A novel Attention ConvNeXt Module (ACM) is first proposed by introducing a parameter-free attention module (i.e. SimAM) into the ConvNeXt backbone network. Then, a nontrivial lightweight network LACN based on a multi-attention mechanism is established through stacking two ACMs and fusing their features. In what follows, an improved hybrid attention mechanism, Selective Kernel Attention Module (SKAM), is adopted to effectively extract both global and local information. Such a module realizes the evaluation of lighting conditions for the whole image and the adaptive adjustment of the receptive field. Finally, through the feature fusion module, the features of different stages are aggregated to improve the ability of network to retain color information. Numerous experiments on low-light image enhancement are implemented via comparison with other state-of-the-art methods. Experiments show that the proposed method significantly improves the brightness and contrast of low-illumination images, preserves color information, and suppresses the generation of noises after image brightening.

Cross-domain learning for underwater image enhancement

The poor quality of underwater images has become a widely-known cause affecting the performance of the underwater development projects, including mineral exploitation, driving photography, and navigation for autonomous underwater vehicles. In recent years, deep learning-based techniques have achieved remarkable successes in image restoration and enhancement tasks. However, the limited availability of paired training data (underwater images and their corresponding clear images) and the requirement for vivid color correction remain challenging for underwater image enhancement, as almost all learning-based methods require paired data for training. In this study, instead of creating the time-consuming paired data, we explore the unsupervised training strategy. Specifically, we introduce a universal cross-domain GAN-based framework to generate high-quality images to address the dependence on paired training data. To ensure the vivid colorfulness, the color loss is designed to constrain the training process. Also, a feature fusion module (FFM) is proposed to increase the capacity of the whole model as well as the dual discriminator channel adopted in the architecture. Extensive quantitative and perceptual experiments show that our approach overcomes the limitation of paired data and obtains superior performance over the state-of-the-art on several underwater benchmarks in terms of both accuracy and model deployment.

Multiscale Low-Light Image Enhancement Network With Illumination Constraint

Images captured under low-light environments typically have poor visibility, affecting many advanced computer vision tasks. In recent years, there have been some low-light image enhancement models based on deep learning, but they have not been able to effectively mine the deep multiscale features in the image, resulting in poor generalization performance and instability of the model. The disadvantages are mainly reflected in the color distortion, color unsaturation and artifacts. Current methods unable to adjust the exposure effectively, resulting in uneven exposure or partial overexposure. To address these issues, we propose an end-to-end low-light image enhancement model, which is called multiscale low-light image enhancement network with illumination constraint (MLLEN-IC), to achieve preferable generalization ability and stable performance. On the one hand, we use the squeeze-and-excitation-Res2Net block (SE-Res2block) as a base unit to enhance the model’s ability by extracting deep multiscale features. On the other hand, to make the model more adaptable in low-light image enhancement tasks, we calculate the illumination constraint by the low-light itself to prevent overexposure, uneven exposure, and unsaturated colors. Extensive experiments are conducted to demonstrate MLLEN-IC not only adjusts light levels, but also has a more natural visual effect, and avoids problems such as color distortion, artifacts, and uneven exposure. In particular, MLLEN-IC has pretty generalization and stability performance. The source code and supplementary are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/CCECfgd/MLLEN-IC</uri> .

Novel Computer Tomography image enhancement deep neural networks for asphalt mixtures

• A method for enhancing CT images of asphalt mixtures. • Multiple image enhancement tasks are solved in one model. • The addition of semantic information can improve processing performance. • This study shows that semantic spaces have similarities. This paper aims to solve two significant problems observed in Computer Tomography(CT) asphalt mixture images, which are 1) low-quality CT image, and 2) low efficiency, by intelligent word embedding methods. The first model is called Asphalt Mixture CT Image Enhancement Network (AMCTEN), which is featured by ignoring the semantic loss, while the second model, AMCTEN+, is feature by considering semantic loss without requiring semantic information. Two models can be integrated to realize end-to-end real-application image enhancement. Experimental results show that peak signal to noise ratio of AMCTEN and AMCTEN + in the test set increased by 79.0 % and 66.4 %, structural similarity increased by 15.1 % and 13.3 %, and mean square error decreased by 97.5 % and 95.3 %, respectively. Analysis of semantic feature spaces generated by the two models indicate that word embedding can construct an effective semantic feature space, fusing the semantic features with image features can improve the image quality, and the proposed model has superiority in multiple image enhancement tasks.

Lighting the darkness in the sea: A deep learning model for underwater image enhancement

Currently, optical imaging cameras are widely used on underwater vehicles to obtain images and support numerous marine exploration tasks. Many underwater image enhancement algorithms have been proposed in the past few years to suppress backscattering noise and improve the signal-to-noise ratio of underwater images. However, these algorithms are mainly focused on underwater image enhancement tasks in a bright environment. Thus, it is still unclear how these algorithms would perform on images acquired in an underwater scene with low illumination. Images obtained in a dark underwater scene usually include more noise and have very low visual quality, which may easily lead to artifacts during the process of enhancement. To bridge this gap, we thoroughly study the existing underwater image enhancement methods and low illumination image enhancement methods based on deep learning and propose a new underwater image enhancement network to solve the problem of serious degradation of underwater image quality in a low illumination environment. Due to the lack of ready-made datasets for training, we also propose the first dataset for low-light underwater image enhancement to train our model. Our method can be implemented to skillfully and simultaneously address low-light degradation and scattering degradation in low-light underwater images. Experimental results also show that our method is robust against different illumination levels, which greatly expands the applicable scenarios of our method. Compared with previous underwater image enhancement methods and low-light image enhancement methods, outstanding performance is achieved using our method in various low-light underwater scenes.

Single low-light image brightening using learning-based intensity mapping

Inspired by image-to-curve transformation and multi-exposure fusion, in this paper, we have developed a new method to treat the low light image enhancement tasks as an extended problem with multiple virtual exposures by a non-linear intensity mapping function. Considering that existing image-to-curve methods have difficulty in obtaining the desired detail and brightness recovery in any one iteration without relying on any ground truth, we propose a virtual multi-exposure fusion strategy to merge the outputs from these different iterations. Specifically, a simple CNN is trained to learn a pixel-wise intensity mapping function and accordingly adjust a given image multiple times. Then the results of all iterations are retained together with the original input image for fusion via a WGIF-based Multi-scale pyramid to obtain a final enhanced output. We present experimental results to demonstrate the effectiveness of the new technique and its state-of-the-art performances.

Structure-Texture Aware Network for Low-Light Image Enhancement

Global structure and local detailed texture have different effects on image enhancement tasks. However, most existing works treated these two components in the same way, without fully considering the characteristics of the global structure and local detailed texture. In this work, we propose a structure-texture aware network (STANet) that successfully exploits structure and texture features of low-light images to improve perceptual quality. To construct STANet, a fine-scale contour map guided filter is introduced to decompose the image into a structure component and a texture component. Then, structure-attention and texture-attention subnetworks are designed to fully exploit the characteristics of these two components. Finally, a fusion subnetwork with attention mechanisms is utilized to explore the internal correlations among the global and local features. Furthermore, to optimize the proposed STANet model, we propose a hybrid loss function; specifically, a color loss function is introduced to alleviate color distortion in the enhanced image. Extensive experiments demonstrate that the proposed method improves the visual quality of images; moreover, STANet outperforms most other state-of-the-art approaches.

UHD Low-light image enhancement via interpretable bilateral learning

Convolutional neural networks (CNNs) have achieved unparalleled success in the single Low-light Image Enhancement (LIE) task. Existing CNN-based LIE models over-focus on pixel-level reconstruction effects, hence ignoring the theoretical guidance for sustainable optimization, which hinders their application to Ultra-High Definition (UHD) images. To address the above problems, we propose a new interpretable network, which capable of performing LIE on UHD images in real time on a single GPU. The proposed network consists of two CNNs: the first part is to use the first-order unfolding Taylor’s formula to build an interpretable network, and combine two UNets in the form of first-order Taylor’s polynomials. Then we use this constructed network to extract the feature maps of the low-resolution input image, and finally process the feature maps to form a multi-dimensional tensor termed a bilateral grid that acts on the original image to yield an enhanced result. The second part is the image enhancement using the bilateral grid. In addition, we propose a polynomial channel enhancement method to enhance UHD images. Experimental results show that the proposed method significantly outperforms state-of-the-art methods for UHD LIE on a single GPU with 24G RAM (100 fps).

Semantically Contrastive Learning for Low-Light Image Enhancement

Low-light image enhancement (LLE) remains challenging due to the unfavorable prevailing low-contrast and weak-visibility problems of single RGB images. In this paper, we respond to the intriguing learning-related question -- if leveraging both accessible unpaired over/underexposed images and high-level semantic guidance, can improve the performance of cutting-edge LLE models? Here, we propose an effective semantically contrastive learning paradigm for LLE (namely SCL-LLE). Beyond the existing LLE wisdom, it casts the image enhancement task as multi-task joint learning, where LLE is converted into three constraints of contrastive learning, semantic brightness consistency, and feature preservation for simultaneously ensuring the exposure, texture, and color consistency. SCL-LLE allows the LLE model to learn from unpaired positives (normal-light)/negatives (over/underexposed), and enables it to interact with the scene semantics to regularize the image enhancement network, yet the interaction of high-level semantic knowledge and the low-level signal prior is seldom investigated in previous methods. Training on readily available open data, extensive experiments demonstrate that our method surpasses the state-of-the-arts LLE models over six independent cross-scenes datasets. Moreover, SCL-LLE's potential to benefit the downstream semantic segmentation under extremely dark conditions is discussed. Source Code: https://github.com/LingLIx/SCL-LLE.

Industrial Image Enhancement Method Based on Cloud Edge Fusion

Aiming at the problems of high mean square error, low peak signal-to-noise ratio, and long enhancement time of traditional industrial image enhancement methods, an industrial image enhancement method based on cloud edge fusion was proposed. Firstly, the industrial image is preprocessed and denoised by median filtering algorithm to detect the edge of denoised image. Then, the image is enhanced by top hat transform. Finally, the cloud edge fusion method is used to complete the task of industrial image enhancement to improve the efficiency of industrial image enhancement. The experimental results show that the mean square error of this method is kept at a low level, the peak signal-to-noise ratio is always above 53 dB, and the average industrial image enhancement time is 0.96 s. This method has good performance and can obtain a good industrial image enhancement effect, which has certain application value.

An end‐to‐end perceptual enhancement method for UHD portrait images

Advances in equipment enable photographers to take ultra-high-definition (UHD) photos, and photo retouching enables them to create an impressive image by artistically adjusting and enhancing the brightness, color etc. However, such image enhancement is an artistic and challenging task, which requires a lot of relevant experience and technology. Thus, using an automated algorithm is an attractive option. In recent years, deep learning has made great progress in image processing, which motivated the authors to explore this application in UHD image enhancement task. In this paper, an end-to-end perceptual enhancement method for UHD portrait images is proposed. Since the artistry of images comes from human perception, including the perception of exposure, structure, color etc., an image preprocessing method is proposed which can help the network learn the brightness change from the original image to the target image more accurately. Then, a composite perceptual loss function is proposed that combines global, structural, and color losses. This loss allows the network to simultaneously optimize the distance and perceptual similarity to a given target image during back-propagation. To preserve the details of the UHD image and reduce the number of network calculations and parameters, a Perceptual Enhancement Network (called PEN) based on dilated convolution is proposed. Extensive experiments on the datasets show that the method excels in baselines in both subjective and objective evaluations.

A Novel Un-Supervised GAN for Fundus Image Enhancement with Classification Prior Loss

Fundus images captured for clinical diagnosis usually suffer from degradation factors due to variation in equipment, operators, or environment. These degraded fundus images need to be enhanced to achieve better diagnosis and improve the results of downstream tasks. As there is no paired low- and high-quality fundus image, existing methods mainly focus on supervised or semi-supervised learning methods for color fundus image enhancement (CFIE) tasks by utilizing synthetic image pairs. Consequently, domain gaps between real images and synthetic images arise. With respect to existing unsupervised methods, the most important low scale pathological features and structural information in degraded fundus images are prone to be erased after enhancement. To solve these problems, an unsupervised GAN is proposed for CFIE tasks utilizing adversarial training to enhance low quality fundus images. Synthetic image pairs are no longer required during the training. A specially designed U-Net with skip connection in our enhancement network can effectively remove degradation factors while preserving pathological features and structural information. Global and local discriminators adopted in the GAN lead to better illumination uniformity in the enhanced fundus image. To better improve the visual quality of enhanced fundus images, a novel non-reference loss function based on a pretrained fundus image quality classification network was designed to guide the enhancement network to produce high quality images. Experiments demonstrated that our method could effectively remove degradation factors in low-quality fundus images and produce a competitive result compared with previous methods in both quantitative and qualitative metrics.

BacklitNet: A dataset and network for backlit image enhancement

Backlit images are usually taken when the light source is opposite to the camera. The uneven exposure (e.g., underexposure on the foreground and overexposure on the background) makes the backlit images more challenging than general image enhancement tasks that only need to increase or decrease the exposure on the whole images. Compared to traditional approaches, Convolutional Neural Networks perform well in enhancing images due to the abilities of exploiting contextual features. However, the lack of large benchmark datasets and specially designed models impedes the development of backlit image enhancement. In this paper, we build the first large-scale BAcklit Image Dataset (BAID), which contains 3000 backlit images and the corresponding ground truth manually adjusted by trained photographers. It covers a broad range of categories under different backlit conditions in both indoor and outdoor scenes. Furthermore, we propose a saliency guided backlit image enhancement network, namely BacklitNet, for robust and natural restoration of backlit images. In particular, our model innovatively combines a nested U-structure with bilateral grids, which enables fully extracting multi-scale saliency information and rapidly enhancing arbitrary resolution images. Moreover, a carefully designed loss function based on prior knowledge of brightness distribution of backlit images is proposed to enforce the network to focus more on backlit regions during the training phase. We evaluate the proposed method on the BAID dataset and two public small-scale backlit image datasets. Experimental results demonstrate that our method performs favorably against the state-of-the-art approaches.

A CLUSTERING-BASED FUSION SYSTEM FOR BLASTOMERE LOCALIZATION

Microscopic digital image processing paves the way for study and evaluation of blastomere identification and localization as a preprocessing step for the embryos selection for the In VitroFertilization (IVF) transfer. Computer vision aims at developing automated image system to localize and grade blastomeres before injection. In this paper, we propose a clustering-based system that supports the localization and counting of blastomeres. The dataset, employed in this study, is formed of 50 Images collected at Assisted Reproduction Technology (ART) Unit, International Islamic Center for Population Studies and Research, Al-Azhar University, Egypt. The proposed system is formed of 2 modules named preprocessing and segmentation modules, where different algorithms were investigated for each module. The preprocessing module includes Image denoising and enhancement tasks. Whereas the edge enhancement investigates the performance of Ostu’s thresholding, Canny and Sobel edge detection techniques, while employing Circular Hough Transform (CHT) for the segmentation task. A fusion-based algorithm was then employed to merge the segmented Blastomeres of the previously defined systems to boost the performance through integrated blastomeres, as well the confidence in localization. The fusion-based algorithm showed very promising results reaching an average Precision, sensitivity, and Overall Quality of 87.9%, 92.9%, and 82.3%, respectively.

TAO-Net: Task-Adaptive Operation Network for Image Restoration and Enhancement

Recently, deep learning based models have been extensively applied to image restoration and enhancement tasks. However, existing approaches neglect the potential correlation among these tasks, and do not fully consider the different intensity factors of degradation which affect image quality. To this end, we propose Task-Adaptive Operation Network (TAO-Net) that stacks a series of operation blocks, guided by the supervised attention mechanism to adaptively enable the model to assign corresponding intensity weights for different degradation factors depending on the input signals. To better recover and preserve the accurate details during the repeated operations, we make full use of the image prior to intruduce structure refinement block as auxiliary information for reconstructing high-quality results. Furthermore, feature aggregation block is also adopted to avoid feature interference caused by the direct concatenation between the backbone operation block and the auxiliary refinement block together. Extensive experiments on multiple benchmark datasets demonstrate that our proposed method outperforms previous baselines in image deraining and low-light image enhancement.

Learn From Unpaired Data for Image Restoration: A Variational Bayes Approach.

Collecting paired training data is difficult in practice, but the unpaired samples broadly exist. Current approaches aim at generating synthesized training data from unpaired samples by exploring the relationship between the corrupted and clean data. This work proposes LUD-VAE, a deep generative method to learn the joint probability density function from data sampled from marginal distributions. Our approach is based on a carefully designed probabilistic graphical model in which the clean and corrupted data domains are conditionally independent. Using variational inference, we maximize the evidence lower bound (ELBO) to estimate the joint probability density function. Furthermore, we show that the ELBO is computable without paired samples under the inference invariant assumption. This property provides the mathematical rationale of our approach in the unpaired setting. Finally, we apply our method to real-world image denoising, super-resolution, and low-light image enhancement tasks and train the models using the synthetic data generated by the LUD-VAE. Experimental results validate the advantages of our method over other approaches.

RawFormer: An Efficient Vision Transformer for Low-Light RAW Image Enhancement

Low-light image enhancement plays a central role in various downstream computer vision tasks. Vision Transformers (ViTs) have recently been adapted for low-level image processing and have achieved a promising performance. However, ViTs process images in a window- or patch-based manner, compromising their computational efficiency and long-range dependency. Additionally, existing ViTs process RGB images instead of RAW data from sensors, which is sub-optimal when it comes to utilizing the rich information from RAW data. We propose a fully endto-end Conv-Transformer-based model, RawFormer, to directly utilize RAW data for low-light image enhancement. RawFormer has a structure similar to that of U-Net, but it is integrated with a thoughtfully designed Conv-Transformer Fusing (CTF) block. The CTF block combines local attention and transposed selfattention mechanisms in one module and reduces the computational overhead by adopting a transposed self-attention operation. Experiments demonstrate that RawFormer outperforms state-ofthe-art models by a significant margin on low-light RAW image enhancement tasks.

Quaternion Screened Poisson Equation for Low-Light Image Enhancement

Image enhancement is a technique to enhance the illumination of dark images while keeping the reality and the naturalness of the enhanced images at the same time. For color images, most methods tackle different color channels in a separate way, which overlooks the connection between the color channels. Therefore, in this paper, we consider a quaternion-based model to reserve the color connectivity, which integrates the color information of a pixel by a quaternion number. Moreover, we propose a regularizer based on the gamma-correction function and incorporate it into a screened Poisson equation&#x00A0;for the image enhancement task. The uniqueness and existence of the solution of the proposed model are analyzed. The numerical results also prove the superiority of our scheme for color image enhancement.