Prior-based Methods Research Articles

Norm constraints pyramid for image dehazing

Dark channel prior-based methods have achieved remarkable performance for image dehazing. However, previous studies are mostly focused on the accuracy of the assumptions used in the target scenes, which incurs color distortion and brightness reduction when the models are used for real-world hazy images.We propose a norm constraints pyramid framework to improve the generalization performance of dehazing. First, a local color adaptive correction approach is devised to ascertain whether there is any color bias and thereafter rectify it automatically. Furthermore, multiple norm constraint methods are developed to improve the transmission and accomplish the first image removal. Finally, a non-linear enhancement method is created via this restriction that precisely modifies the brightness of an image.Through extensive experiments, we demonstrate that our framework establishes the new state-of- the-art performance for real-world dehazing, in terms of visual quality assessed by no-reference quality metrics as well as subjective evaluation and downstream task performance indicator.

Genomic prediction based on selective linkage disequilibrium pruning of low-coverage whole-genome sequence variants in a pure Duroc population

BackgroundAlthough the accumulation of whole-genome sequencing (WGS) data has accelerated the identification of mutations underlying complex traits, its impact on the accuracy of genomic predictions is limited. Reliable genotyping data and pre-selected beneficial loci can be used to improve prediction accuracy. Previously, we reported a low-coverage sequencing genotyping method that yielded 11.3 million highly accurate single-nucleotide polymorphisms (SNPs) in pigs. Here, we introduce a method termed selective linkage disequilibrium pruning (SLDP), which refines the set of SNPs that show a large gain during prediction of complex traits using whole-genome SNP data.ResultsWe used the SLDP method to identify and select markers among millions of SNPs based on genome-wide association study (GWAS) prior information. We evaluated the performance of SLDP with respect to three real traits and six simulated traits with varying genetic architectures using two representative models (genomic best linear unbiased prediction and BayesR) on samples from 3579 Duroc boars. SLDP was determined by testing 180 combinations of two core parameters (GWAS P-value thresholds and linkage disequilibrium r2). The parameters for each trait were optimized in the training population by five fold cross-validation and then tested in the validation population. Similar to previous GWAS prior-based methods, the performance of SLDP was mainly affected by the genetic architecture of the traits analyzed. Specifically, SLDP performed better for traits controlled by major quantitative trait loci (QTL) or a small number of quantitative trait nucleotides (QTN). Compared with two commercial SNP chips, genotyping-by-sequencing data, and an unselected whole-genome SNP panel, the SLDP strategy led to significant improvements in prediction accuracy, which ranged from 0.84 to 3.22% for real traits controlled by major or moderate QTL and from 1.23 to 11.47% for simulated traits controlled by a small number of QTN.ConclusionsThe SLDP marker selection method can be incorporated into mainstream prediction models to yield accuracy improvements for traits with a relatively simple genetic architecture, however, it has no significant advantage for traits not controlled by major QTL. The main factors that affect its performance are the genetic architecture of traits and the reliability of GWAS prior information. Our findings can facilitate the application of WGS-based genomic selection.

Guided Depth Map Super-Resolution: A Survey

Guided depth map super-resolution (GDSR), which aims to reconstruct a high-resolution depth map from a low-resolution observation with the help of a paired high-resolution color image, is a longstanding and fundamental problem that has attracted considerable attention from computer vision and image processing communities. Myriad novel and effective approaches have been proposed recently, especially with powerful deep learning techniques. This survey is an effort to present a comprehensive survey of recent progress in GDSR. We start by summarizing the problem of GDSR and explaining why it is challenging. Next, we introduce some commonly used datasets and image quality assessment methods. In addition, we roughly classify existing GDSR methods into three categories: filtering-based methods, prior-based methods, and learning-based methods. In each category, we introduce the general description of the published algorithms and design principles, summarize the representative methods, and discuss their highlights and limitations. Moreover, depth-related applications are introduced. Furthermore, we conduct experiments to evaluate the performance of some representative methods based on unified experimental configurations, so as to offer a systematic and fair performance evaluation to readers. Finally, weconclude this survey with possible directions and open problems for further research. All related materials can be found at https://github.com/zhwzhong/Guided-Depth-Map-Super-resolution-A-Survey .

From shallow sea to deep sea: research progress in underwater image restoration

Underwater images play a crucial role in various fields, including oceanographic engineering, marine exploitation, and marine environmental protection. However, the quality of underwater images is often severely degraded due to the complexities of the underwater environment and equipment limitations. This degradation hinders advancements in relevant research. Consequently, underwater image restoration has gained significant attention as a research area. With the growing interest in deep-sea exploration, deep-sea image restoration has emerged as a new focus, presenting unique challenges. This paper aims to conduct a systematic review of underwater image restoration technology, bridging the gap between shallow-sea and deep-sea image restoration fields through experimental analysis. This paper first categorizes shallow-sea image restoration methods into three types: physical model-based methods, prior-based methods, and deep learning-based methods that integrate physical models. The core concepts and characteristics of representative methods are analyzed. The research status and primary challenges in deep-sea image restoration are then summarized, including color cast and blur caused by underwater environmental characteristics, as well as insufficient and uneven lighting caused by artificial light sources. Potential solutions are explored, such as applying general shallow-sea restoration methods to address color cast and blur, and leveraging techniques from related fields like exposure image correction and low-light image enhancement to tackle lighting issues. Comprehensive experiments are conducted to examine the feasibility of shallow-sea image restoration methods and related image enhancement techniques for deep-sea image restoration. The experimental results provide valuable insights into existing methods for addressing the challenges of deep-sea image restoration. An in-depth discussion is presented, suggesting several future development directions in deep-sea image restoration. Three main points emerged from the research findings: i) Existing shallow-sea image restoration methods are insufficient to address the degradation issues in deep-sea environments, such as low-light and uneven illumination. ii) Combining imaging physical models with deep learning to restore deep-sea image quality may potentially yield desirable results. iii) The application potential of unsupervised and zero-shot learning methods in deep-sea image restoration warrants further investigation, given their ability to work with limited training data.

An optimized GAN method based on the Que-Attn and contrastive learning for underwater image enhancement.

Research on underwater image processing has increased significantly in the past decade due to the precious resources that exist underwater. However, it is still a challenging problem to restore degraded underwater images. Existing prior-based methods show limited performance in many cases due to their reliance on hand-crafted features. Therefore, in this paper, we propose an effective unsupervised generative adversarial network(GAN) for underwater image restoration. Specifically, we embed the idea of contrastive learning into the model. The method encourages two elements (corresponding patches) to map the similar points in the learned feature space relative to other elements (other patches) in the data set, and maximizes the mutual information between input and output through PatchNCE loss. We design a query attention (Que-Attn) module, which compares feature distances in the source domain, and gives an attention matrix and probability distribution for each row. We then select queries based on their importance measure calculated from the distribution. We also verify its generalization performance on several benchmark datasets. Experiments and comparison with the state-of-the-art methods show that our model outperforms others.

Prior-combined dehazing network based on mutual learning

Single-image dehazing is an important problem for high-level computer vision tasks since the existence of haze severely degrades the recognition ability of computers. Most recent works tend to combine prior-based dehazing method with a convolutional neural network to improve the dehazing effect in real scenes. However, these methods do not tackle with the color shifts caused by prior-based methods effectively. In this paper, we propose a prior-combined dehazing network based on mutual learning. Specifically, we build two sub-networks to achieve dehazing by both supervised and unsupervised ways. The supervised sub-network is optimized by ground truth, which provides color fidelity but may acquire under-dehazed images when applied to real scenes. The unsupervised sub-network is optimized by the dehazed images of dark channel prior, which improves the generalization ability but introduces some color shifts or artifacts. Since the dehazing of these two sub-networks shows complementary advantages, a mutual learning mechanism is built for the joint optimization. And we propose a feature fusion module based on the perceptual differences to acquire the final results. The experimental results demonstrate that our method surpasses previous state-of-the-arts on both synthetic and real-world datasets.

Blind deblurring with fractional-order calculus and local minimal pixel prior

Fractional-order calculus is an extension of integer order calculus.In signal processing, fractional-order calculus can non-linearly enhance the low-frequency signal and suppress the high-frequency signal. In this paper, a new fractional-order local minimum pixel prior (FOLMP) is proposed by combining fractional-order calculus with the local minimum pixel prior. The FOLMP of the sharp images includes fewer non-zero pixels than the blur images. A new blur kernel estimation algorithm is proposed by combining L0 regularized FOLMP with the maximum posterior probability. Furthermore, thekernel similarity is employed to adjust the iteration times to accelerate the computational efficiency. Comparative experiments show that the proposed algorithm can perform better on different types of datasets than the most advanced algorithms. In addition, non-overlapping image patches are adopted to compute the FOLMP, and the kernel similarity is used to suppress excessive iterations.Therefore, the proposed algorithm is several times or even tens of times more efficient than the classical prior-based methods.

A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging

Hyperspectral imaging enables many versatile applications for its competence in capturing abundant spatial and spectral information, which is crucial for identifying substances. However, the devices for acquiring hyperspectral images are typically expensive and very complicated, hindering the promotion of their application in consumer electronics, such as daily food inspection and point-of-care medical screening, etc. Recently, many computational spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from widely available RGB images. These reconstruction methods can exclude the usage of burdensome spectral camera hardware while keeping a high spectral resolution and imaging performance. We present a thorough investigation of more than 25 state-of-the-art spectral reconstruction methods which are categorized as prior-based and data-driven methods. Simulations on open-source datasets show that prior-based methods are more suitable for rare data situations, while data-driven methods can unleash the full potential of deep learning in big data cases. We have identified current challenges faced by those methods (e.g., loss function, spectral accuracy, data generalization) and summarized a few trends for future work. With the rapid expansion in datasets and the advent of more advanced neural networks, learnable methods with fine feature representation abilities are very promising. This comprehensive review can serve as a fruitful reference source for peer researchers, thus paving the way for the development of computational hyperspectral imaging.

A Novel Transformer-Based Attention Network for Image Dehazing.

Image dehazing is challenging due to the problem of ill-posed parameter estimation. Numerous prior-based and learning-based methods have achieved great success. However, most learning-based methods use the changes and connections between scale and depth in convolutional neural networks for feature extraction. Although the performance is greatly improved compared with the prior-based methods, the performance in extracting detailed information is inferior. In this paper, we proposed an image dehazing model built with a convolutional neural network and Transformer, called Transformer for image dehazing (TID). First, we propose a Transformer-based channel attention module (TCAM), using a spatial attention module as its supplement. These two modules form an attention module that enhances channel and spatial features. Second, we use a multiscale parallel residual network as the backbone, which can extract feature information of different scales to achieve feature fusion. We experimented on the RESIDE dataset, and then conducted extensive comparisons and ablation studies with state-of-the-art methods. Experimental results show that our proposed method effectively improves the quality of the restored image, and it is also better than the existing attention modules in performance.

Anisotropic Spatial–Spectral Total Variation Regularized Double Low-Rank Approximation for HSI Denoising and Destriping

Hyperspectral images (HSIs) can finely discriminate distinct objects with a high spectral resolution, and they are widely employed in various applications. However, mixed noise severely degrades the quality of HSI and restricts the performance of subsequent tasks. As one of the critical pre-processing steps, HSI denoising has been developed rapidly, among which low-rank (LR) prior-based methods have achieved superior performance. Nevertheless, the existing approaches frequently fail to completely remove noise and reconstruct high-quality HSIs when tackling complicated mixed noise with multiple types of high-intensity stripe noise. To solve this problem, we propose an HSI denoising and destriping method based on anisotropic spatial and spectral total variation regularized double low-rank approximation (ATVDLR). The double low-rank approximation framework is devoted to separating the clean image from the mixed noise by exploiting both the global correlations of HSI tensor and the LR structure of stripe noise. Furthermore, the anisotropic spatial and spectral total variation regularization is introduced to preserve the spatial–spectral smoothness of HSI and the directional feature of stripes, thereby further suppressing high-level stripes and Gaussian noise. Finally, the alternating direction method of multipliers (ADMM) technique is designed to solve the proposed ATVDLR model. Extensive experimental results indicate that the proposed method outperforms other state-of-the-art techniques in multi-type high-intensity mixed noise reduction and image structural information protection, and has superior performance in complex mixed noise removal of real Gaofen-5 HSIs.

Restoration of Single Sand-Dust Image Based on Style Transformation and Unsupervised Adversarial Learning

Since dust particles in the air scatter and absorb light, images captured in sand-dust weather mostly show low contrast, color deviation and blurriness, seriously affecting the reliability of visual tasks. Currently, pixel-level enhancement and prior-based methods are used to restore sand-dust images.However, these methods cannot accurately extract semantic information from the images due to the loss of information and the complexity of the scene depth, which may lead to color distortion and blurred textures of the restored image.We thus presents a two-stage restoration method based on style transformation and unsupervised sand-dust image restoration network(USDR-Net). In the first stage, the grayscale distribution compensation (GDC) method is used to transform the style of the sand-dust image. After transformation, color shift is eliminated and potential information is restored in the balanced image. In the second stage,USDR-Net firstly employs the dark channel prior and the transmission map enhancement network(TME-Network) to generate and refine the transmission map of the balanced image to improve the accuracy of scene depth.Then, it reconstructs a clear image with actual color and high contrast via adversarial learning with unpaired sand-dust and clear images.Extensive experimental results show that our method outperforms state-of-the-art algorithms based on both qualitative and quantitative evaluations.The mean average precision for the target inspection datasets has increased from 16.79% to 68.82%.

On the Convergence of Non-Convex Phase Retrieval With Denoising Priors

Denoising priors have achieved empirical success in solving non-convex phase retrieval but still lack convergence guarantees in theory. In this paper, we provide a novel insight on the convergence guarantees for a general class of non-convex phase retrieval with denoising priors. Specifically, we propose a Wirtinger flow based framework, named DWF, that allows monotone and contractive iterative optimization. We demonstrate in theory that, under a milder demi-Lipschitz condition on denoisers, the proposed DWF framework converges to the actual signal (up to a global sign) at a geometric rate. The derived convergence guarantees and rates are general to accommodate to complex-valued and real-valued signals in the presence of noise perturbation. Furthermore, we exemplify the proposed framework with two prevailing denoising priors, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , plug-and-play priors (PnP) and regularization by denoising (RED), including specific conditions on denoisers and convergence rates. To our best knowledge, this paper is the first attempt to provide theoretical guarantees of convergence for non-convex phase retrieval with denoising priors. Numerical evaluations demonstrate the theoretical results for analytic denoiser like arithmetic mean filter and deep learning based denoiser DnCNN. Furthermore, extensive experiments under the Gaussian model and coded diffraction pattern show that the proposed framework outperforms existing denoising prior-based methods and evidently reduces the necessary sampling rate for stable reconstruction with a guarantee of convergence in theory.

Infrared Small UAV Target Detection Based on Residual Image Prediction via Global and Local Dilated Residual Networks

Thermal infrared imaging possesses the ability to monitor unmanned aerial vehicles (UAVs) in both day and night conditions. However, long-range detection of the infrared UAVs often suffers from small/dim targets, heavy clutter, and noise in the complex background. The conventional local prior-based and the nonlocal prior-based methods commonly have a high false alarm rate and low detection accuracy. In this letter, we propose a model that converts small UAV detection into a problem of predicting the residual image (i.e., background, clutter, and noise). Such novel reformulation allows us to directly learn a mapping from the input infrared image to the residual image. The constructed image-to-image network integrates the global and the local dilated residual convolution blocks into the U-Net, which can capture local and contextual structure information well and fuse the features at different scales both for image reconstruction. Additionally, subpixel convolution is utilized to upscale the image and avoid image distortion during upsampling. Finally, the small UAV target image is obtained by subtracting the residual image from the input infrared image. The comparative experiments demonstrate that the proposed method outperforms state-of-the-art ones in detecting real-world infrared images with heavy clutter and dim targets.

Prior‐guided multiscale network for single‐image dehazing

Single-image dehazing is an important problem because it is a key prerequisite for most high-level computer vision tasks. Traditional prior-based methods adopt priors generated from clear images to restrain the atmospheric scattering model and then recover haze-free images. However, these prior-based methods always encounter over-enhancement, such as halos and colour distortion. To solve this problem, many works use a convolutional neural network to retrieve original images. However, without priors as guidance, these learning-based methods dehaze effectively in synthetic datasets but perform poorly in real scenes. Hence, in this paper, we propose a prior-guided multiscale network for single-image dehazing named PGMNet. Specifically, prior-based methods are adopted to acquire dehazed images of the training dataset in advance and then send these dehazed images to a parameter-shared encoder to form multiscale features. During the decoding process, these multiscale features are adopted to guide the prior-guided multiscale network to recover more image details. Moreover, considering that these prior-based dehazed images usually contain some over-enhanced regions, a spatial attention guided feature aggregation module and squeeze-and-excitation module are adopted to alleviate colour distortion. The proposed PGMNet takes the advantage of prior-based methods in real haze removal and provides superior performance compared with the state-of-the-art methods on both synthetic and real-world datasets.

SMGAN: A self-modulated generative adversarial network for single image dehazing

Single image dehazing has become a key prerequisite for most high-level computer vision tasks since haze severely degrades the input images. Traditional prior-based methods dehaze images by some assumptions concluded from haze-free images, which recover high-quality results but always cause some halos or color distortion. Recently, many methods have been using convolutional neural networks to learn the haze-relevant features and then retrieve the original images. These learning-based methods achieve better performance in synthetic scenes but can hardly restore a clear image with discriminative texture when applied to real-world images, mainly because these networks are trained on synthetic datasets. To solve these problems, a self-modulated generative adversarial network for single image dehazing named SMGAN is proposed. The SMGAN inputs prior-dehazed images into a parameter-shared encoder to produce some latent information of these dehazed images. During the hazy image decoding process, the latent information is sent to self-modulated batch normalization layers, which makes the network fit in real haze removal. Moreover, consider that there are some over-enhanced regions in the guidance images, and a refine module is proposed to alleviate the negative information. The proposed SMGAN combines the advantages of prior-based methods and learning-based methods, which provides superior performance compared with the state-of-the-art methods on both synthetic and real-word datasets.

Image super-resolution by learning weighted convolutional sparse coding

Single image super-resolution (SISR) has witnessed substantial progress recently by deep learning-based methods, due to the data-driven end-to-end training. However, most existing DL-based models are built intuitively, with little thought on priors. And the lack of interpretability limits their further improvements. To avoid this, this paper presents an end-to-end trainable unfolding network which leverages both DL- and prior-based methods. Specifically, we introduce the reweighted algorithm into CSC model and solve it by learning weighted iterative soft thresholding algorithm in a convolutional manner. Based on this, we present a SISR model by learning weighted convolutional sparse coding, in which the channel attention is resorted to learn the weight. Extensive experiments demonstrate the superiority of our method to recent state-of-the-art SISR methods, in terms of both quantitative and qualitative results.

RefineDNet: A Weakly Supervised Refinement Framework for Single Image Dehazing

Haze-free images are the prerequisites of many vision systems and algorithms, and thus single image dehazing is of paramount importance in computer vision. In this field, prior-based methods have achieved initial success. However, they often introduce annoying artifacts to outputs because their priors can hardly fit all situations. By contrast, learning-based methods can generate more natural results. Nonetheless, due to the lack of paired foggy and clear outdoor images of the same scenes as training samples, their haze removal abilities are limited. In this work, we attempt to merge the merits of prior-based and learning-based approaches by dividing the dehazing task into two sub-tasks, i.e., visibility restoration and realness improvement. Specifically, we propose a two-stage weakly supervised dehazing framework, RefineDNet. In the first stage, RefineDNet adopts the dark channel prior to restore visibility. Then, in the second stage, it refines preliminary dehazing results of the first stage to improve realness via adversarial learning with unpaired foggy and clear images. To get more qualified results, we also propose an effective perceptual fusion strategy to blend different dehazing outputs. Extensive experiments corroborate that RefineDNet with the perceptual fusion has an outstanding haze removal capability and can also produce visually pleasing results. Even implemented with basic backbone networks, RefineDNet can outperform supervised dehazing approaches as well as other state-of-the-art methods on indoor and outdoor datasets. To make our results reproducible, relevant code and data are available at https://github.com/xiaofeng94/RefineDNet-for-dehazing.

A Model-Driven Deep Dehazing Approach by Learning Deep Priors

Photos taken in hazy weather are usually covered with white masks and lose important details. Haze removal is a fundamental task and a prerequisite to many other vision tasks. Single image dehazing is an ill-posed inverse problem that has attracted much attention in recent years. Generally, current single dehazing methods can be categorized into the traditional prior-based methods and the data-driven deep learning methods that respectively investigate haze-related image priors and deep architectures. In this paper, we propose a novel model-driven deep learning approach that combines the advantages of both kinds of methods. First, we build an energy model for single image dehazing with physical constraints in both color image space and haze-related feature space (implemented as dark channel space in this work), regularized by haze-related image priors. Then, we design an iterative optimization algorithm for solving the proposed dehazing energy model based on the half-quadratic splitting algorithm, and the priors are transformed to their corresponding proximal operators. Finally, inspired by the optimization algorithm, we design a deep dehazing neural network, dubbed as proximal dehaze-net, by learning the proximal operators for haze-related image priors using CNNs. Our network incorporates physical model constraints of hazes and haze-related prior learning into a novel deep architecture. Extensive experiments show that our method achieves promising performance for single image dehazing.

Image Dehazing in Disproportionate Haze Distributions

Haze removal techniques employed to increase the visibility level of an image play an important role in many vision-based systems. Several traditional dark channel prior-based methods have been proposed to remove haze formation and thereby enhance the robustness of these systems. However, when the captured images contain disproportionate haze distributions, these methods usually fail to attain effective restoration in the restored image. Specifically, disproportionate haze distribution in an image means that the background region possesses heavy haze density and the foreground region possesses little haze density. This phenomenon usually occurs in a hazy image with a deep depth of field. In response, a novel hybrid transmission map-based haze removal method that specifically targets this situation is proposed in this work to achieve clear visibility restoration and effective information maintenance. Experimental results via both qualitative and quantitative evaluations demonstrate that the proposed method is capable of performing with higher efficacy when compared with other state-of-the-art methods, in respect to both background regions and foreground regions of restored test images captured in real-world environments.

Learning a Convolutional Autoencoder for Nighttime Image Dehazing

Currently, haze removal of images captured at night for foggy scenes rely on the traditional, prior-based methods, but these methods are frequently ineffective at dealing with night hazy images. In addition, the light sources at night are complicated and there is a problem of inconsistent brightness. This makes the estimation of the transmission map complicated in the night scene. Based on the above analysis, we propose an autoencoder method to solve the problem of overestimation or underestimation of transmission captured by the traditional, prior-based methods. For nighttime hazy images, we first remove the color effect of the haze image with an edge-preserving maximum reflectance prior (MRP) method. Then, the hazy image without color influence is input into the self-encoder network with skip connections to obtain the transmission map. Moreover, instead of using the local maximum method, we estimate the ambient illumination through a guiding image filtering. In order to highlight the effectiveness of our experiments, a large number of comparison experiments were conducted between our method and the state-of-the-art methods. The results show that our method can effectively suppress the halo effect and reduce the effectiveness of glow. In the experimental part, we calculate that the average Peak Signal to Noise Ratio (PSNR) is 21.0968 and the average Structural Similarity (SSIM) is 0.6802.