Effective Image Restoration Research Articles

Hybrid network via key feature fusion for image restoration

In the field of artificial intelligence, combining transformers and convolutional neural networks (CNNs) to improve performance has become a popular solution for various image restoration tasks. However, the hyperparameters related to feature levels are empirical, leading to the inevitable presence of redundant features that hinder effective image restoration. Additionally, the current method of fusing global and local information is simple and direct, failing to fully exploit the potential of hybrid architectures. To address this issue, we propose a key feature fusion hybrid network (KF2H-Net) that reduces redundancy and dynamically fuses key features. On one hand, we create different learnable selection mechanisms within the hybrid network’s various units to choose global key features and local key features, enhancing the depth perception and selection capabilities for different features. On the other hand, through a parameter fusion module for dynamic feature fusion, we refine the multi-feature fusion method to emphasize the more critical features for image restoration. In order to verify the general performance of the proposed KF2H-Net, we specially selected three typical scenarios (underwater, low-light, and haze) for testing. KF2H-Net represents a novel approach to hybrid models addressing practical applications in the field of artificial intelligence. Extensive experiments show that KF2H-Net achieves state-of-the-art performance across different scenarios.

Flare Removal Model Based on Sparse-UFormer Networks.

When a camera lens is directly faced with a strong light source, image flare commonly occurs, significantly reducing the clarity and texture of the photo and interfering with image processing tasks that rely on visual sensors, such as image segmentation and feature extraction. A novel flare removal network, the Sparse-UFormer neural network, has been developed. The network integrates two core components onto the UFormer architecture: the mixed-scale feed-forward network (MSFN) and top-k sparse attention (TKSA), creating the sparse-transformer module. The MSFN module captures rich multi-scale information, enabling the more effective addressing of flare interference in images. The TKSA module, designed with a sparsity strategy, focuses on key features within the image, thereby significantly enhancing the precision and efficiency of flare removal. Furthermore, in the design of the loss function, besides the conventional flare, background, and reconstruction losses, a structural similarity index loss has been incorporated to ensure the preservation of image details and structure while removing the flare. Ensuring the minimal loss of image information is a fundamental premise for effective image restoration. The proposed method has been demonstrated to achieve state-of-the-art performance on the Flare7K++ test dataset and in challenging real-world scenarios, proving its effectiveness in removing flare artefacts from images.

Optimization of Digital Inheritance Pathway of Folk Art in Information Age

How to make full use of digital information technology to realize the digital heritage of folk art has become a key concern in the field of non-legacy. Based on digital information technology, this paper proposes image restoration technology to assist folk art digital heritage research. Folk art images are acquired by double mirror reflection method, and pre-processing operations such as defogging, denoising and edge detection are carried out on the images to reduce the influence of interference information in the original pictures on the image restoration effect. The Criminisi algorithm is improved and optimized by combining the structure tensor theory, the improved priority calculation function, the matching criterion and the confidence updating method, and then the research design of image restoration technology assisting folk art digitization is completed and analyzed by examples. The results show that the repair effect of the improved Criminisi algorithm is better than that of the classical Criminisi algorithm, but the repair time of the improved Criminisi algorithm is longer than that of the traditional algorithm by 64.25%, but it is still less than 0.500 seconds, which is within the acceptable range, and it has certain practicability for the digitization and inheritance of folk art in the information age. algorithm has some practicality. Compared with other digital inheritance paths, the method proposed in this study can restore the original appearance of folk art to the maximum extent, and it has theoretical guidance value for the study of optimizing the digital inheritance of traditional folk art.

The Effects of AI-Driven Face Restoration on Forensic Face Recognition

In biometric recognition, face recognition is a mature and widely used technique that provides a fast, accurate, and reliable method for human identification. This paper aims to study the effects of face image restoration for forensic face recognition and then further analyzes the advantages and limitations of the four state-of-the-art face image restoration methods in the field of face recognition for forensic human image identification. In total, 100 face image materials from an open-source face image dataset are used for experiments. The Gaussian blur processing is applied to simulate the effect of blurred face images in actual cases of forensic human image identification. Four state-of-the-art AI-driven face restoration methods are used to restore the blurred face images. We use three mainstream face recognition systems to evaluate the recognition performance changes of the blurred face images and the restored face images. We find that although face image restoration can effectively remove facial noise and blurring effects, the restored images do not significantly improve the recognition performance of the face recognition systems. Face image restoration may change the original features in face images and introduce new made-up image features, thereby affecting the accuracy of face recognition. In current conditions, the improvement in face image restoration on the recognition performance of face recognition systems is limited, but it still has a positive role in the application of forensic human image identification.

Image Inpainting based on Dilated Neighborhood Attention

In response to the phenomenon that existing image restoration algorithms have blurry edges, incoherent textures, and lack clarity and delicacy for large-area missing images, a two-stage generative adversarial image restoration algorithm based on dilated neighborhood attention is proposed. This algorithm decouples image restoration into edge structure restoration and texture structure restoration, introduces a dilated neighborhood attention mechanism, and enhances the generator's focus on important information and structures in the image by constructing a residual attention network, thereby improving the perception and utilization of texture details, resulting in more realistic image views and finer texture details. This paper introduces the Binary Cross-Entropy with Logits loss function in the discriminators of the two stages, which can help the discriminator learn how to more effectively identify real and generated images, thus improving the overall network performance. The Ranger21 optimizer is introduced to accelerate learning without affecting generalization, addressing the problem of traditional optimizers systematically staying in poor initial states. The datasets used in this paper are Paris Street View and CelebA-HQ. Comparative experiments with other image restoration methods show that both peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) have improved, and the larger the mask area, the more significant the improvement. Experiments prove that the images restored by the proposed algorithm have more reasonable structures and richer details, and the image restoration effect is superior.

Inter-module gap filling method for photon counting detectors based on dual acquisition.

The use of photon counting detectors in X-ray imaging missions can effectively improve the signal-to-noise ratio and image resolution. However, the stitching of photon counting detector modules leads to large-size localized information loss in the acquired projected image, which seriously affects the regional observation. In this paper, we propose a method to fill the inter-module gap based on dual acquisition, referred to as the GFDA algorithm, which is divided into three main steps: (i) acquire the main projection by short-exposure scanning, and then scan again by vertically moving the carrier table to acquire the reference projection; (ii) use the alignment method to locate the projected region of interest; (iii) use image stitching and image fusion to recover the missing information. We analyzed the gray value of the region of interest of the Siemens star projection and the reconstructed conch slice data, and proved that the proposed method can recover the information more smoothly and perfectly. The GFDA algorithm is able to achieve a better image restoration effect without additional scanning time and better retain image details. In addition, the GFDA algorithm is scalable, which is demonstrated in the task of filling the stitching of multiple types of photonic technology detectors.

Review of Matrix Rank Constraint Model for Impulse Interference Image Inpainting

Camera failure or loss of storage components in imaging equipment may result in the loss of important image information or random pulse noise interference. The low-rank prior is one of the most important priors in image optimization processing. This paper reviews and compares some low-rank constraint models for image matrices. Firstly, an overview of image-inpainting models based on nuclear norm, truncated nuclear norm, weighted nuclear norm, and matrix-factorization-based F norm is presented, and corresponding optimization iterative algorithms are provided. Then, we use different image matrix low-order constraint models to recover satellite images from three types of pulse interference and provide our experimental visual and numerical results. Finally, it can be concluded that the method based on the weighted nuclear norm can achieve the best image restoration effect. The F norm method based on matrix factorization has the shortest computational time and can be used for large-scale low-rank matrix calculations. Compared with nuclear norm-based methods, weighted nuclear norm-based methods and truncated nuclear norm-based methods can significantly improve repair performance.

Towards a Nash game strategy approach to blind image deconvolution: a fractional-order derivative variational framework

Image restoration is a critical process aimed at recovering degraded images, often impacted by factors including motion blur, sensor blurring, defocused photography, optical aberrations, atmospheric distortions, and noise-induced blur. The inherent challenge lies in the typical scenario where both the original image and the blur kernel (Point Spread Function, PSF) are unknown. This restorative process finds applications in various fields, including sensing, medical imaging, astronomy, remote sensing, and criminal investigations. This paper introduces an innovative approach to blind image deconvolution based on Nash game theory. Our focus is placed on restoring linearly corrupted images without processing explicit knowledge of the original image or the blur kernel (PSF). The proposed method formulates blind deconvolution as a two-player static game, with one player dedicated to image deblurring and the other focused on estimating the PSF. The optimal solution is characterized as Nash equilibrium, resulting in effective image restoration. Moreover, we present an enhanced game formulation that incorporates fractional-order derivatives. This unique extension has the potential to improve image restoration accuracy and resilience, leading to breakthroughs in blind image deconvolution and practical applications.

Hir-net: a simple and effective heterogeneous image restoration network

Hir-net: a simple and effective heterogeneous image restoration network

Focusing on features with higher importance degree: A significance-wise mechanism for image restoration

With the aim of retrieving high-quality images from corrupted versions, image restoration meets the extensive demand of application scenarios. State-of-the-art methods solve this problem by means of designing convolution blocks in multistage architectures. However, the existing methods blindly extract and fuse features without considering which parts of the features are more effective for image restoration. In this paper, we propose a significance-wise mechanism with the goal of identifying the importance degree of each region in the feature representation and providing crucial feature information. Through the calculation of nonlinear function in the significance-wise mechanism, the region which is more conducive to image restoration will get higher importance values. Two important types of information are generated within the attention mechanism, including: (i) feature-sufficient maps with abundant feature representations, and (ii) significance-wise maps with the importance degree of patch information in the corrupted images. With the coordination of feature-sufficient maps and significance-wise maps, the network architecture can focus higher attention on crucial parts of feature information. Furthermore, we design a multistage feature fusion block with the significance-wise mechanism. Compared with existing attention mechanisms, our significance-wise mechanism has the ability to identify and generate crucial feature representation for multiple image restoration tasks. Due to the novel design of the network, we successfully implement multiple restoration tasks only by fine-tuning the number of channels once on a network. Abundant quantitative and qualitative experiments demonstrate that the effective image restoration network (EIRN) outperforms existing state-of-the-art algorithms on eleven datasets across a series of image restoration tasks, including image deblurring, denoising, super-resolution, and deraining. The source code and pretrained models will be available at https://github.com/XinyueZhangqdu/EIRN.

Low-Rankness Guided Group Sparse Representation for Image Restoration.

As a spotlighted nonlocal image representation model, group sparse representation (GSR) has demonstrated a great potential in diverse image restoration tasks. Most of the existing GSR-based image restoration approaches exploit the nonlocal self-similarity (NSS) prior by clustering similar patches into groups and imposing sparsity to each group coefficient, which can effectively preserve image texture information. However, these methods have imposed only plain sparsity over each individual patch of the group, while neglecting other beneficial image properties, e.g., low-rankness (LR), leads to degraded image restoration results. In this article, we propose a novel low-rankness guided group sparse representation (LGSR) model for highly effective image restoration applications. The proposed LGSR jointly utilizes the sparsity and LR priors of each group of similar patches under a unified framework. The two priors serve as the complementary priors in LGSR for effectively preserving the texture and structure information of natural images. Moreover, we apply an alternating minimization algorithm with an adaptively adjusted parameter scheme to solve the proposed LGSR-based image restoration problem. Extensive experiments are conducted to demonstrate that the proposed LGSR achieves superior results compared with many popular or state-of-the-art algorithms in various image restoration tasks, including denoising, inpainting, and compressive sensing (CS).

Development of Adaptive Point-Spread Function Estimation Method in Various Scintillation Detector Thickness for X-ray Imaging.

An indirect conversion X-ray detector uses a scintillator that utilizes the proportionality of the intensity of incident radiation to the amount of visible light emitted. A thicker scintillator reduces the patient's dose while decreasing the sharpness. A thin scintillator has an advantage in terms of sharpness; however, its noise component increases. Thus, the proposed method converts the spatial resolution of radiographic images acquired from a normal-thickness scintillation detector into a thin-thickness scintillation detector. Note that noise amplification and artifacts were minimized as much as possible after non-blind deconvolution. To accomplish this, the proposed algorithm estimates the optimal point-spread function (PSF) when the structural similarity index (SSIM) and feature similarity index (FSIM) are the most similar between thick and thin scintillator images. Simulation and experimental results demonstrate the viability of the proposed method. Moreover, the deconvolution images obtained using the proposed scheme show an effective image restoration method in terms of the human visible system compared to that of the traditional PSF measurement technique. Consequently, the proposed method is useful for restoring degraded images using the adaptive PSF while preventing noise amplification and artifacts and is effective in improving the image quality in the present X-ray imaging system.

Adaptive Optimized Discriminative Learning based Image Deblurring using Deep CNN

Image degradation plays a major problem in many image processing applications. Due to blurring, the quality of an image is degraded and there will be a reduction in bandwidth. Blur in an image is due to variations in atmospheric turbulence, focal length, camera settings, etc. Various types of blurs include Gaussian blur, Motion blur, Out-of-focus blur. The effect of noise along with blur further corrupts the captured image. Many techniques have evolved to deblur the degraded image. The leading approach to solve various degraded images are either based on discriminative learning models or on optimization models. Each method has its own advantages and disadvantages. Learning by discriminative methods is faster but restricted to a specific task whereas optimization models handle flexibly but consume more time. Integrating optimization models suitably by learning with discriminative manner results in effective image restoration. In this paper, a set of effective and fast Convolutional Neural Networks (CNNs) are employed to deblur the Gaussian, motion and out-of-focus blurred images that integrate with optimization models to further avoid noise effects. The proposed methods work more efficiently for applications with low-level vision.

Numerical implementation of level set image recovery based on partial differential equation model

Abstract Image recovery is a class of ill-posed inverse problems, so image recovery has been a challenging and fundamental topic in image processing. In this paper, firstly, to recover blurred images with noise, an adaptive fully variable partial differential image recovery model is proposed based on the fully variable partial differential equation model and Gaussian thermal diffusion model, where the adaptiveness is achieved logarithmically by regularizing the edge detection operator. Secondly, a finite difference method, which covers the gradient and scatters operators in the partial differential equation, is applied to the problem of fixing the global range boundary conditions of the image. Finally, the gradient of the image is restored, and then the gradient domain of the image is solved to obtain the complete recovered image. The results show that in the evaluation of the image restoration effect aspect by two indexes, PSNR and SSIM, the value of the adaptive fully variable partial differential image restoration model increases from 28 to 32, and the value of SSIM increases from 0.70 to 0.88, which has good image restoration performance. The adaptive fully variable partial differential image restoration model proposed in this paper has the characteristics of good edge protection and fast convergence, which is of guiding significance for the numerical implementation of image restoration techniques.

A novel image reconstruction algorithm based on texture aware multiscale GAN for veneer defects

Veneer is the critical raw material for manufacturing man-made board products, therefore the quality of the veneer determines the level of the man-made board. However, defects in the veneer may significantly lower its grade. Currently, identifying veneer defects requires manual inspection and subsequent inpainting using a veneer digging machine. Unfortunately, this method only removes the defects of the veneer but ignore the consistency of its texture. To address this issue, we propose a feasible veneer defect reconstruction method that utilizes a texture-aware-multiscale-GAN architecture. Our method performs texture reconstruction of veneer defects to increase the texture information of the reconstructed image while improving the model efficiency, so that generates natural-looking textures in the reconstructed defect areas. The model is trained by end-to-end updating of four cascades of efficient generators and discriminators. We also employed a loss function based on local binary patterns (LBP) to ensure that the restored images contain sufficient texture information. Finally, region normalization is used in the model to enhance the accuracy of the model. Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM) are used to evaluate the effectiveness of image restoration, the results show that PSNR of the method reacheds 35.32 and SSIM reaches 0.971. By minimizing the difference between the generated texture and that of the original image, our model produces high-quality results.

A Second-Order Method for Removing Mixed Noise from Remote Sensing Images.

Remote sensing image denoising is of great significance for the subsequent use and research of images. Gaussian noise and salt-and-pepper noise are prevalent noises in images. Contemporary denoising algorithms often exhibit limitations when addressing such mixed noise scenarios, manifesting in suboptimal denoising outcomes and the potential blurring of image edges subsequent to the denoising process. To address the above problems, a second-order removal method for mixed noise in remote sensing images was proposed. In the first stage of the method, dilated convolution was introduced into the DnCNN (denoising convolutional neural network) network framework to increase the receptive field of the network, so that more feature information could be extracted from remote sensing images. Meanwhile, a DropoutLayer was introduced after the deep convolution layer to build the noise reduction model to prevent the network from overfitting and to simplify the training difficulty, and then the model was used to perform the preliminary noise reduction on the images. To further improve the image quality of the preliminary denoising results, effectively remove the salt-and-pepper noise in the mixed noise, and preserve more image edge details and texture features, the proposed method employed a second stage on the basis of adaptive median filtering. In this second stage, the median value in the original filter window median was replaced by the nearest neighbor pixel weighted median, so that the preliminary noise reduction result was subjected to secondary processing, and the final denoising result of the mixed noise of the remote sensing image was obtained. In order to verify the feasibility and effectiveness of the algorithm, the remote sensing image denoising experiments and denoised image edge detection experiments were carried out in this paper. When the experimental results are analyzed through subjective visual assessment, images denoised using the proposed method exhibit clearer and more natural details, and they effectively retain edge and texture features. In terms of objective evaluation, the performance of different denoising algorithms is compared using metrics such as mean square error (MSE), peak signal-to-noise ratio (PSNR), and mean structural similarity index (MSSIM). The experimental outcomes indicate that the proposed method for denoising mixed noise in remote sensing images outperforms traditional denoising techniques, achieving a clearer image restoration effect.

Multi-Objective Reptile Search Algorithm Based Effective Image Deblurring and Restoration

Images are frequently affected because of blurring, data loss occurred by sampling and noise occurrence. The images are getting blurred because of object movement in the scenario, atmospheric misrepresentations and optical aberrations. The main objective of image restoration is to evaluate the original image from the corrupted data. To overcome this issue, the Multi-Objective Reptile Search Algorithm is proposed for performing an effective Image Deblurring and Restoration (MORSA-IDR). The proposed MORSA is used in two different processes such as threshold and kernel parameter calculation. In that, threshold values are used for detecting and replacing the noisy pixel removal using Deep Residual Network (DRN), and estimation of kernel is performed for deblurring the images. The main objective of the proposed MORSA-IDR is to enhance the process of deblurring for recovering low-level contextual information. The MORSA-IDR is evaluated using Peak SignalNoise Ratio (PSNR) and structural similarity index (SSIM). The existing research such as Enhanced Local Maximum Intensity (ELMI) prior and Deep Unrolling for Blind Deblurring (DUBLID) are used to evaluate the MORSA-IDR.The PSNR of MORSA-IDR for image 6 is 30.98 dB which is high when compared to the ELMI and DUBLID.

Image denoising using adaptive bi-dimensional stochastic resonance system

Using stochastic resonance (SR) mechanism, the output signal can be enhanced by adding noise to the nonlinear system. Therefore, an image denoising algorithm based on adaptive bi-dimensional stochastic resonance (ABSR) is proposed in this paper. Firstly, the image is sampled as a bi-dimensional signal, and an adaptive bi-dimensional dynamic nonlinear system model is constructed. The peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the output image are used as the double evaluation model of the adaptive system, and the optimal parameters of the model are automatically obtained by adjusting the parameters of the dynamic nonlinear system using the reverse positioning method. Compared with the traditional mean filter, median filter and one-dimensional stochastic resonance, the image restoration effect of dynamic adaptive bi-dimensional stochastic resonance is more closer to the original image, and the histogram, PSNR and SSIM of the output image are also significantly better than the other three methods. The results show that dynamic adaptive bi-dimensional stochastic resonance has better denoising effect and better robustness to the change of noise intensity in image processing.

Restoration of Spatially Variant Blurred Images with Wide-Field Telescope Based on Deep Learning.

The wide-field telescope is a research hotspot in the field of aerospace. Increasing the field of view of the telescope can expand the observation range and enhance the observation ability. However, a wide field will cause some spatially variant optical aberrations, which makes it difficult to obtain stellar information accurately from astronomical images. Therefore, we propose a network for restoring wide-field astronomical images by correcting optical aberrations, called ASANet. Based on the encoder-decoder structure, ASANet improves the original feature extraction module, adds skip connection, and adds a self-attention module. With these methods, we enhanced the capability to focus on the image globally and retain the shallow features in the original image to the maximum extent. At the same time, we created a new dataset of astronomical aberration images as the input of ASANet. Finally, we carried out some experiments to prove that the structure of ASANet is meaningful from two aspects of the image restoration effect and quality evaluation index. According to the experimental results, compared with other deblur networks, the PSNR and SSIM of ASANet are improved by about 0.5 and 0.02 db, respectively.

Learning Enriched Features for Fast Image Restoration and Enhancement.

Given a degraded input image, image restoration aims to recover the missing high-quality image content. Numerous applications demand effective image restoration, e.g., computational photography, surveillance, autonomous vehicles, and remote sensing. Significant advances in image restoration have been made in recent years, dominated by convolutional neural networks (CNNs). The widely-used CNN-based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatial details are preserved but the contextual information cannot be precisely encoded. In the latter case, generated outputs are semantically reliable but spatially less accurate. This paper presents a new architecture with a holistic goal of maintaining spatially-precise high-resolution representations through the entire network, and receiving complementary contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing the following key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange across the multi-resolution streams, (c) non-local attention mechanism for capturing contextual information, and (d) attention based multi-scale feature aggregation. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on six real image benchmark datasets demonstrate that our method, named as MIRNet-v2, achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement. The source code and pre-trained models are available at https://github.com/swz30/MIRNetv2.