Image Restoration Method Research Articles

A study on the prediction of development strategies of art education in universities based on deep learning model

Abstract In this paper, we propose a deep learning restoration method for fine art painting images based on a contextual encoder and select a convolutional self-encoder-based image restoration model for workflow analysis, including forward inference and backward network propagation. We design and develop a software system for predicting the grades of college art and painting students and apply the neural network prediction model to this system to realize the grade prediction of college art students. The accuracy of the GDPN model is 0.922, the precision is 0.904, and the recall is 0.966, which can consider both the temporal and overall information in the click behavior and achieve a better prediction effect.

Super-resolution image restoration for microlens array imaging system

Microlens array (MLA) has been widely applied in optical imaging fields. However, the existing MLA imaging system (MLAIS) often suffers the problems of low imaging resolutions and luminance contrast due to the limitation of the sensor size and microlens aperture. To address these problems, we propose a super-resolution (SR) image restoration method for the microlens array imaging system (MLAIS). The proposed method uses an Otsu-Kmeans gravity-based center extraction method to segment the image captured by MLAIS into many sub-images and an enhanced correlation coefficient-based approach to align the sub-images. A maximum a posteriori probability algorithm is developed to recover the SR image from the aligned sub-images. A two-step gray transformation algorithm is designed to enhance the SR image. Numerical simulations and practical experiments are implemented to verify the effectiveness of the proposed method, and its SR restoration performance is evaluated by comparing the results with classical and state-of-the-art methods. The results indicate that the proposed method can achieve high-quality SR image restoration for the MLAIS, and its SR performance is better than the comparison methods. The proposed method may find applications in multispectral SR imaging, remote sensing SR imaging, muti-sensor SR imaging, and so on.

Development of Adaptive Point-Spread Function Estimation Method in Various Scintillation Detector Thickness for X-ray Imaging.

An indirect conversion X-ray detector uses a scintillator that utilizes the proportionality of the intensity of incident radiation to the amount of visible light emitted. A thicker scintillator reduces the patient's dose while decreasing the sharpness. A thin scintillator has an advantage in terms of sharpness; however, its noise component increases. Thus, the proposed method converts the spatial resolution of radiographic images acquired from a normal-thickness scintillation detector into a thin-thickness scintillation detector. Note that noise amplification and artifacts were minimized as much as possible after non-blind deconvolution. To accomplish this, the proposed algorithm estimates the optimal point-spread function (PSF) when the structural similarity index (SSIM) and feature similarity index (FSIM) are the most similar between thick and thin scintillator images. Simulation and experimental results demonstrate the viability of the proposed method. Moreover, the deconvolution images obtained using the proposed scheme show an effective image restoration method in terms of the human visible system compared to that of the traditional PSF measurement technique. Consequently, the proposed method is useful for restoring degraded images using the adaptive PSF while preventing noise amplification and artifacts and is effective in improving the image quality in the present X-ray imaging system.

Specular highlight removal based on dichromatic reflection model and priority-based adaptive direction with light field camera

In this paper, an image restoration method with light field camera is proposed to effectively remove specular highlight, and it only needs single shot and has no special requirements for light source and environment. The x-means clustering method based on dichromatic reflection model is proposed to recover unsaturated highlight pixels, and a priority-based adaptive direction method is designed to eliminate saturated specular highlight pixels. After that, a method based on information entropy and mean gradient ratio, called EMGR, is established to quantitatively evaluate the image quality of specular highlight removal. The experimental results showed that our method can effectively utilized the multi-view advantage of the light field to remove the specular highlight of images in different scenes while completely preserving the images texture information, and has the advantages of low complexity and wide applicability.

Modeling multiply scattered light in full-field optical coherence tomographic imaging of biological tissues

In this work, we have constructed a handheld full-field optical coherence tomography (FFOCT) system with high spatial resolution, and the Michelson interferometer (served as compensating optical path) and Fizeau interferometer (served as detection) were mounted in series in this system. The expressions of the multiple scattered light decomposed as the sum of incoherent and coherent contribution were derived based on the diffusion equation and mutual coherence function, and the variations of resolution with imaging depth were derived by using the model of multiple scattering. Experiments were conducted on samples composed of the USAF 1951 resolution target covered by the onion slice and human finger-skin slice, respectively, to validate and evaluate the accuracy of our model. A method of image restoration is then presented in which the relation between the resolution and measuring depth and the simulated PSF in the absence of multiple scattering are employed. With the help of this method the lateral resolution of 2.2 μm and 3.9 μm were separately achieved in the onion slice (measuring depth of 64 μm) and human finger-skin slice (measuring depth of 60 μm), and more tiny structures in the en face image of human finger-skin and moth wings were visible. In addition, a novel method for estimating the values of mean free path and anisotropy of scattering medium was proposed.

Graph laplacian regularization with sparse coding in secure image restoration and representation for Internet of Things

Image restoration (IR) attempts to recreate the original (ideal) scene from a degraded observation. The goal of image restoration is to avert the process of image deterioration that happens during image acquisition and processing. Blurring and noise are two common types of picture capture degradation. When the blurring function is unknown, the image restoration issue is referred to as blind image restoration. Existing methods of image restoration do not provide efficient results due to time complexity, computational complexity, large-scale scaling factor and limited input. Therefore, in order to overcome those problems, an Improved Graph Laplacian Regularization with Sparse Coding by integrating Internet of Things (IoT) is developed in this research. The process of removing de-noised portions of image to texture layer and cartoon layer is carried out by Morphological Component Analysis (MCA). An improved Graph Laplacian regularized method and Simultaneous Sparse Coding with Gaussian Scale Mixture (SSC-GSM) methods is performed on texture layer, cartoon layer and restored image. A Levin’s dataset and a real-time dataset such as industrial manufacturing products are analyzed in this research. The proposed Graph Laplacian algorithm and sparse coding model are better efficient in identifying sharper texture information and getting a restored image. The proposed method proves that it has better performance in terms of Peak Signal-to-Noise Ratio (PSNR) of 35.82 and Structural Similarity Index Measure (SSIM) of 0.94 than the existing methods.

Underwater image restoration with multi-scale shallow feature extraction and detail enhancement network

Underwater imaging and image processing techniques have recently attracted increasing attention in underwater target detection, archaeology, resource exploration, etc. However, the captured underwater images usually suffer from colour deviation (blue, green, yellow, etc.) and detail blurring problems due to the light absorption, light scattering effects, noises and target occlusions in the water. To solve the above two degradation problems, we propose an underwater image restoration method with multi-scale shallow feature extraction and detail enhancement network. Specifically, we develop a multi-scale shallow feature extraction module to address local detail blurring and alleviate colour deviation issues. Additionally, we design a local detail enhancement module based on an encoder-decoder structure. Quantitative and qualitative experiment results indicate that the proposed method outperforms recently reported state-of-the-art deep-learning-based underwater image restoration methods on four public underwater image datasets. Besides, we implement many ablation experiments to validate the effectiveness of each module in the proposed network.

Newly Designed Identification Scheme for Monitoring Ice Thickness on Power Transmission Lines

Overhead power transmission line icing (PTLI) disasters are one of the most severe dangers to power grid safety. Automatic iced transmission line identification is critical in various fields. However, existing methods primarily focus on the linear characteristics of transmission lines, employing a two-step process involving edge and line detection for PTLI identification. Nonetheless, these traditional methods are often complicated when confronted with challenges such as background noise or variations in illumination, leading to incomplete identification of the target area, missed target regions, or misclassification of background pixels as foreground. This paper proposes a new iced transmission line identification scheme to overcome this limitation. In the initial stage, we integrate the image restoration method with image filter enhancement to restore the image’s color information. This combined approach effectively retains valuable information and preserves the original image quality, thereby mitigating the noise presented during the image acquisition. Subsequently, in the second stage, we introduce an enhanced multi-threshold algorithm to separate background and target pixels. After image segmentation, we enhance the image and obtain the region of interest (ROI) through connected component labeling modification and mathematical morphology operations, eliminating background regions. Our proposed scheme achieves an accuracy value of 97.72%, a precision value of 96.24%, a recall value of 86.22%, and a specificity value of 99.48% based on the average value of test images. Through object segmentation and location, the proposed method can avoid background interference, effectively solve the problem of transmission line icing identification, and achieve 90% measurement accuracy compared to manual measurement on the collected PTLI dataset.

Improved Generative Adversarial Network for Super-Resolution Reconstruction of Coal Photomicrographs.

Analyzing the photomicrographs of coal and conducting maceral analysis are essential steps in understanding the coal's characteristics, quality, and potential uses. However, due to limitations of equipment and technology, the obtained coal photomicrographs may have low resolution, failing to show clear details. In this study, we introduce a novel Generative Adversarial Network (GAN) to restore high-definition coal photomicrographs. Compared to traditional image restoration methods, the lightweight GAN-based network generates more explicit and realistic results. In particular, we employ the Wide Residual Block to eliminate the influence of artifacts and improve non-linear fitting ability. Moreover, we adopt a multi-scale attention block embedded in the generator network to capture long-range feature correlations across multiple scales. Experimental results on 468 photomicrographs demonstrate that the proposed method achieves a peak signal-to-noise ratio of 31.12 dB and a structural similarity index of 0.906, significantly higher than state-of-the-art super-resolution reconstruction approaches.

Learning Moore-Penrose based residuals for robust non-blind image deconvolution

This paper proposes a deep learning-based method for image restoration given an inaccurate knowledge of the degradation. We first show how the impulse response of a Wiener filter can approximate the Moore-Penrose pseudo-inverse of the blur convolution operator. The deconvolution problem is then cast as the learning of a residual in the null space of the blur kernel, which, when added to the Wiener restoration, will satisfy the image formation model. This approach is expected to make the network capable of dealing with different blurs since only residuals associated with the Wiener filter have to be learned. Artifacts caused by inaccuracies in the blur estimation and other image formation model inconsistencies are removed by a Dynamic Filter Network. The extensive experiments carried out on several synthetic and real image datasets assert the proposed method's performance and robustness and demonstrate the advantage of the proposed method over existing ones.

An Improved GAN-Based Image Restoration Method for Imaging Logging Images

An improved GAN-based imaging logging image restoration method is presented in this paper for solving the problem of partially missing micro-resistivity imaging logging images. The method uses FCN as the generative network infrastructure and adds a depth-separable convolutional residual block to learn and retain more effective pixel and semantic information; an Inception module is added to increase the multi-scale perceptual field of the network and reduce the number of parameters in the network; and a multi-scale feature extraction module and a spatial attention residual block are added to combine the channel attention. The multi-scale module adds a multi-scale feature extraction module and a spatial attention residual block, which combine the channel attention mechanism and the residual block to achieve multi-scale feature extraction. The global discriminative network and the local discriminative network are designed to gradually improve the content and semantic structure coherence between the restored parts and the whole image by playing off each other and the generative network. According to the experimental results, the average structural similarity measure of the five sets of imaged logging images with different sizes of missing regions in the test set is 0.903, which is an improvement of about 0.3 compared with other similar methods. It is shown that the method in this thesis can be used for the restoration of micro-resistivity imaging log images with good improvement in semantic structural coherence and texture details, thus providing a new deep learning method to ensure the smooth advancement of the subsequent interpretation of micro-resistivity imaging log images.

Pyramid Attention Network for Image Restoration

Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales. However, recent advanced deep convolutional neural network-based methods for image restoration do not take full advantage of self-similarities by relying on self-attention neural modules that only process information at the same scale. To solve this problem, we present a novel Pyramid Attention module for image restoration, which captures long-range feature correspondences from a multi-scale feature pyramid. Inspired by the fact that corruptions, such as noise or compression artifacts, drop drastically at coarser image scales, our attention module is designed to be able to borrow clean signals from their “clean” correspondences at the coarser levels. The proposed pyramid attention module is a generic building block that can be flexibly integrated into various neural architectures. Its effectiveness is validated through extensive experiments on multiple image restoration tasks: image denoising, demosaicing, compression artifact reduction, and super resolution. Without any bells and whistles, our PANet (pyramid attention module with simple network backbones) can produce state-of-the-art results with superior accuracy and visual quality. Our code is available at https://github.com/SHI-Labs/Pyramid-Attention-Networks

DAResNet Based on double-layer residual block for restoring industrial blurred images

Image acquisition and restoration have always been an indispensable part of industrial production. Due to the harsh environment in which industrial cameras are used and problems such as network transmission loss, the images captured are not high definition enough, making image restoration particularly important. Existing image restoration methods have a large number of parameters and time-consuming training, resulting in inefficiency. In this paper, we propose DAResNet for the problem of balancing model size and image restoration performance to improve image restoration while reducing the number of parameters. By analyzing the characteristics of industrial blurred images and combining the features of the proposed model, we propose the Half-Channel Attention Mechanism (HCAM) while constructing the Half-Channel Attention Instance Normalization Block (HCAIN Block) and the Double-Layer Residual Block (DARes Block). Meanwhile, we establish the Industrial Blurred Image Dataset (IBID). Finally, four state-of-the-arts (SOTAs) are selected for comparison on IBID, and DAResNet achieves excellent results.

Restoration and Enhancement of Fuzzy Defect Image Based on Neural Network

<p>In contrast enhancement of fuzzy defect image, details loss and noise expansion are east to occur, which brings difficulties to subsequent image analysis and defect recognition. Therefore, a fuzzy defect image restoration and enhancement method based on neural network is proposed. A double fusion neural network composed of a depth generation network and a discrimination network is designed. The residual of the denoised fuzzy image and the real image is output by the network, which is input into the discrimination network together with the real image, and the difference between the two is judged by the total loss function. To solve the problem of pixel coordinate value of fuzzy defect image, neural network is used to build a fast correction algorithm. Therefore, a fuzzy image restoration and enhancement method based on neural network is proposed to improve the image quality. By reconstructing the resolution of fuzzy defect image, a hierarchical enhancement method of fuzzy defect image region is constructed to achieve fuzzy defect image restoration and enhancement. The results show that the proposed method has high image processing ability in restoration and enhancement of fuzzy defect images. The fitting value of neural network is 0.92, which is significantly higher than that of the other two methods, indicating that the image restoration and enhancement method based on neural network has higher accuracy. Therefore, the restoration and enhancement method of fuzzy defect image based on neural network has a good restoration and enhancement effect, and can effectively meet the actual needs of people for high-quality images.</p> <p> </p>

A blurred star image restoration method based on gyroscope data and enhanced sparse model

Abstract Star sensors usually have a fixed exposure time to guarantee detection of adequate navigation stars. In a high dynamic environment, star images suffer from degradation due to spacecraft movement, which will severely affects both centroid extraction and attitude accuracy. This paper presents an algorithm for the restoration of motion-blurred star images. The algorithm employs gyroscope assistance and consists of two steps: preprocessing and motion-blurred image restoration. In the preprocessing step, the angular velocity of the gyroscope predicts the motion trajectory, position, and shape of each star point during exposure. This step ensures a good initial estimate of the blur kernel for image restoration. The image restoration step employs an enhanced sparse model inspired by blind deblurring method to solve blur kernel and latent image alternately. Simulations and experiments have verified the effectiveness of the proposed algorithm.

A novel variable exponent non-convex TV[formula omitted] model in image restoration

Most of the existing image restoration methods either fail to preserve edges or produce a staircase effect and over-smooth phenomenon. To overcome these drawbacks, this paper proposes a novel variable exponent non-convex TVp,q(x) model in image restoration. The proposed method combines a TVp regularization and a TVq(x) regularization and inherits the advantages of TVp model and TVq(x) model well. The proposed model avoids the staircase effect while preserving good edges. The iterative support shrinking algorithm with proximal linearization (ISSAPL) is employed to solve the proposed variable exponent model. Experimental results illustrate the effectiveness of the proposed model in image restoration.

Underwater image clarifying based on human visual colour constancy using double‐opponency

AbstractUnderwater images are often with biased colours and reduced contrast because of the absorption and scattering effects when light propagates in water. Such images with degradation cannot meet the needs of underwater operations. The main problem in classic underwater image restoration or enhancement methods is that they consume long calculation time, and often, the colour or contrast of the result images is still unsatisfied. Instead of using the complicated physical model of underwater imaging degradation, we propose a new method to deal with underwater images by imitating the colour constancy mechanism of human vision using double‐opponency. Firstly, the original image is converted to the LMS space. Then the signals are linearly combined, and Gaussian convolutions are performed to imitate the function of receptive fields (RFs). Next, two RFs with different sizes work together to constitute the double‐opponency response. Finally, the underwater light is estimated to correct the colours in the image. Further contrast stretching on the luminance is optional. Experiments show that the proposed method can obtain clarified underwater images with higher quality than before, and it spends significantly less time cost compared to other previously published typical methods.

Accurate regularized Tucker decomposition for image restoration

We propose a new accurate regularized Tucker decomposition (ARTD) method for image restoration (IR), which considers global low-rankness and local similarity of intrinsic image characteristics. Specifically, global low-rankness is represented by a sparse Tucker core tensor, whereas the local similarity is captured using nonnegative factor matrices and manifold regularization terms. Sparse nonnegative Tucker decomposition (SNTD) and graph nonnegative Tucker decomposition (GNTD) can be considered a special case of ARTD. We propose and implement an effective Alternating Proximal Gradient (APG) based algorithm to solve the ARTD model and deduce the closed-form updating rules. Notably, ARTD does not need to tune the Tucker rank and provides an initialization strategy and a first-order feedback control rule to accelerate its convergence. Experiments on real IR problems show that our method outperforms some existing state-of-the-art methods.

LVRNet: Lightweight Image Restoration for Aerial Images under Low Visibility (Student Abstract)

Learning to recover clear images from images having a combination of degrading factors is a challenging task. That being said, autonomous surveillance in low visibility conditions caused by high pollution/smoke, poor air quality index, low light, atmospheric scattering, and haze during a blizzard, etc, becomes even more important to prevent accidents. It is thus crucial to form a solution that can not only result in a high-quality image but also which is efficient enough to be deployed for everyday use. However, the lack of proper datasets available to tackle this task limits the performance of the previous methods proposed. To this end, we generate the LowVis-AFO dataset, containing 3647 paired dark-hazy and clear images. We also introduce a new lightweight deep learning model called Low-Visibility Restoration Network (LVRNet). It outperforms previous image restoration methods with low latency, achieving a PSNR value of 25.744 and an SSIM of 0.905, hence making our approach scalable and ready for practical use.

Underwater Image Restoration Method Based on Multi-Frame Image under Artificial Light Source

This paper studies the underwater image restoration problem in autonomous operation of AUV guided by underwater visual. An improved underwater image restoration method is developed based on multi-frame neighboring images under artificial light source. At first, multi-frame neighboring images are collected during AUV approaching the targets, and a transmittance estimation method is developed based on the multi-frame images to avoid the assumption of the known normalized residual energy ratio in the traditional methods. Then, the foreground and background regions of the images are segmented by locking the small area where the background light is located. Hence, the accuracy of background light estimation is improved for the underwater mages in turbid water to improve the accuracy of image restoration. Finally, the performance of the developed underwater image restoration method is verified by the comparative results in the pool environment.