Blur Kernel Research Articles

Visual sensitivity filtering based local maximum variation prior for blind image deblurring

Blind image deblurring aims to recover the latent sharp image from a given blurred image when the blur kernel is unknown. To solve this ill-posed problem, many researchers have studied various priors to guide the deblurring process. However, whether the image is sharp or not is mostly based on the subjective judgment of human eyes, and there is no unified objective standard. Since the response of blur to visual sensitivity can be modeled by a filtering method, we use human visual system (HVS) convolution filter to filter the target image to simulate the corresponding visual sensitivity. At the same time, we find that the local maximum variation values of the filtered image will be reduced after blurring process. Therefore, we propose a local maximum variation prior (LMV) based on visual sensitivity filtering and construct a new energy function according to LMV. Our method performs well in various scenarios. Compared with the most advanced algorithms, our model achieves good results on both benchmark datasets and natural images.

Blind Image Deblurring Using Laplacian of Gaussian (LoG) Based Image Prior

Blind image deconvolution, a technique for obtaining restored image as well as the blur kernel from an inexact image. This research uses spatial characteristics to tackle the problem of blind image deconvolution. To work, the proposed method does not necessitate prior information about the blur kernel. Many applications, such as remote sensing, astronomy, and medical X-ray imaging, necessitate blind image deconvolution algorithms. This study used the maximum a posteriori (MAP) paradigm to create a new blind deblurring approach for removing blur from images. In beginning, we employed a Laplacian of Gaussian (LoG)-based image before regularising the gradients of an image. In the second phase, we used an operator known as the Iterative Shrinkage Thresholding Algorithm (ISTA) to cope with the non-convex challenge that develops during the entire deblurring procedure. Finally, we compared our method to several well-known methods in terms of quantitative and qualitative qualities, and we were able to determine which strategy was the most effective. Our findings show that the strategy we propose outperforms the others by a large margin.

Compressed Sensing Super-Resolution Method for Improving the Accuracy of Infrared Diagnosis of Power Equipment

The infrared image of power equipment plays a crucial role in identifying faults, monitoring equipment condition, and so on. The low resolution and low definition of infrared images in applications contribute to the low accuracy of infrared diagnosis. A super-resolution reconstruction method of infrared image, based on compressed sensing theory, is proposed. Firstly, by analyzing the variation of high-frequency information in infrared images with different blurring degrees, the image gradient norm ratio is introduced to estimate the blur kernel matrix in the degradation model a priori. Then, in the process of image reconstruction, we add the full variational regularization term to the traditional compressed sensing model, and design a two-step full variational sparse reconstruction algorithm. Experimental results verify the effectiveness of the method. Compared with the existing classical super-resolution methods, this method offers improvement in subjective visual effect and objective evaluation index. In addition, the final image recognition and infrared diagnosis experiments show that this method is helpful to improve the accuracy of infrared diagnosis of power equipment.

Retinal fundus image superresolution generated by optical coherence tomography based on a realistic mixed attention GAN.

Optical coherence tomography (OCT) is widely used to diagnose retinal diseases. However, due to the limited resolution of OCT imaging systems, the quality of fundus images displayed is not satisfactory, which hinders the diagnosis of patients by ophthalmologists. This is an inevitable problem of OCT imaging systems, but few people have given attention to it. We attempt to solve this problem through deep learning methods. In this paper, we propose a single-image superresolution (SISR) model that is based on a generative adversarial network (GAN) for restoring low-resolution (LR) OCT fundus images to high-resolution (HR) counterparts. To obtain more realistic images, we craft the training data set by obtaining the real blur kernels of the LR images instead of using the bicubic interpolation kernel. The baseline of our generator is similar to that of an enhanced superresolution generative adversarial network (ESRGAN), but we creatively propose a mixed attention block (MAB). In contrast to other superresolution (SR) tasks, to adapt to the characteristics of OCT imaging systems, our network can reconstruct LR images with different upscaling factors in the height and widthdirections. The results of qualitative and quantitative experiments prove that our model is capable of reconstructing retinal fundus images clearly andaccurately. We propose a new GAN model for enhancing the quality of displayed OCT retinal fundus images and achieve state-of-the-artresults.

Motion Blur Kernel Rendering Using an Inertial Sensor: Interpreting the Mechanism of a Thermal Detector.

Various types of motion blur are frequently observed in the images captured by sensors based on thermal and photon detectors. The difference in mechanisms between thermal and photon detectors directly results in different patterns of motion blur. Motivated by this observation, we propose a novel method to synthesize blurry images from sharp images by analyzing the mechanisms of the thermal detector. Further, we propose a novel blur kernel rendering method, which combines our proposed motion blur model with the inertial sensor in the thermal image domain. The accuracy of the blur kernel rendering method is evaluated by the task of thermal image deblurring. We construct a synthetic blurry image dataset based on acquired thermal images using an infrared camera for evaluation. This dataset is the first blurry thermal image dataset with ground-truth images in the thermal image domain. Qualitative and quantitative experiments are extensively carried out on our dataset, which show that our proposed method outperforms state-of-the-art methods.

Insulator Hydrophobic Image Edge Detection Algorithm considering Deconvolution and Deblurring Algorithm

In this paper, the Gram matrix is used to calculate the correlation of the filter response sets under different scale kernels learned by each layer of the network in the deconvolution, and the loss between the corresponding feature response correlations in the multilayer network is calculated. Linear summation is used to obtain a stable, multiscale image model representation. This paper extracts the contours of the salient areas of the image and adjusts the parameters of the deconvolution network to learn the salient area patterns of the image. At the same time, for the image to be generated, a shape template is used to limit the range of the area to be generated in order to obtain a shape image with similar patterns. When the spatial relative relationship characteristics of the image constituent objects are obvious, we appropriately add high-level semantic feature activation values for reinforcement. This paper solves the estimation of the unknown blur kernel by using image prior knowledge, filtering and gradient domain algorithms and other different technologies to obtain image jitter or scene movement information and estimate the size, location, and density of the blur kernel. This paper studies a relatively robust deconvolution model, which is insensitive to random noise, has stable effects, and can overcome the water ripple effect caused by the usual convolution process. This paper attempts to study the fuzzy model with variable space. The usual blur is a spatial invariant model; that is, a single kernel is used to describe the motion of all pixels on the image. By selecting different characteristic parameters, this paper conducts experimental research on some existing hydrophobic indicator function methods and calculates the relationship between characteristic parameters and hydrophobicity when different hydrophobic indicator functions are adopted. One characteristic of the hydrophobic image of composite insulators is low contrast. The traditional method of converting color images to grayscale images cannot improve the image contrast. This paper analyzes the hydrophobic image of the composite insulator, and the extracted B channel component image of the hydrophobic image improves the contrast of the image and facilitates the subsequent segmentation of water traces and background. In this paper, the water repellent image's watermark area is counted, and connected-domain wave processing is used to limit the area of water droplets retained, thereby improving the efficiency of filtering water droplets without having a big impact on the image as a whole. The problem of uneven illumination is an unavoidable problem in the field of image processing, and the resulting reflection problem brings difficulties to image processing. This article regards the reflective area of the watermark as a “hole” and uses the idea of “hole filling” to eliminate the reflective point, which weakens the reflection problem to a certain extent.

Un-supervised learning for blind image deconvolution via Monte-Carlo sampling

Deep learning has been a powerful tool for solving many inverse imaging problems. The majority of existing deep-learning-based solutions are supervised on an external dataset with many blurred/latent image pairs. Recently, there has been an increasing interest on developing dataset-free deep learning methods for image recovery without any prerequisite on external training dataset, including blind deconvolution. This paper aims at developing an un-supervised learning method for blind image deconvolution, which does not call any training sample yet provides very competitive performance. Based on the re-parametrization of latent image using a deep network with random weights, this paper proposed to approximate the maximum-a posteriori estimator of the blur kernel using the Monte-Carlo (MC) sampling method. The MC sampling is efficiently implemented by using dropout and random noise layer, which does not require conjugate model as traditional variational inference does. Extensive experiments on popular benchmark datasets for blind image deconvolution showed that the proposed method not only outperformed existing non-learning methods, but also noticeably outperformed existing deep learning methods, including both supervised and un-supervised ones.

Video Satellite Imagery Super-Resolution via Model-Based Deep Neural Networks

Video satellite imagery has become a hot research topic in Earth observation due to its ability to capture dynamic information. However, its high temporal resolution comes at the expense of spatial resolution. In recent years, deep learning (DL) based super-resolution (SR) methods have played an essential role to improve the spatial resolution of video satellite images. Instead of fully considering the degradation process, most existing DL-based methods attempt to learn the relationship between low-resolution (LR) satellite video frames and their corresponding high-resolution (HR) ones. In this paper, we propose model-based deep neural networks for video satellite imagery SR (VSSR). The VSSR is composed of three main modules: degradation estimation module, intermediate image generation module, and multi-frame feature fusion module. First, the blur kernel and noise level of LR video frames are flexibly estimated by the degradation estimation module. Second, an intermediate image generation module is proposed to iteratively solve two optimal subproblems and the outputs of this module are intermediate SR frames. Third, a three-dimensional (3D) feature fusion subnetwork is leveraged to fuse the features from multiple video frames. Different from previous video satellite SR methods, the proposed VSSR is a multi-frame-based method that can merge the advantages of both learning-based and model-based methods. Experiments on real-world Jilin-1 and OVS-1 video satellite images have been conducted and the SR results demonstrate that the proposed VSSR achieves superior visual effects and quantitative performance compared with the state-of-the-art methods.

Foreign object detection for railway ballastless trackbeds: A semisupervised learning method

This paper proposes a semisupervised algorithm for detecting foreign objects in ballastless beds based on the improved deep SVDD (Support Vector Data Description) algorithm. First, we use the improved Mask R-CNN algorithm to extract the rail and fastener areas in images, assuming that no foreign object exists in the rail and fastener areas. Second, we deepen the backbone network of the deep SVDD to enhance its ability to extract deep semantics from complex images. We perform pure color coverage processing with different colors and mean blur processing with different blur kernels on the rails and fastener regions extracted by the improved Mask R-CNN. The results show that the AUC (Area Under the Curve) of our improved deep SVDD algorithm is 89.23% and improves the AUC compared to that of the original model by 11.09%.

Deep Joint Estimation Network for Satellite Video Super-Resolution With Multiple Degradations

Super-resolution (SR) for satellite video data has been a hot research topic in the field of remote sensing video analysis. The existing satellite video SR methods assume that the blur kernel in the imaging degradation model is known. However, the blur kernel in real satellite videos is usually unknown, which inevitably results in poor performance when the true blur kernel is not consistent with a predefined blur kernel. To address this issue, this article proposes a deep joint estimation network for satellite video SR (JENSVSR), which jointly estimates blur kernels and SR frames. Specifically, JENSVSR is composed of a video SR subnetwork and a blur kernel estimation subnetwork. On one hand, the video SR subnetwork makes use of multiple video frames to generate super-resolved satellite frames. To effectively fuse information from adjacent frames, an alignment and fusion module is proposed in the feature space. On the other hand, the blur estimation subnetwork is also proposed to predict blur kernels. The two subnetworks are coupled by cross-task feature fusion modules (CTFFMs) to achieve joint estimation rather than two-step independent estimation. The performance of our proposed method is evaluated on synthetic and real satellite videos. The experimental results show that our proposed method is superior to the current state-of-the-art SR methods.

Fast and High-Quality Blind Multi-Spectral Image Pansharpening

Blind pansharpening addresses the problem of generating a high spatial-resolution multi-spectral (HRMS) image given a low spatial-resolution multi-spectral (LRMS) image with the guidance of its associated spatially misaligned high spatial-resolution panchromatic (PAN) image without parametric side information. In this paper, we propose a fast approach to blind pansharpening and achieve state-of-the-art image reconstruction quality. Typical blind pansharpening algorithms are often computationally intensive since the blur kernel and the target HRMS image are often computed using iterative solvers and in an alternating fashion. To achieve fast blind pansharpening, we decouple the solution of the blur kernel and of the HRMS image. First, we estimate the blur kernel by computing the kernel coefficients with minimum total generalized variation that blur a downsampled version of the PAN image to approximate a linear combination of the LRMS image channels. Then, we estimate each channel of the HRMS image using local Laplacian prior to regularize the relationship between each HRMS channel and the PAN image. Solving the HRMS image is accelerated by both parallelizing across the channels and by fast numerical algorithms for each channel. Due to the fast scheme and the powerful priors we used on the blur kernel coefficients (total generalized variation) and on the cross-channel relationship (local Laplacian prior), numerical experiments demonstrate that our algorithm outperforms state-of-the-art model-based counterparts in terms of both computational time and reconstruction quality of the HRMS images.

Solar Speckle Image Deblurring With Deep Prior Constraint Based on Regularization

The solar speckle image has the characteristics with single features, more noise, and blurred local details. Most of the existing deep learning deblurring methods for solar speckle images have some problems, such as high-frequency loss, artifact generation, and dependence on the paired image. In this paper, a deep prior deblurring method fusing the regularization model and prior constraint network is proposed. Firstly, the traditional handcrafted regularization priors are added to the network parameterized blind deconvolution model. The image gradient prior and blur kernel initial parameters are respectively used to the network parameterization process of two variables in the blind deconvolution model, which are the latent clean image variables and blur kernel variables. After that, the solar speckle image deep prior deblurring model is established. Secondly, the blur kernel generation network input is estimated by using the atmospheric point spread function (PSF) to improve the model convergence speed. Thirdly, a latent clean image generation network including joint gradient branching and Feature Pyramid Network (FPN) structure is designed to enhance image local edge details. Finally, a joint loss function including pixel loss, image prior loss, and mean squared error (MSE) loss is introduced to guide the model for alternate training. It can obtain the best parameter values of latent clean image and blur kernel, and achieve the solar speckle image high-resolution reconstruction. The experimental results show that the proposed method can eliminate the dependence on the reference image, and the reconstructed image has less noise and more obvious high-frequency details, faster network convergence, and two evaluation indicators of Peak Signal Noise Ratio (PSNR) and Structural Similarity (SSIM) are significantly improved.

Robust Blind Deblurring Under Stripe Noise for Remote Sensing Images

The blind image deblurring methods have achieved great progress for Gaussian random noise. Few works have paid attention to the image deblurring under the structural noise, which is a very common degradation in multi-detectors imaging systems. This paper considers the practical yet challenging problem of blind deblurring in presence of the line-pattern stripe noise for remote sensing images. To overcome this issue, we explicitly formulate the structural noise into a novel and robust blind image deblurring framework. We observe that the structural line-pattern stripe noise would deteriorate both the kernel estimation and non-blind deblurring, and propose a three-stage restoration framework to progressively estimate the blur kernel and clean image. Specifically, we first estimate an intermediate blur kernel by getting rid of the negative influence of the stripe noise in the unidirectional gradient domain. Next, a learning-based kernel refinement network is introduced to rectify the missing details of the inaccurate kernel. Finally, a low-rank decomposition-based non-blind deblurring model is proposed to simultaneously estimate the clean image and stripe noise. Experimental results on real and synthetic datasets demonstrate that the proposed RBDS method outperforms the state-of-the-art blind deblurring methods.

A Deep Framework for Hyperspectral Image Fusion Between Different Satellites.

Recently, fusing a low-resolution hyperspectral image (LR-HSI) with a high-resolution multispectral image (HR-MSI) of different satellites has become an effective way to improve the resolution of an HSI. However, due to different imaging satellites, different illumination, and adjacent imaging time, the LR-HSI and HR-MSI may not satisfy the observation models established by existing works, and the LR-HSI and HR-MSI are hard to be registered. To solve the above problems, we establish new observation models for LR-HSIs and HR-MSIs from different satellites, then a deep-learning-based framework is proposed to solve the key steps in multi-satellite HSI fusion, including image registration, blur kernel learning, and image fusion. Specifically, we first construct a convolutional neural network (CNN), called RegNet, to produce pixel-wise offsets between LR-HSI and HR-MSI, which are utilized to register the LR-HSI. Next, according to the new observation models, a tiny network, called BKLNet, is built to learn the spectral and spatial blur kernels, where the BKLNet and RegNet can be trained jointly. In the fusion part, we further train a FusNet by downsampling the registered data with the learned spatial blur kernel. Extensive experiments demonstrate the superiority of the proposed framework in HSI registration and fusion accuracy.

Unsupervised Blur Kernel Estimation and Correction for Blind Super-Resolution

Blind super-resolution (blind-SR) is an important task in the field of computer vision and has various applications in real-world. Blur kernel estimation is the main element of blind-SR along with the adaptive SR networks and a more accurately estimated kernel guarantees a better performance. Recently, generative adversarial networks (GANs), comparing recurrence patches across scales, have been the most successful unsupervised kernel estimation methods. However, they still involve several problems. ① Their sharpness discrimination ability has been noted as being too weak, causing them to focus more on pattern shapes than sharpness. ② In some cases, kernel correction processes were omitted; however, these are essential because the optimally generated kernel may be narrower than a point spread function (PSF) except when the PSF is ideal low-pass filter. ③ Previous studies also did not consider that GANs are affected by the thickness of edges as well as PSF. Thus, in this paper, 1) we propose a degradation and ranking comparison process designed to induce GAN models to became sensitive to image sharpness, and 2) propose a scale-free kernel correction technique using Gaussian kernel approximation including a thickness parameter. To improve the kernel accuracy further, we 3) propose a combination model of the proposed GAN and DIP(deep image prior) for more supervision, and designed a kernel correction network to propagate gradients through developed correction method. Several experiments demonstrate that our methods enhanced the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{2}$ </tex-math></inline-formula> error and the shape of the kernel significantly. In addition, by combining with ordinary blind-SR algorithms, the best reconstruction accuracy was achieved among unsupervised blur kernel estimation methods.

Blur-Robust Object Detection Using Feature-Level Deblurring via Self-Guided Knowledge Distillation

Images captured from real-world environments often include blur artifacts resulting from camera movement, dynamic object motion, or out-of-focus. Although such blur artifacts are inevitable, most object detection methods do not have special considerations for them; therefore, they may fail to detect objects in blurry images. One possible solution is applying image deblurring prior to object detection. However, this solution is computationally demanding and its performance heavily depends on image deblurring results. In this study, we propose a novel blur-aware object detection framework. First, we construct a synthetic but realistic dataset by applying a diverse set of motion blur kernels to blur-free images. Subsequently, we leverage self-guided knowledge distillation between the teacher and student networks that perform object detection using blur-free and blurry images, respectively. The teacher and student networks share most of their network parameters and jointly learn in a fully-supervised manner. The teacher network provides image features as hints for feature-level deblurring and also renders soft labels for the training of the student network. Guided by the hints and the soft labels from the teacher, the student network learns and expands their knowledge on object detection in blurry images. Experimental results show that the proposed framework improves the robustness of several widely used object detectors against image blurs.

SVDN: A spatially variant degradation network for blind image super-resolution

• The inherent degradations are not exactly the same in different regions of the same image. • A pixel-wise kernel estimation is crucial in Blind SR task. • The affine transformation layer is an effective method to exploit the degradation information. • Our SVDN can produce favorable results on both synthetic and real LR images. Blind super-resolution (SR) aims at generating a high-resolution image from a low-resolution image where the degradation kernels are unknown. Most existing blind super-resolution approaches apply an estimated blur kernel as a degradation prior to the entire image. However, due to the intrinsic degradation characteristics of camera lens or different depth-of-filed in the image, the degradation kernels in different regions are not exactly the same. To address this issue, we propose a spatially variant degradation network (SVDN) for blind SR with a pixel-wise kernel estimation block to handle the complex and unknown degradations in blind SR tasks. Moreover, to fully exploit the estimated degradation information, the output of network is influenced by the estimated kernel via an affine transformation applied to the feature maps in each middle layer. The proposed method permits a lot of freedom in adapting the image degradation locally. Extensive experiments on synthetic and real-world images demonstrate that the proposed method generates more accurate SR results under the complex degradation settings and provide naturally visual results both on the in-focus and out-of-focus regions of the images.

Photon-Limited Blind Deconvolution Using Unsupervised Iterative Kernel Estimation

Blind deconvolution is a challenging problem, but in low-light it is even more difficult. Existing algorithms, both classical and deep-learning based, are not designed for this condition. When the photon shot noise is strong, conventional deconvolution methods fail because (1) the image does not have enough signal-to-noise ratio to perform the blur estimation; (2) While deep neural networks are powerful, many of them do not consider the forward process. When the noise is strong, these networks fail to simultaneously deblur and denoise; (3) While iterative schemes are known to be robust in the classical frameworks, they are seldom considered in deep neural networks because it requires a differentiable non-blind solver. This paper addresses the above challenges by presenting an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unsupervised</i> blind deconvolution method. At the core of this method is a reformulation of the general blind deconvolution framework from the conventional image-kernel alternating minimization to a purely kernel-based minimization. This kernel-based minimization leads to a new iterative scheme that backpropagates an unsupervised loss through a pre-trained non-blind solver to update the blur kernel. Experimental results show that the proposed framework achieves superior results than state-of-the-art blind deconvolution algorithms in low-light conditions.

Analysis of the Impact of the Development Level of Aerobics Movement on the Public Health of the Whole Population Based on Artificial Intelligence Technology.

With the enhancement of China's comprehensive national power and the improvement of people's living standards, health has become the goal that people pursue. While people are thirsty for extensive knowledge and a healthy body, they also pay more attention to the cultivation of elegant temperament and the enjoyment of beauty, and aerobics has become a hot spot for national fitness with its advantages of coordinated and beautiful movements, bright and cheerful rhythm and obvious fitness effects. Aerobics is a new popular fitness sports, from the beginning of development by most fitness enthusiasts, especially it is a women's favorite. To this end, the characteristics, value, status, and role of aerobics in the public health of all people are discussed, and the problems of poor recognition effect in the existing aerobics difficulty aerobics action recognition methods are proposed to apply the graph convolutional neural network to the aerobics difficulty aerobics action recognition. The video of aerobics is divided into several images, and the background of the aerobics difficult aerobics action image is eliminated, and the gray scale co-generation matrix is set to estimate the local area blur kernel of the difficult action image to correct the visual error of the difficult action image. “change to” The aerobics action is divided into several difficult action images, and the gray-scale symbiosis matrix is set to estimate the local area fuzzy core of the difficult action image, and correct the visual error of the difficult action image. On this basis, the graph convolutional neural network is pre-trained to construct a human-directed spatial-temporal skeleton map, and the human-directed spatial-temporal map representation is modeled with temporal dynamic information to achieve aerobics difficult aerobics action recognition. The experimental results show that the recognition time of the difficult aerobics movements based on the graph convolutional neural network is shorter and the number of false recognitions is less in complex and simple backgrounds, which proves that the proposed method improves the recognition of difficult aerobics movements to achieve the goal of promoting the development level of aerobics and improving the public health of all people.

SRDRL: A Blind Super-Resolution Framework With Degradation Reconstruction Loss

Recent years have witnessed the remarkable success of deep learning-based single image super-resolution (SISR) methods. However, most of the existing SISR methods assume that low-resolution (LR) images are purely bicubic downsampled from high-resolution (HR) images. Once the actual degradation is not bicubic, their outstanding performance is hard to maintain. Since the real-world image degradation process can be modeled by a combination of downsampling, blurring, and noise, several SR methods have been proposed to super-resolve LR images with multiple blur kernels and noise levels. However, these SR methods require prior knowledge of the degradation process, which is difficult to obtain in practical applications. To address these issues, we propose a degradation reconstruction loss (DRL), which captures the degradation-wise differences between SR images and HR images via a degradation simulator. Empowered by the degradation simulator, the proposed loss, and an efficient SR network, a blind SR framework (SRDRL) without prior knowledge that can handle multiple degradations is formed. Extensive experimental results demonstrate that the proposed SRDRL outperforms the state-of-the-art blind SR methods and denosing+SR methods on multi-degraded datasets. The degradation reconstruction loss can be a plug-and-play loss for existing SR methods to handle multiple degradations. The source code can be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/FVL2020/SRDRL</uri> .