Motion Deblurring Research Articles

Pixel-Level and Perceptual-Level Regularized Adversarial Learning for Joint Motion Deblurring and Super-Resolution

Pixel-Level and Perceptual-Level Regularized Adversarial Learning for Joint Motion Deblurring and Super-Resolution

LNNet: Lightweight Nested Network for motion deblurring

Motion deblurring methods based on convolutional neural networks (CNN) have recently demonstrated their advantages over conventional methods. However, repetitions of scaling or slicing operations of these methods on the input images inevitably lead to spatial information loss. Meanwhile, some recent methods based on complex models inevitably bring a large model size and huge computing cost. It is still challenging to balance the deblurring performance and the cost. To this end, we propose a lightweight nested network (LNNet) for the motion-deblurring task. Our LNNet leverages several simple yet efficient sub-networks to process motion deblurring features at each stage. We design a nested connection, which is conducive to the model size reduction when connecting sub-networks so as to reuse deblurring information and facilitate deblurring information diversity. Meanwhile, we introduce the feature-fusion module to improve deblurring performance further. We perform extensive experiments on a workstation platform and an embedded mobile edge computing (MEC) platform to evaluate our LNNet as well as other existing methods. Extensive experimental results demonstrate that our LNNet achieves superior deblurring performance than state-of-the-art methods with a small model size within a short running time. Moreover, experimental results also show that our model is quite suitable for other embedded devices.

A texture detail-oriented generative adversarial network: motion deblurring for multi-textured images

A texture detail-oriented generative adversarial network: motion deblurring for multi-textured images

Decoupling Induction and Multi-Order Attention Drop-Out Gating Based Joint Motion Deblurring and Image Super-Resolution

Resolution decrease and motion blur are two typical image degradation processes that are usually addressed by deep networks, specifically convolutional neural networks (CNNs). However, since real images are usually obtained through multiple degradations, the vast majority of current CNN methods that employ a single degradation process inevitably need to be improved to account for multiple degradation effects. In this work, motivated by degradation decoupling and multiple-order attention drop-out gating, we propose a joint deep recovery model to efficiently address motion blur and resolution reduction simultaneously. Our degradation decoupling style improves the continence and the efficiency of model construction and training. Moreover, the proposed multi-order attention mechanism comprehensively and hierarchically extracts multiple attention features and fuses them properly by drop-out gating. The proposed approach is evaluated using diverse benchmark datasets including natural and synthetic images. The experimental results show that our proposed method can efficiently complete joint motion blur and image super-resolution (SR).

A Motion Deblurring Disentangled Representation Network

We present a Motion Deblurring Disentangled Representation Network (MDDRNet), an end-to-end learned method for motion deblurring. There are three main parts in MDDRNet, Blur Loss Function, Disentangled Representation Network (DRN) module, and Structural Convolutional Neural Network (SCNN) module. By converting matched Gram matrix into minimized Maximum Mean Discrepancy (MMD), the Blur Loss Function is obtained for extracting the motion blur features. And by means of novel convolution and pooling layers, the DRN module is designed for motion deblurring. Furthermore, by the SCNN module, the deblurred image is further corrected and restored. Experiment results show that the MDDRNet has best performance compare with five methods, under three kinds of datasets.

Impact of low injected activity on data driven respiratory gating for PET/CT imaging with continuous bed motion.

Data driven respiratory gating (DDG) in positron emission tomography (PET) imaging extracts respiratory waveforms from the acquired PET data obviating the need for dedicated external devices. DDG performance, however, degrades with decreasing detected number of coincidence counts. In this paper, we assess the clinical impact of reducing injected activity on a new DDG algorithm designed for PET data acquired with continuous bed motion (CBM_DDG) by evaluating CBM_DDG waveforms, tumor quantification, and physician's perception of motion blur in resultant images. Forty patients were imaged on a Siemens mCT scanner in CBM mode. Reduced injected activity was simulated by generating list mode datasets with 50% and 25% of the original data (100%). CBM_DDG waveforms were compared to that of the original data over the range between the aortic arch and the center of the right kidney using the Pearson correlation coefficient (PCC). Tumor quantification was assessed by comparing the maximum standardized uptake value (SUVmax) and peak SUV (SUVpeak) of reconstructed images from the various list mode datasets using elastic motion deblurring (EMDB) reconstruction. Perceived motion blur was assessed by three radiologists of one lesion per patient on a continuous scale from no motion blur (0) to significant motion blur (3). The mean PCC of the 50% and 25% dataset waveforms was 0.74 ± 0.18 and 0.59 ± 0.25, respectively. In comparison to the 100% datasets, the mean SUVmax increased by 2.25% (p = 0.11) for the 50% datasets and by 3.91% (p = 0.16) for the 25% datasets, while SUVpeak changes were within ±0.25%. Radiologist evaluations of motion blur showed negligible changes with average values of 0.21, 0.3, and 0.28 for the 100%, 50%, and 25% datasets. Decreased injected activities degrades the resultant CBM_DDG respiratory waveforms; however this decrease has minimal impact on quantification and perceived image motion blur.

A motion blur QR code identification algorithm based on feature extracting and improved adaptive thresholding

Motion blur can easily affect the quality of images. For example, Quick Response (QR) code is hard to be identified with severe motion blur caused by camera shaking or object moving. In this paper, a motion blur QR code identification algorithm based on feature extraction and improved adaptive thresholding is proposed. First, this work designs a feature extraction framework using a deep convolutional network for motion deblurring. The framework consists of a basic end-to-end network for feature extraction, an encoder-decoder structure for increasing training feasibility and several ResBlocks for producing large receptive fields. Then an improved adaptive thresholding method is used to avoid influence caused by uneven illumination. Finally, the proposed algorithm is compared with several recent methods on a dataset including QR code images influenced by both motion blur and uneven illumination. Experimental results demonstrate that the processing time and identification accuracy of the proposed algorithm are improved in executing motion blur QR code identification missions compared with other competing methods.

Fast Motion-Deblurring of IR Images

Camera gimbal systems pervade various applications such as navigation, target tracking, security and surveillance. The need for higher steering rate (rotation angle per second) of gimbal often results in motion blur in the captured video frames. Motion deblurring in real-time is difficult with existing blind restoration methods which incur large execution times while attempting to retrieve latent images from blurry inputs using high-dimensional optimization. On the other hand, deep learning methods for motion deblurring, though fast, do not generalize satisfactorily with domain shifts. In this work, we address the problem of real-time motion deblurring in infrared (IR) images captured by a real gimbal-based system. We propose two blur-kernel estimation methods and reveal how <i>a priori</i> knowledge of the blur-kernel can be used in conjunction with non-blind deblurring methods to achieve real-time performance. We experimentally show that, in comparison to the state-of-the-art techniques in deblurring, our method is better-suited for practical gimbal-based imaging systems.

High-Frequency Attention Residual GAN Network for Blind Motion Deblurring

The moving image deblurring method based on deep learning has achieved good results. However, some methods are not effective in restoring image texture detail information. Therefore, this paper proposes a High-Frequency Attention Residual Module (HFAR), which is used to guide the network to learn more high-frequency texture information in the image to improve the quality of image detail restoration. The designed attention residual module consists of two sub-modules, Fourier Channel Attention module (FCA) and Edge Spatial Attention module (ESA). The FCA module gives more weight to the feature maps that contain more high-frequency information in multiple channels. While the ESA module gives more weight to the areas in the feature maps which contain more high-frequency information to guide the network to learn image details and texture information. Extensive experiments on different datasets show that our method achieves state-of-the-art performance in motion deblurring.

Holistic 3D Body Reconstruction From a Blurred Single Image

Holistic human pose and shape reconstruction receive huge interest since it restores detailed human pose and shape including facial expression and finger-level hand shape. Existing deep 3D holistic human pose and shape reconstruction methods utilize sharp images as their input which leads to inaccurate human mesh when given a blurred image. Although there exist lots of image deblurring methods, a simple cascaded approach could not produce satisfactory results. In this paper, we introduce D2R (Deblurring-to-Reconstruction), a novel joint framework of human motion deblurring and 3D holistic body reconstruction to solve both problems simultaneously. In addition, we generate a new large-scale dataset that contains sharp/blur image pairs and corresponding 3D body pose/shape. We train the proposed joint framework in an alternating scheme to refine each module’s performance by utilizing an additional structure-aware module. Experimental results show that the proposed method achieves outperforming results 3D holistic human reconstruction qualitatively as well as quantitatively while input image is deblurred.

Continuous Facial Motion Deblurring

We introduce a novel framework for continuous facial motion deblurring that restores the continuous sharp moment latent in a single motion-blurred face image via a moment control factor. Although a motion-blurred image is the accumulated signal of continuous sharp moments during the exposure time, most existing single image deblurring approaches aim to restore a fixed number of frames using multiple networks and training stages. To address this problem, we propose a continuous facial motion deblurring network based on GAN (CFMD-GAN), which is a novel framework for restoring the continuous moment latent in a single motion-blurred face image with a single network and a single training stage. To stabilize the network training, we train the generator to restore continuous moments in the order determined by our facial motion-based reordering process (FMR) utilizing domain-specific knowledge of the face. Moreover, we propose an auxiliary regressor that helps our generator produce more accurate images by estimating continuous sharp moments. Furthermore, we introduce a control-adaptive (ContAda) block that performs spatially deformable convolution and channel-wise attention as a function of the control factor. Extensive experiments on the 300VW datasets demonstrate that the proposed framework generates a various number of continuous output frames by varying the moment control factor. Compared with the recent single-to-single image deblurring networks trained with the same 300VW training set, the proposed method show the superior performance in restoring the central sharp frame in terms of perceptual metrics, including LPIPS, FID and Arcface identity distance. The proposed method outperforms the existing single-to-video deblurring method for both qualitative and quantitative comparisons. In our experiments on the 300VW test set, the proposed framework reached 33.14 dB and 0.93 for recovery of 7 sharp frames in PSNR and SSIM, respectively.

SRDN: A Unified Super-Resolution and Motion Deblurring Network for Space Image Restoration

Space target super-resolution (SR) is a domain-specific single image SR problem aiming to help distinguish the satellite and spacecrafts from numerous space debris. Compared to the other object SR problem, images for space target are always in low quality with varies of degradation condition, as a result of long distance and motion blur, which significantly reduces the manual classification reliability, especially for these small targets, e.g., satellite payloads. To address this challenge, we present an end-to-end SR and deblurring network (SRDN). Concretely, focusing on the low-resolution (LR) space target images with blind motion blur, we integrate the SR and deblur function together, improving the image quality by a unified generative adversarial network (GAN)-based framework. We implement a deblur module by using contrastive learning to extract degradation feature and add symmetrical downsampling and upsampling modules to the SR network in order to restore texture information, while shortcut connections are redesigned to maintain the global similarity. Extensive experiments on the public satellite dataset, BUAA-SID-share1.5, demonstrate that our network outperforms the state-of-the-art SR and deblur methods.

Multi-scale recurrent attention network for image motion deblurring

Multi-scale recurrent attention network for image motion deblurring

Motion Blur Removal for Uav-Based Wind Turbine Blade Images Using Synthetic Datasets

Unmanned air vehicle (UAV) based imaging has been an attractive technology to be used for wind turbine blades (WTBs) monitoring. In such applications, image motion blur is a challenging problem which means that motion deblurring is of great significance in the monitoring of running WTBs. However, an embarrassing fact for these applications is the lack of sufficient WTB images, which should include better pairs of sharp images and blurred images captured under the same conditions for network model training. To overcome the challenge of image pair acquisition, a training sample synthesis method is proposed. Sharp images of static WTBs were first captured, and then video sequences were prepared by running WTBs at different speeds. The blurred images were identified from the video sequences and matched to the sharp images using image difference. To expand the sample dataset, rotational motion blurs were simulated on different WTBs. Synthetic image pairs were then produced by fusing sharp images and images of simulated blurs. Finally, a total of 4000 image pairs were obtained. To conduct motion deblurring, a hybrid deblurring network integrated with DeblurGAN and DeblurGANv2 was deployed. The results show that the integration of DeblurGANv2 and Inception-ResNet-v2 provides better deblurred images, in terms of both metrics of signal-to-noise ratio (80.138) and structural similarity (0.950) than those obtained from the comparable networks of DeblurGAN and MobileNet-DeblurGANv2.

Single Image Joint Motion Deblurring and Super-Resolution Using the Multi-Scale Channel Attention Modules

During the last decade, deep convolutional neural networks have significantly advanced the single image super-resolution techniques reconstructing realistic textural and spatial details. In classical image super-resolution problems, it is assumed that the low-resolution image has a certain downsampling degradation. However, complicated image degradations are inevitable in real-world scenarios, and motion blur is a common type of image degradation due to camera or scene motion during the image capturing process. This work proposes a fully convolutional neural network to reconstruct high-resolution sharp images from the given motion blurry low-resolution images. The deblurring subnetwork is based on multi-stage progressive architecture, while the super-resolution subnetwork is designed using the multi-scale channel attention modules. A simple and effective training strategy is employed where a pre-trained frozen deblurring module is used to train the super-resolution module. The deblurring module is unfrozen in the last training phase. Experiments show that, unlike the other methods, the proposed method reconstructs relatively small structures and textural details while successfully removing the complex motion blur. The implementation code and the pre-trained model are publicly available at https://github.com/misakshoyan/joint-motion-deblur-and-sr.

A Particle Method for Solving Fredholm Equations of the First Kind

Fredholm integral equations of the first kind are the prototypical example of ill-posed linear inverse problems. They model, among other things, reconstruction of distorted noisy observations and indirect density estimation and also appear in instrumental variable regression. However, their numerical solution remains a challenging problem. Many techniques currently available require a preliminary discretization of the domain of the solution and make strong assumptions about its regularity. For example, the popular expectation maximization smoothing (EMS) scheme requires the assumption of piecewise constant solutions which is inappropriate for most applications. We propose here a novel particle method that circumvents these two issues. This algorithm can be thought of as a Monte Carlo approximation of the EMS scheme which not only performs an adaptive stochastic discretization of the domain but also results in smooth approximate solutions. We analyze the theoretical properties of the EMS iteration and of the corresponding particle algorithm. Compared to standard EMS, we show experimentally that our novel particle method provides state-of-the-art performance for realistic systems, including motion deblurring and reconstruction of cross-section images of the brain from positron emission tomography.

Motion-Blurring Free Integral Imaging Reconstruction via Parallax Information

In practical applications, the target objects or cameras are usually moving during the exposure time of the camera array in synthetic aperture integral imaging. Thus, the elemental images we captured are all blurred and can not be used to 3D reconstruction. The existing deblurring methods are most designed for single-image deblurring and just a few for multi-image deblurring. Nevertheless, all of these methods can not be used for our problem because they contain no parallax information. To address this problem, we proposed a motion-blurring free 3D reconstruction in synthetic aperture integral imaging with parallax information. We calculate all the PSFs by using reference PSF and parallax information to get a 3D reconstruction result rapidly. The results of experiments we performed show that our proposed method is effective in integral imaging motion deblurring and 3D reconstruction.

Dual attention per-pixel filter network for spatially varying image deblurring

Spatially varying motion deblurring has recently witnessed substantial progress due to the development of deep neural network. However, most existing CNN-based methods involve two major shortcomings: (1) The CNN weights are space-sharing, and these methods thus ignore the properties of complex spatially variant blurs which vary from pixel to pixel in natural blurry images. (2) Stacked convolution layers with a large kernel or recurrent neural networks (RNNs) cannot capture the global contextual dependence of features, they thus cannot exploit the relationship between different blur pixels at a distance. To solve these problems, we propose a new dual attention per-pixel filter network (DAPFN). First, we develop a multiscale per-pixel filter network (MSPFN) to learn a specific deblurring mapping for different blur pixels, which predicts the per-pixel spatially adaptive convolution kernel for each blur pixel in the input blurry image of different scales and restores the clean pixel by performing channel-wise spatially adaptive convolution with the local neighborhood pixels. Second, we develop a dual attention enhanced residual network (DAERN) to capture the global contextual dependence of the blurry images, which introduces a dual attention (DA) module consisting of the spatial self-attention module (SSA) and channel self-attention module (CSA). The fusion of the two attention modules helps to further improve the deblurring performance. Third, we propose a new receptive field selection (RFS) block to learn the nonlinear characteristics of spatially variant blurs, which enables the adaptive fusing of the features with different receptive fields and effectively enhances the network nonlinear representation ability. The experimental results on GOPRO dataset indicate that the average PSNR and SSIM of the proposed method reached 31.8455 and 0.9231, respectively. The results of extensive experiments pertaining to spatially varying image deblurring demonstrate that the proposed method outperforms the state-of-the-art image deblurring methods.

Deblurring Method for Motion Blurred Images based on GAN

The purpose of image motion deblur is to recover the underlying clear image from the corresponding blur image. In most traditional methods, the image recovery task is formulated as a problem of blur core estimation and use a priori to calculate. In this paper we proposes a generative adversarial network(GAN) model based on the mobilenet-V3 network structure to meet the needs of motion blurred image recovery on mobile devices. Based on traditional evaluation indicators, we propose a new evaluation metric on mobile device. Extensive experiments show that our method is superior to the competing methods.

Attentive deep network for blind motion deblurring on dynamic scenes

Non-uniform blind motion deblurring is a challenging yet important problem in image processing that receives enduring attention in the last decade. The non-uniformity nature of motion blurring leads to great variations on the blurring effects across image regions and over different images, which makes it very difficult to train an end-to-end deblurring neural network (NN) with good generalization performance. This paper introduces an attention mechanism for the blind deblurring NN, including both spatial and channel attention, so as to effectively handle the significant spatial variations on blurring effects. In the attention mechanism, the spatial attention is introduced in both the encoder for discriminative exploitation of image edges and smooth regions and the decoder for discriminative treatment on different regions with different blurring effects. The channel attention is introduced for better generalization performance of the NN, as it allows adaptive weighting on intermediate features for a particular image. Building such an attention mechanism into a multi-scale encoder–decoder framework, an attentive NN is developed for practical non-uniform blind image deblurring. The experiments on several benchmark datasets show that the proposed NN can effectively restore the images degraded by spatially-varying blurring, with state-of-the-art performance.