Unsupervised Coordinate-Based Video Denoising

Integrating video coding and denoising is a novel processing paradigm, bringing mutual benefits to both video processing tools. In this paper, we propose a novel video denoising approach of which the main idea is reusing motion estimation resources from the video coding module for video denoising. In most cases, the motion fields produced by real-time video codecs cannot be directly employed in video denoising, since they, as opposed to noise filters, tolerate errors in the motion field. In order to solve this problem, we propose a novel motion-field filtering step that refines the accuracy of the motion estimates to a degree that is required for denoising. Additionally, a novel temporal filter is proposed that is robust against errors in the estimated motion field. Numerical results demonstrate that the proposed denoising scheme is of low-complexity and compares favorably to the state-of-the-art video denoising methods.

The most challenging aspect of video and image denoising is to preserve texture and small details, while filtering out noise. To tackle such problem, we present two novel variants of the 3D Non-Local Means (NLM3D) which are suitable for videos and 3D images. The first proposed algorithm computes texture patterns for each pixel by using the LBP-TOP descriptor to modify the NLM3D weighting function. It also uses MSB (Most Significant Bits) quantization to improve robustness to noise. The second proposed algorithm filters homogeneous and textured regions differently. It analyses the percentage of non-uniform LBP patterns of a region to determine whether or not the region exhibits textures and/or small details. Quantitative and qualitative experiments indicate that the proposed approaches outperform well known methods for the video denoising task, especially in the presence of textures and small details.

In this paper, we present local and nonlocal algorithms for video denoising and simplification based on discrete regularization on graphs. The main difference between video and image denoising is the temporal redundancy in video sequences. Recent works in the literature showed that motion compensation is counter-productive for video denoising. Our algorithms do not require any motion estimation. In this paper, we consider a video sequence as a volume and not as a sequence of frames. Hence, we combine the contribution of temporal and spatial redundancies in order to obtain high quality results for videos. To enhance the denoising quality, we develop a robust method that benefits from local and nonlocal regularities within the video. We propose an optimized method that is faster than the nonlocal approach, while producing equally attractive results. The experimental results show the efficiency of our algorithms in terms of both Peak Signal to Noise Ratio and subjective visual quality.

Cascaded UNet for progressive noise residual prediction for structure-preserving video denoising

We investigate image and video denoising using adaptive dual-tree discrete wavelet packets (ADDWP), which is extended from the dual-tree discrete wavelet transform (DDWT). With ADDWP, DDWT subbands are further decomposed into wavelet packets with anisotropic decomposition, so that the resulting wavelets have elongated support regions and more orientations than DDWT wavelets. To determine the decomposition structure, we develop a greedy basis selection algorithm for ADDWP, which has significantly lower computational complexity than a previously developed optimal basis selection algorithm, with only slight performance loss. For denoising the ADDWP coefficients, a statistical model is used to exploit the dependency between the real and imaginary parts of the coefficients. The proposed denoising scheme gives better performance than several state-of-the-art DDWT-based schemes for images with rich directional features. Moreover, our scheme shows promising results without using motion estimation in video denoising. The visual quality of images and videos denoised by the proposed scheme is also superior.

Deep learning has become a prominent tool for video denoising. However, most existing deep video denoising methods require supervised training using noise-free videos. Collecting noise-free videos can be costly and challenging in many applications. Therefore, this paper aims to develop an unsupervised deep learning method for video denoising that only uses a single test noisy video for training. To achieve this, an unsupervised loss function is presented that provides an unbiased estimator of its supervised counterpart defined on noise-free video. Additionally, a temporal attention mechanism is proposed to exploit redundancy among frames. The experiments on video denoising demonstrate that the proposed unsupervised method outperforms existing unsupervised methods and remains competitive against recent supervised deep learning methods.

In this paper, we combine two powerful tools to handle the video denoising problem: one is an effective video denoising method based on highly sparse signal representation in local 3D transform domain, and the other is a low-rank matrix completion based video denoising method. Similarly, in our algorithm, a noisy video is processed in block-wise manner and for each processed block we form a 3D data array that we call “group” by stacking together blocks found similar to the currently processed one. “Collaborative filtering” exploits the correlation between grouped blocks and the corresponding highly sparse representation of the true signal in the transform domain. By employ low-rank matrix completion method in our framework, our technique is also robust to different types of noise, such as Gaussian additive noise and impulsive noise. Experiments demonstrate that our techniques produce state-of-the-art results for video denoising

Challenging motion, which tends to cause artifacts, is a key problem in the video denoising task. Recent video denoising methods have attempted to address this problem. However, they usually provide general performance evaluation on the overall dataset and cannot provide a comprehensive analysis for the influence of different motion levels. Thus, we questioned whether these methods can effectively deal with different scene motions. To this end, we synthesize a dataset containing videos with different motion levels and capture a new dataset that consists of videos involving large-scale motion. Then, we provide a comprehensive analysis on the elaborately collected datasets and find that, as the motion level increases, the performance of the denoising models based on implicit motion estimation (IME) declines sharply, while explicit motion estimation (EME) contributes to a more robust denoising quality. Therefore, in this work, we present an EME-embedded progressive denoising framework that fully considers the relationship between the noise removal and motion estimation. Specifically, we decouple video denoising into spatial denoising, EME-based frame reconstruction, and temporal refining processes. Spatial denoising improves the accuracy of EME process in the case of videos suffering from heavy noise, while the temporal refining process refines the denoised frame by utilizing temporal redundancy of the reconstructed motion-free frames. Extensive experiments demonstrate that the proposed method outperforms existing state-of-the-art methods, especially for videos containing large-scale motion.

In this paper, we propose a simultaneous rate control and video de-noising algorithm based on rate distortion optimization. According to our previous works, video de-noising can be performed by using rate distortion optimization with a lower bound quantization parameter (QP) constraint, where the lower bound QP is determined by the noise variance. Then, we find that the macroblock level rate control method in H.264 can be seen as an approximate solution of a rate distortion optimization problem with a specified rate distortion function. Based on these two studies, we integrate the video de-noising problem and rate control problem to a rate distortion optimization problem. We show the convexity of the problem and derive the optimal solution. To reduce the complexity, we propose to use a suboptimal solution based on simply thresholding. Some experiments are conducted to demonstrate the efficiency and effectiveness of the proposed method.

In this paper, we introduce the idea of using motion estimation resources from a video codec for video denoising. This is not straightforward because the motion estimators aimed for video compression and coding, tolerate errors in the estimated motion field and hence are not directly applicable to video denoising. To solve this problem, we propose a novel motion field filtering step that refines the accuracy of the motion estimates to a degree that is required for denoising. We illustrate the use of the proposed motion estimation method within a wavelet-based video denoising scheme. The resulting video denoising method is of low-complexity and receives comparable results with respect to the latest video denoising methods.

In this paper we propose a variational approach for video denoising, based on a total directional variation (TDV) regulariser proposed in [20, 21] for image denoising and interpolation. In the TDV regulariser, the underlying image structure is encoded by means of weighted derivatives so as to enhance the anisotropic structures in images, e.g. stripes or curves with a dominant local directionality. For the extension of TDV to video denoising, the space-time structure is captured by the volumetric structure tensor guiding the smoothing process. We discuss this and present our whole video denoising workflow. The numerical results are compared with some state-of-the-art video denoising methods.

Deep learning-based video denoising methods have achieved great performance improvements in recent years. However, the expensive computational cost arising from sophisticated network design has severely limited their applications in real-world scenarios. To address this practical weakness, we propose a multiscale spatio-temporal memory network for fast video denoising, named MSTMN, aiming at striking an improved trade-off between cost and performance. To develop an efficient and effective algorithm for video denoising, we exploit a multiscale representation based on the Gaussian-Laplacian pyramid decomposition so that the reference frame can be restored in a coarse-to-fine manner. Guided by a model-based optimization approach, we design an effective variance estimation module, an alignment error estimation module and an adaptive fusion module for each scale of the pyramid representation. For the fusion module, we employ a reconstruction recurrence strategy to incorporate local temporal information. Moreover, we propose a memory enhancement module to exploit the global spatio-temporal information. Meanwhile, the similarity computation of the spatio-temporal memory network enables the proposed network to adaptively search the valuable information at the patch level, which avoids computationally expensive motion estimation and compensation operations. Experimental results on real-world raw video datasets have demonstrated that the proposed lightweight network outperforms current state-of-the-art fast video denoising algorithms such as FastDVDnet, EMVD, and ReMoNet with fewer computational costs.

For the English classroom teaching video denoising algorithm, it is not only necessary to consider whether the noise removal of the output video is thorough, but also to consider the actual operating efficiency and robustness of the algorithm. In the process of the thesis research, after reading a large number of internal and external documents on video denoising algorithms and analyzing the pros and cons of various denoising algorithms, this paper proposes a new video denoising algorithm, which uses the recently proposed grid flow motion model based on camera motion compensation to generate denoised video. Compared with the current advanced video denoising schemes, our method processes noisy frames faster and has good robustness. In addition, this article improves the algorithm framework so that the algorithm can not only deal with offline video denoising, but also deal with online video denoising.

Video denoising can be described as the problem of mapping from a specific length of noisy frames to clean one. We propose a deep architecture based on Recurrent Neural Network (RNN) for video denoising. The model learns a patch-based end-to-end mapping between the clean and noisy video sequences. It takes the corrupted video sequences as the input and outputs the clean one. Our deep network, which we refer to as deep Recurrent Neural Networks (deep RNNs or DRNNs), stacks RNN layers where each layer receives the hidden state of the previous layer as input. Experiment shows (i) the recurrent architecture through temporal domain extracts motion information and does favor to video denoising, and (ii) deep architecture have large enough capacity for expressing mapping relation between corrupted videos as input and clean videos as output, furthermore, (iii) the model has generality to learned different mappings from videos corrupted by different types of noise (e.g., Poisson-Gaussian noise). By training on large video databases, we are able to compete with some existing video denoising methods.

Existing denoising methods typically restore clear results by aggregating\npixels from the noisy input. Instead of relying on hand-crafted aggregation\nschemes, we propose to explicitly learn this process with deep neural networks.\nWe present a spatial pixel aggregation network and learn the pixel sampling and\naveraging strategies for image denoising. The proposed model naturally adapts\nto image structures and can effectively improve the denoised results.\nFurthermore, we develop a spatio-temporal pixel aggregation network for video\ndenoising to efficiently sample pixels across the spatio-temporal space. Our\nmethod is able to solve the misalignment issues caused by large motion in\ndynamic scenes. In addition, we introduce a new regularization term for\neffectively training the proposed video denoising model. We present extensive\nanalysis of the proposed method and demonstrate that our model performs\nfavorably against the state-of-the-art image and video denoising approaches on\nboth synthetic and real-world data.\n