Unidirectional Total Variation Research Articles

Video Deraining Mechanism by Preserving the Temporal Consistency and Intrinsic Properties of Rain Streaks

The problem of video deraining is one of the most focused research areas where several techniques are introduced to achieve higher visual quality in the results. The process of video deraining tries to eliminate the presence of rain streaks in the videos so that the overall quality of the video can be enhanced. The existing frameworks tried to accurately eliminate the rain streaks in videos, but there is still room for improvement to preserve the temporal consistencies and intrinsic properties of rain streaks. This work does the job with the combination of handcrafted and deep priors that deeply extract and identify the nature of rain streaks in different dimensions. The proposed work included three main steps: Prior extraction, Derain modelling and optimization. Four major priors are extracted from the frames where the gradient prior (GP) and sparse prior (SP) are extracted from the rain streaks, and the smooth temporal prior (STP) and deep prior (DP) are extracted from the clean video. The unidirectional total variation (UTV) is applied to extract the GP, and the L1 normalization method is followed to extract the SP and STP. The DP is then extracted from the clean frames using the residual gated recurrent deraining network (Res-GRRN) model based on deep learning. Derain modelling is carried out based on the extracted priors, and the stochastic alternating direction multiplier method (SADMM) algorithm is utilized to solve the optimization problem. The proposed approach is then implemented in python and evaluated using the real-world dataset. The overall PSNR achieved by the proposed approach is 39.193dB, which is more optimal than the existing methods.

Unsupervised Seismic Footprint Removal With Physical Prior Augmented Deep Autoencoder

Seismic acquisition footprints appear as stably faint and dim structures and emerge fully spatially coherent, causing inevitable damage to useful signals during the suppression process. Various footprint removal methods, including filtering and sparse representation (SR), have been reported to attain promising results for surmounting this challenge. However, these methods, e.g., SR, rely solely on the handcrafted image priors of useful signals, which is sometimes an unreasonable demand if complex geological structures are contained in the given seismic data. As an alternative, this article proposes a footprint removal network (dubbed FR-Net) for the unsupervised suppression of acquired footprints without any assumptions regarding valuable signals. The key to the FR-Net is to design a unidirectional total variation (UTV) model for footprint acquisition according to the intrinsically directional property of noise. By strongly regularizing a deep convolutional autoencoder (DCAE) using the UTV model, our FR-Net transforms the DCAE from an entirely data-driven model to a \textcolor{black}{prior-augmented} approach, inheriting the superiority of the DCAE and our footprint model. Subsequently, the complete separation of the footprint noise and useful signals is projected in an unsupervised manner, specifically by optimizing the FR-Net via the backpropagation (BP) algorithm. We provide qualitative and quantitative evaluations conducted on three synthetic and field datasets, demonstrating that our FR-Net surpasses the previous state-of-the-art (SOTA) methods.

Improved Low-Rank Tensor Approximation for Seismic Random Plus Footprint Noise Suppression

Random plus footprint noise provokes severe seismic image deterioration and makes it challenging for interpreters to recognize and analyze accurate subsurface responses. Thus, as an elementary and indispensable preprocessing step, diverse footprint removal approaches, including filtering in the frequency or time-frequency domain and dictionary learning (DL), have been documented to achieve promising results in tackling this challenge. However, the prevailing denoising methods tend to treat 3-D seismic data as images for processing, but such flattening or matricization operations inevitably obliterate the three-dimensional image structures concealed in noisy observational seismic data, which hinders the removal performance of these approaches. To resolve this issue, this article proposes a new tensor model for 3-D seismic random plus footprint noise suppression, and this model is based on unidirectional total variation regularized low-rank tensor approximation (UTV-LRTA). In this model, UTV regularization is imposed to obtain the innately structural and directional behavior of the acquisition footprint. In this way, the footprint is removed by effectively decomposing the footprint-contaminated seismic image into a footprint-free image and a footprint component by UTV. In contrast, random noise is mitigated by regularizing the low-rankness of the third-order seismic tensors using the tensor nuclear norm. Moreover, a simple and powerful optimization algorithm based on the split Bregman iteration is introduced to resolve the proposed UTV-LRTA model. The suggested model is thoroughly assessed on synthetic and field datasets and significantly surpasses the state-of-the-art approaches quantitatively and qualitatively evaluated in the analyzed field examples.

Flexible infrared images destriping algorithm with L1-based sparse regularization for wide-field astronomical images

A novel infrared image destriping method is proposed from a single image. The proposed method can estimate the clean image and stripe noise image simultaneously. Firstly, we analyze the common challenges in current destriping algorithms, and then integrate an unidirectional total variation constraint to remove stripe noise effectively. Wavelet transform is introduced to constrain the spatial mixture Gaussian noise in the degraded images. We employ the total variation regularization to smooth the distribution of Wavelet coefficients. Namely, the distribution of Wavelet coefficients of clean image is sparser than that of degraded stripe image. Finally, an efficient optimization strategy is described to alternately estimate the clean image and stripe image until convergence. Compared experiments are carried out on the simulated and real infrared images with the state-of-the-art models. The proposed method is able to recover high resolution infrared image in low computation cost.

Nonlocal Block-Term Decomposition for Hyperspectral Image Mixed Noise Removal

Since the facility restrictions and weather conditions, hyperspectral image (HSI) is generally seriously polluted by a variety of noises. Recently, the method based on block-term decomposition with rank- $(L, L, 1)$ (BTD) has attracted wide attention in HSI mixed noise removal. BTD factorizes third-order HSI data into the sum of a series of component tensors, where each of the component tensors is represented by the outer product of a rank- $L$ matrix $\mathbf {A}_r\mathbf {B}_r^T$ and a column vector $\mathbf {c}_r$ . BTD has clear physical interpretation because its latent factors $\mathbf {A}_r\mathbf {B}_r^T$ and $\mathbf {c}_r$ can be interpreted abundance map and spectral signature, respectively. The essential uniqueness of BTD is under the low-rank assumption of $\mathbf {A}_r\mathbf {B}_r^T$ . However, the low-rank assumption is not always held in real scenarios. The BTD-based method usually sets $L$ to full rank to achieve satisfactory results. In this article, we suggest a novel model based on nonlocal block-term decomposition (NLBTD) for HSI mixed noise removal. More specifically, for each grouped similar image block, BTD is introduced to capture nonlocal self-similarity and global spectral low-rankness, the unidirectional total variation is introduced to preserve local spectral smoothness. By faithfully exploring nonlocal self-similarity, global spectral low-rankness, and local spectral smoothness, the proposed method is expected to produce promising results with guarantee the essential uniqueness of BTD. To tackle the resulting model, we design an efficient algorithm based on the proximal alternating minimization with the theoretical guarantees. Extensive numerical experiments in HSI mixed noise removal demonstrate that the proposed NLBTD method achieves satisfactory performance compared with state-of-the-art methods.

Thick Cloud Removal of Remote Sensing Images Using Temporal Smoothness and Sparsity Regularized Tensor Optimization

In remote sensing images, the presence of thick cloud accompanying shadow can affect the quality of subsequent processing and limit the scenarios of application. Hence, to make good use of such images, it is indispensable to remove the thick cloud and cloud shadow as well as recover the cloud-contaminated pixels. Generally, the thick cloud and cloud shadow element are not only sparse but also smooth along the spatial horizontal and vertical direction, while the clean element is smooth along the temporal direction. Guided by the above insight, a novel thick cloud removal method for remote sensing images based on temporal smoothness and sparsity regularized tensor optimization (TSSTO) is proposed in this paper. Firstly, the sparsity norm is utilized to boost the sparsity of the cloud and cloud shadow element, and unidirectional total variation (UTV) regularizers are applied to ensure the smoothness in different directions. Then, through thresholding, the cloud mask and the cloud shadow mask can be acquired and used to guide the substitution. Finally, the reference image is selected to reconstruct details of the repairing area. A series of experiments are conducted both on simulated and real cloud-contaminated images from different sensors and with different resolutions, and the results demonstrate the potential of the proposed TSSTO method for removing cloud and cloud shadow from both qualitative and quantitative viewpoints.

Fast stripe noise removal from hyperspectral image via multi-scale dilated unidirectional convolution

Hyperspectral images (HSIs) are often contaminated by noises due to the multi-detector imaging systems, which greatly affects the subsequent HSIs interpretation and application. The 3D HSIs deliver extra spectral information, which makes the most existing destriping algorithms hardly satisfied, and the complete stripes removal and less test time consuming remain to be overcome. To meet these challenges, we present a multi-scale dilated unidirectional convolution network (MsDUC) with the following contributions. First, the deep learning-based method can fully exploit and preserve spatial-spectral correlations in 3D HSIs while the conventional methods failed to realize it. Second, different dilated convolution learns different scale features, so the introduced multi-scale dilated convolution could get more contextual information for the final restoration. Third, the clear directional signature of stripe noise and the unidirectional total variation (UTV) model inspired us to put forward the unidirectional convolution to capture the directional signature of stripe, meanwhile, the less trainable parameters and the utilized residual strategy speed up the learning process. Experimental results have shown that our method outperforms many of the state-of-the-art methods in both image restoration performance and test running time. Our code can be download from .

Mixed-Noise Band Selection for Hyperspectral Images

Hyperspectral images (HSIs) with abundant spectral information are generally susceptible to various types of noise, such as Gaussian noise and stripe noise. Recently, a few quality-based selection algorithms have been proposed to remove noise bands from HSIs. However, these methods suffer from an inability to discriminate the mixed-noise bands of HSIs and are sensitive to image content variation and luminance changes. Here, we develop a mixed-noise band selection framework that can separate the Gaussian and stripe noise bands from HSIs effectively. We first improve tensor decomposition to reconstruct the mixed-noise components and low-rank components, which reduces the influence of image content and luminance changes. Spectral smoothness constraints and unidirectional total variation are incorporated into the decomposition model to enhance the separation performance for Gaussian and stripe noise. Then, different statistical features, including Weibull and histogram of oriented gradient (HOG) features, are applied to extract the robust parameters from mixed-noise components. More importantly, an extreme learning machine (ELM) is trained to predict the noise bands. The ELM has an extremely fast learning speed and tends to achieve better performance than other networks. Finally, by aggregating all these strategies, our methods can select the mixed-noise bands efficiently. The experimental results on both synthetic and real noise HSIs indicate that the proposed method outperforms the state-of-the-art methods.

Hyperspectral Image Superresolution Using Unidirectional Total Variation With Tucker Decomposition

The hyperspectral image superresolution (HSI-SR) problem aims to improve the spatial quality of a low spatial resolution HSI by fusing the LR-HSI with the corresponding high spatial resolution multispectral image. The generated HSI with high spatial quality, i.e., the target high spatial resolution hyperspectral image (HR-HSI), generally has some fundamental latent properties, e.g., the sparsity, and the piecewise smoothness along with the three modes (i.e., width, height, and spectral mode). However, limited works consider both properties in the HSI-SR problem. In this work, a novel unidirectional total variation (TV) based approach is been proposed. On the one hand, we consider that the target HR-HSI exhibits both the sparsity and the piecewise smoothness on the three modes, and they can be depicted well by the $\ell _1$ -norm and TV, respectively. On the other hand, we utilize the classical Tucker decomposition to decompose the target HR-HSI (a three-mode tensor) as a sparse core tensor multiplied by the dictionary matrices along with the three modes. Especially, we impose the $\ell _1$ -norm on core tensor to characterize the sparsity and the unidirectional TV on three dictionaries to characterize the piecewise smoothness. The proximal alternating optimization scheme and the alternating direction method of multipliers are used to iteratively solve the proposed model. Experiments on three common datasets illustrate that the proposed approach has better performance than some current state-of-the-art HSI-SR methods. Please find source code from: https://liangjiandeng.github.io/ .

Reweighted Block Sparsity Regularization for Remote Sensing Images Destriping

Destriping has attracted a lot of interest in the field of remote sensing images processing. Recent research works consider the characteristics of stripes, among which low-rankness and group sparsity regularizations are widely used. However, for real scenarios, these regularizations sometimes fail to work when the underlying assumptions are violated. Motivated by this observation, we propose a novel and flexible regularization, named as the reweighted block sparsity (RBS), for complex stripes (e.g., the partial stripes). RBS divides the stripe layer into several blocks along the stripe-perpendicular direction and boosts the group sparsity of each block, where the sparsity level is adaptively controlled by updating weights. Based on RBS, we further propose a destriping model by integrating RBS and unidirectional total variation regularization, which can better detect and remove complex stripes. Moreover, we develop an alternating direction method of multipliers algorithm to solve the proposed model. Experimental results demonstrate that the proposed method outperforms the state-of-the-art competitors qualitatively and quantitatively.

Video deraining via nonlocal low-rank regularization

Outdoor videos captured in rainy weather may be significantly corrupted by the undesired rain streaks, which severely affect the video processing tasks in outdoor computer vision systems. In this paper, we propose a tensor-based video rain streaks removal method using the nonlocal low-rank regularization. Specifically, we first divide videos into overlapped spatial–temporal patches. Then for each patch, we group its nonlocal similar spatial–temporal patches to form a third-order tensor. To model the clean videos, we characterize the wealth redundancy by adopting the tensor nuclear norm to regularize the low-rankness of the third-order tensors formed by similar spatial–temporal patches of clean videos. We also consider the piecewise smoothness and the temporal continuity of clean videos and utilize the unidirectional total variation to enhance the smoothness and continuity. Moreover, as rain streaks are sparse and smooth along the rain direction, we model the rain streaks by employing an ℓ1 norm and the unidirectional total variation penalty to boost the sparsity and directional smoothness, respectively. We develop an efficient alternating direction method of multipliers to solve the proposed model. Experimental results on both synthetic and real rainy videos show that our method outperforms the state-of-the-art methods quantitatively and qualitatively.

A Method for the Destriping of an Orbita Hyperspectral Image with Adaptive Moment Matching and Unidirectional Total Variation

The Orbita hyperspectral satellite (OHS) is the first hyperspectral satellite with surface coating technology for sensors in the world. It includes 32 bands from visible to near-infrared wavelengths. However, technology such as the fabricating process of complementary metal–oxide–semiconductor (CMOS) sensors makes the image contain a lot of random and unsystematic stripe noise, which is so bad that it seriously affects visual interpretation, object recognition and the application of the OHS data. Although a large number of stripe removal algorithms have been proposed, very few of them take into account the characteristics of OHS sensors and analyze the causes of OHS data noise. In this paper, we propose a destriping algorithm for OHS data. Firstly, we use both the adaptive moment matching method and multi-level unidirectional total variation method to remove stripes. Then a model based on piecewise linear least squares fitting is proposed to restore the vertical details lost in the first step. Moreover, we further utilize the spectral information of the OHS image, and extend our 2-D destriping method to the 3-D case. Results demonstrate that the proposed method provides the optimal destriping result on both qualitative and quantitative assessments. Moreover, the experimental results show that our method is superior to the existing single-band and multispectral destriping methods. Also, we further use the algorithm to the stripe noise removal of other real remote sensing images, and excellent image quality is obtained, which proves the universality of the algorithm.

Remote sensing images destriping using unidirectional hybrid total variation and nonconvex low-rank regularization

In this paper, we propose a novel model for remote sensing images destriping, which includes the Schatten 1 ∕ 2 -norm and the unidirectional first-order and high-order total variation regularization. The main idea is that the stripe layer is low-rank, and the desired image possesses smoothness across stripes. Therefore, we use the Schatten 1 ∕ 2 -norm regularization to depict the low-rankness of stripes, and use the unidirectional total variation and the unidirectional high-order total variation to guarantee the smoothness of the underlying image. We develop the alternating direction method of multipliers algorithm to solve the proposed model. Extensive experiments on synthetic and real data are reported to show the superiority of the proposed method over state-of-the-art methods in terms of both quantitative and qualitative assessments.

A sheared low-rank model for oblique stripe removal

Stripe noise removal from remote sensing imagery is a key preprocessing step for many applications, such as urban planning, military, and environmental monitoring. Two significant priors of the stripe are low-rankness and group sparsity, which play a central role in many destriping methods. However, for oblique stripes, these two priors are no longer explicitly applicable. To overcome this difficulty, we propose a novel sheared decomposition framework for oblique stripe removal. The main idea is to perform a shear operation on the observed image, so that the oblique stripe is transformed to nearly vertical lines. For the stripe component, we characterize the low-rankness and group sparsity priors in the sheared domain. For the image component, we characterize the smoothness prior in the original domain by unidirectional total variation (UTV), which can preserve the strong edges and geometric features while suppressing stripes. To solve the proposed model, we design an efficient alternating direction method of multipliers (ADMM) algorithm with guaranteed convergence. Experiments using simulated and real data show that the proposed method can remove the oblique stripe effectively and outperform the state-of-the-art methods qualitatively and quantitatively.

Remote Sensing Image Stripe Detecting and Destriping Using the Joint Sparsity Constraint with Iterative Support Detection

Remote sensing images have been applied to a wide range of fields, but they are often degraded by various types of stripes, which affect the image visual quality and limit the subsequent processing tasks. Most existing destriping methods fail to exploit the stripe properties adequately, leading to suboptimal performance. Based on a full consideration of the stripe properties, we propose a new destriping model to achieve stripe detection and stripe removal simultaneously. In this model, we adopt the unidirectional total variation regularization to depict the directional property of stripes and the weighted ℓ 2 , 1 -norm regularization to depict the joint sparsity of stripes. Then, we combine the alternating direction method of multipliers and iterative support detection to solve the proposed model effectively. Comparison results on simulated and real data suggest that the proposed method can remove and detect stripes effectively while preserving image edges and details.

Removing ring artifacts in cone-beam CT via TV-Stokes and unidirectional total variation model.

It often happens that cone-beam corrupted tomography (CBCT) images have some ring artifacts due to the inconsistent response of detector pixels. Removing ring artifacts in CBCT images without impairing the image quality is critical. The purpose of this study is to implement a whole new method of removing ring artifacts in CBCT images based on TV-Stokes denoising equationand unidirectional total variation (UTV). This method is based on the polar coordinates, where the ring artifacts are shown as horizontal parallel stripes. To begin with, we design a UTV model with the constraint of zero divergence condition to only smooth vertical tangent vectors, and keep horizontal tangent vectors unchanged. In turn, the corresponding smooth normal vectors can be obtained. Next, in order to reconstruct the clean image that fits the obtained normal vectors, the UTV model about the potential clean image is added to the original TV-Stokes denoising equation. Our method was applied to simulated data and real corrupted data to verify its performance. High-quality corrected images without ring artifacts were obtained. In the simulated experiments, our method is not only able to obtain the most complete corrected images, but also capable of acquiring the best quantitative assessments among several different methods. In the experiments of real data, the proposed method was effective in removing ring artifacts and in preserving the original details. Comparative results on several experiments illustrated that our algorithm corrects ring artifacts effectively and outperforms the two compared methods on objective indices and subjective image quality. Benefiting from the TV-Stokes equationand the UTV model, the proposed scheme can be effectively applied to remove ring artifacts in CBCT images, simultaneously preserving the original image details and structure well. Furthermore, this new algorithm can be applied directly on reconstructed images, thus eliminating the requirement for additional imaging data.

A Tensor-Based Low-Rank Model for Single-Image Rain Streaks Removal

Images were taken in rainy weather always contain unexpected rain streaks, which severely affect the subsequent image processing procedures in outdoor vision systems. Removing rain streaks from a single image is a challenging task and recently has been investigated extensively. In this paper, we propose a novel tensor-based low-rank model for removing rain streaks from a single rainy image. Our method fully exploits the similar repeated patterns and directional smoothness of rain streaks. Different from the existing matrix-based methods, we stack the rain patches to construct a three-order tensor and characterize the similarity and repeated patterns by considering the low-rankness in tensor form. We further regularize the low-rankness by the efficient tensor nuclear norm (TNN) so that the intrinsic spatial structures of rain streaks can be preserved. Moreover, two unidirectional total variation terms are employed to depict the directional smoothness of rain streaks and the rain-free image. The sparsity of rain streaks is also enhanced by an l 1 norm. We develop an efficient alternating direction method of multipliers (ADMM) to tackle the proposed model. The experimental results on synthetic and real-world rain images show that our method outperforms the state-of-the-art methods quantitatively and visually.

A total variation and group sparsity based tensor optimization model for video rain streak removal

Rain streak removal is an important issue of the outdoor vision system and has been investigated extensively. In this paper, we propose a novel tensor optimization model for video rain streak removal by fully considering the discriminatively intrinsic characteristics of the rain streaks and the clean video. In specific, the rain streaks are group sparse and smooth along the rain streaks direction; the clean video is smooth along the perpendicular direction of the rain streaks and the temporal direction. For the rain streaks, we use ℓ2,1 norm to characterize the group sparsity and the unidirectional total variation (UTV) to promote the smoothness along the rain streaks direction. For the clean video, we use two UTV regularizers to enhance the smoothness along the perpendicular direction of the rain streaks and the temporal direction. To solve the proposed model we develop an efficient alternating direction method of multipliers (ADMM) algorithm. Experiments on synthetic and real data demonstrate the superiority of the proposed method over state-of-the-art methods both quantitatively and qualitatively.

Destriping of Multispectral Remote Sensing Image Using Low-Rank Tensor Decomposition

Multispectral image (MSI) destriping is a challenging topic and has been attracting much research attention in remote sensing area due to its importance in improving the image qualities and subsequent applications. The existing destriping methods mainly focus on matrix-based modeling representation, which fails to fully discover the correlation of the stripe component in both spatial dimensions. In this paper, we propose a novel low-rank tensor decomposition framework based MSI destriping method by decomposing the striped image into the image component and stripe component. Specifically, for the image component, we use the anisotropic spatial unidirectional total variation (TV) and spectral TV regularization to enhance the piecewise smoothness in both spatial and spectral domains. Moreover, for the stripe component, we adopt tensor Tucker decomposition and $\ell _{2,1}$ -norm regularization to model the spatial correlation and group sparsity characteristic among all bands, respectively. An efficient algorithm using the augmented Lagrange multiplier method is designed to solve the proposed optimization model. Experiments under various cases of simulated data and real-world data demonstrate the effectiveness of the proposed model over the existing single-band and MSI destriping methods in terms of the qualitative and quantitative.

FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors.

Rain streaks removal is an important issue in outdoor vision systems and has recently been investigated extensively. In this paper, we propose a novel video rain streak removal approach FastDeRain, which fully considers the discriminative characteristics of rain streaks and the clean video in the gradient domain. Specifically, on the one hand, rain streaks are sparse and smooth along the direction of the raindrops, whereas on the other hand, clean videos exhibit piecewise smoothness along the rain-perpendicular direction and continuity along the temporal direction. Theses smoothness and continuity results in the sparse distribution in the different directional gradient domain, respectively. Thus, we minimize 1) the ℓ1 norm to enhance the sparsity of the underlying rain streaks, 2) two ℓ1 norm of unidirectional Total Variation (TV) regularizers to guarantee the anisotropic spatial smoothness, and 3) an ℓ1 norm of the time-directional difference operator to characterize the temporal continuity. A split augmented Lagrangian shrinkage algorithm (SALSA) based algorithm is designed to solve the proposed minimization model. Experiments conducted on synthetic and real data demonstrate the effectiveness and efficiency of the proposed method. According to comprehensive quantitative performance measures, our approach outperforms other state-of-the-art methods, especially on account of the running time. The code of FastDeRain can be downloaded at https://github.com/TaiXiangJiang/FastDeRain.