Nuclear Norm Minimization Research Articles

Hyperspectral and Multispectral Image Fusion via Superpixel-Based Weighted Nuclear Norm Minimization

Hyperspectral and Multispectral Image Fusion via Superpixel-Based Weighted Nuclear Norm Minimization

Coarray Tensor Completion for DOA Estimation

Sparse array direction-of-arrival (DOA) estimation using tensor model has been developed to handle multi-dimensional sub-Nyquist sampled signals. Furthermore, augmented virtual arrays can be derived for Nyquist-matched coarray tensor processing. However, the partially augmentable sparse array corresponds to a discontinuous virtual array, whereas the existing methods can only utilize its continuous part. Conventional virtual linear array interpolation techniques complete coarray covariance matrices with dispersed missing elements, but fail to complete the coarray tensor with whole missing slices. In this paper, we propose a coarray tensor completion algorithm for two-dimensional DOA estimation, where the coarray tensor statistics can be entirely exploited. In particular, in order to impose an effective low-rank regularization on the slice-missing coarray tensor, we propose shift dimensional augmenting and coarray tensor reshaping approaches to reformulate a structured coarray tensor with sufficiently dispersed missing elements. Furthermore, the shape of the reformulated coarray tensor is optimized by maximizing the dispersion-to-percentage ratio of missing elements. As such, a coarray tensor nuclear norm minimization problem can be designed to optimize the completed coarray tensor corresponding to a filled virtual array, based on which the closed-form DOA estimation is achieved. Meanwhile, the global convergence of the coarray tensor completion is theoretically proved. Simulation results demonstrate the effectiveness of the proposed algorithm compared to other matrix-based and tensor-based methods.

Randomized Tensor Robust PCA for Noisy Hyperspectral Image Classification

Removing noise from hyperspectral images can be very beneficial for improving classification accuracy. Recently, tensor robust principal component analysis (TRPCA) has been successfully employed to reduce noise in hyperspectral images. In TRPCA, a minimization involving a tensor nuclear norm and a ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm is employed to separate the low-rank hyperspectral image from the sparse noise. Tensor nuclear norm minimization is solved by iteratively performing tensor singular value thresholding (T-SVT). However, TRPCA possesses high computational complexity primarily due to the implementation of the T-SVT operator. The conventional approach for T-SVT is first to perform full tensor singular value decomposition (T-SVD), and then to shrink the singular values of the frontal slices in the frequency domain. However, when the solution of tensor nuclear norm minimization is a low-rank tensor, a good strategy is to incrementally find the singular values until they fall below the threshold. In this letter, we propose a randomized blocked algorithm for computing tensor singular value thresholding, and we leverage the compression attained by the fast Fourier transform (FFT) to accelerate TRPCA. Numerical experiments indicate that our method is significantly faster than performing TRPCA via the full T-SVD while maintaining classification accuracy.

Impulse Noise Removal by L1 Weighted Nuclear Norm Minimization

Impulse Noise Removal by L1 Weighted Nuclear Norm Minimization

A WTC suppression algorithm for weather radar based on weighted schatten p-norm minimization

ABSTRACT The Matrix Completion (MC) method can effectively suppress the wind turbine clutter (WTC) of the weather radar, however its recovery accuracy suffers from the low signal-to-noise ratio (SNR) of the real measured weather signal. Moreover, the singular value distribution of the observation matrix constructed by the measured radar data has no sharp cut-off characteristic, which results the rank can not be well approximated and also reduces the recovery accuracy. In this paper, a non-convex MC algorithm based on weighted Schatten p-norm is proposed for the WTC suppression in Doppler frequency domain. Experiments are carried out on the measured signal data, and the results demonstrate that compared with the commonly used nuclear norm minimization (NNM) model, the weighted Schatten p-norm minimization (WSNM) model can further improve the recovery accuracy of weather signal and realizes the superior estimation performance of spectral moment.

DOA Estimation via Tensor Truncated Convolution Nuclear Norm Minimization in MIMO Radar Under Array Sensor Failures

Exploring the inherent multidimensional structures has great potential to obtain more latent information for reconstructing the missing data resulting from array sensor failures in multiple-input–multiple-output (MIMO) radar. Most tensor completion (TC) methods based on the minimization of tensor nuclear norms are adopted to fill randomly distributed missing entries. However, the randomness of missing entries is violated in MIMO radar under sensor failures, where some slices are missing entirely. Thus, these existing methods become ineffective when faced with such type of structurally missing entries. To remedy this, we propose an improved TC method based on tensor truncated convolution nuclear norm minimization (CNNM) to reconstruct the failed sensor data in MIMO radar in order to handle sensor failure in direction-of-arrival (DOA) estimation. First, we formulate a new tensor by convoluting the third-order tensor of MIMO radar with a kernel tensor in a truncated manner and establish a low-rank TC model equipped with the truncated CNNM. Then, we introduce a tractable relaxation of our minimization function by deducing the equality relation between the truncated convolution nuclear norm of a tensor and the nuclear norm of the truncated convolution matrix of the same tensor. Finally, we derive an efficient algorithm implementation based on alternating direction method of multipliers (ADMM) to tackle our problem. Simulation results show that the proposed method is quite effective in completing tensors with structurally missing entries and facilitating more accurate data recovery by exploiting the inherent multidimensional nature. This provides superior DOA estimation performance in MIMO radar.

DOA Estimation of Mixed Circular and Noncircular Sources Using Nonuniform Linear Array

In this article, an algorithm for accurately estimating the directions-of-arrival (DOAs) of mixed circular and noncircular signals with a nonuniform linear array is devised. Via applying coarray interpolation on the difference and sum coarrays, we first formulate two nuclear norm minimization problems to reconstruct an augmented covariance matrix and an augmented unconjugated covariance matrix, respectively. Based on the reconstructed matrices, we further build the extended covariance matrix in the coarray domain. Finally, we construct several spectral functions to successively estimate the DOAs of the nonstrictly noncircular, strictly noncircular, and circular signals. Unlike the existing schemes, the proposed algorithm can identify accurately circular, strictly noncircular, and nonstrictly noncircular signals. Numerical results demonstrate the superiority of the proposed algorithm over existing techniques.

Remote Sensing Image Reconstruction Based on Improved Adaptive Tensor Nuclear Norm Minimization

Incomplete remote sensing image observation data require proper processing to restore the original data. Nevertheless, the lack of information, due to noise and dead pixels, along with the quality of the acquired images that may be severely degraded, limits the accuracy of later processing. Therefore, this paper proposes a remote sensing image restoration scheme that employs the low-rank structure of the Bayesian probability tensor latent space and a new tensor completion method, which has strong robustness for the model selection process of the following remote sensing image data analysis. Considering the data space, we incorporate an adaptive nuclear norm regularization into the processing of latent factors that performs singular value decomposition on a more accurate scale. The proposed model adopts two algorithms: Augmented Lagrange Multiplier (ALM) and Alternating Least Squares (ALS). Several synthetic and real-data-based experiments illustrate the inherent ability of our method in restoring original information and preventing overfitting problems, even when a vast number of items are missing. In addition, the combination of adaptive nuclear norm regularization and tensor decomposition is stable, settling a high sensitivity of rank selection and low computational efficiency in traditional tensor reconstruction methods.

Iteratively Reweighted Minimax-Concave Penalty Minimization for Accurate Low-rank Plus Sparse Matrix Decomposition.

Low-rank plus sparse matrix decomposition (LSD) is an important problem in computer vision and machine learning. It has been solved using convex relaxations of the matrix rank and l0-pseudo-norm, which are the nuclear norm and l1-norm, respectively. Convex approximations are known to result in biased estimates, to overcome which, nonconvex regularizers such as weighted nuclear-norm minimization and weighted Schatten p-norm minimization have been proposed. However, works employing these regularizers have used heuristic weight-selection strategies. We propose weighted minimax-concave penalty (WMCP) as the nonconvex regularizer and show that it admits an equivalent representation that enables weight adaptation. Similarly, an equivalent representation to the weighted matrix gamma norm (WMGN) enables weight adaptation for the low-rank part. The optimization algorithms are based on the alternating direction method of multipliers technique. We show that the optimization frameworks relying on the two penalties, WMCP and WMGN, coupled with a novel iterative weight update strategy, result in accurate low-rank plus sparse matrix decomposition. The algorithms are also shown to satisfy descent properties and convergence guarantees. On the applications front, we consider the problem of foreground-background separation in video sequences. Simulation experiments and validations on standard datasets, namely, I2R, CDnet 2012, and BMC 2012 show that the proposed techniques outperform the benchmark techniques.

Matrix Completion With Data-Dependent Missingness Probabilities

The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> such that each entry of the matrix is available independently with probability <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$f$ </tex-math></inline-formula> of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$f$ </tex-math></inline-formula> .

Sparsity-Based Two-Dimensional DOA Estimation for Co-Prime Planar Array via Enhanced Matrix Completion

In this paper, the two-dimensional (2-D) direction-of-arrival (DOA) estimation problem is explored for the sum-difference co-array (SDCA) generated by the virtual aperture expansion of co-prime planar arrays (CPPA). Since the SDCA has holes, this usually causes the maximum virtual aperture of CPPA to be unavailable. To address this issue, we propose a complex-valued, sparse matrix recovery-based 2-D DOA estimation algorithm for CPPA via enhanced matrix completion. First, we extract the difference co-arrays (DCA) from SDCA and construct the co-array interpolation model via nuclear norm minimization to initialize the virtual uniform rectangular array (URA) that does not contain the entire rows and columns of holes. Then, we utilize the shift-invariance structure of the virtual URA to construct the enhanced matrix with a two-fold Hankel structure to fill the remaining empty elements. More importantly, we apply the alternating direction method of the multipliers (ADMM) framework to solve the enhanced matrix completion model. To reduce the computational complexity of the traditional vector-form, sparse recovery algorithm caused by the Kronecker product operation between dictionary matrices, we derive a complex-valued sparse matrix-recovery model based on the fast iterative shrinkage-thresholding (FISTA) method. Finally, simulation results demonstrate the effectiveness of the proposed algorithm.

Spatio-Temporal Changes in Air Quality of the Urban Area of Chongqing from 2015 to 2021 Based on a Missing-Data-Filled Dataset

Air pollution is one of the severe environmental issues in Chongqing. Many measures made by the government for improving air quality have been put into use these past few years, while the influence of these measures remains unknown. This study analyzed the changes in the air quality of the urban area of Chongqing between 2015 and 2021 using a complete in situ observation dataset that all missing data were filled by the interpolation of a low-rank tensor completion model with truncate nuclear norm minimization (LRTC-TNN). The results include: (1) the LRTC-TNN model robustly performs to reconstruct missing data of pollutant concentrations with an R2 of 0.93 and an RMSE of 7.78; (2) the air quality index (AQI) decreases by 15.96%, and the total polluted days decrease by 21.05% from 2015 to 2021, showing an obvious promotion in air quality; and (3) the changing air quality is attributed to decreasing concentrations of PM2.5 (34.10%), PM10 (25.03%), and NO2 (5.53%) from 2015 to 2021, whereas an increasing concentration of O3 (10.49%) is observed. The processing method for missing data, intact AQI datasets, and analysis of changes are beneficial to policy-making for environmental improvement and fill the gap in the field of data interpolation for air quality datasets in mountainous areas.

Clustering for Topological Interference Management

To reduce the overhead and complexity of channel state information acquisition in interference alignment, the topological interference management (TIM) was proposed to manage interference, which only relied on the network topology information. The previous research on topological interference management via the low-rank matrix completion approach is known to be NP-hard. This paper considers the clustering method for the topological interference management problem, namely, the low-rank matrix completion for TIM is applied within each cluster. Based on the clustering result, we solve the low-rank matrix completion problem via nuclear norm minimization and Frobenius norm minimization function. Simulation results demonstrate that the proposed clustering method combined with TIM leads to significant gain on the achievable degrees of freedom.

Large-Scale Affine Matrix Rank Minimization With a Novel Nonconvex Regularizer.

Low-rank minimization aims to recover a matrix of minimum rank subject to linear system constraint. It can be found in various data analysis and machine learning areas, such as recommender systems, video denoising, and signal processing. Nuclear norm minimization is a dominating approach to handle it. However, such a method ignores the difference among singular values of target matrix. To address this issue, nonconvex low-rank regularizers have been widely used. Unfortunately, existing methods suffer from different drawbacks, such as inefficiency and inaccuracy. To alleviate such problems, this article proposes a flexible model with a novel nonconvex regularizer. Such a model not only promotes low rankness but also can be solved much faster and more accurate. With it, the original low-rank problem can be equivalently transformed into the resulting optimization problem under the rank restricted isometry property (rank-RIP) condition. Subsequently, Nesterov's rule and inexact proximal strategies are adopted to achieve a novel algorithm highly efficient in solving this problem at a convergence rate of O(1/K) , with K being the iterate count. Besides, the asymptotic convergence rate is also analyzed rigorously by adopting the Kurdyka- ojasiewicz (KL) inequality. Furthermore, we apply the proposed optimization model to typical low-rank problems, including matrix completion, robust principal component analysis (RPCA), and tensor completion. Exhaustively empirical studies regarding data analysis tasks, i.e., synthetic data analysis, image recovery, personalized recommendation, and background subtraction, indicate that the proposed model outperforms state-of-the-art models in both accuracy and efficiency.

Predict potential miRNA-disease associations based on bounded nuclear norm regularization

Increasing evidences show that the abnormal microRNA (miRNA) expression is related to a variety of complex human diseases. However, the current biological experiments to determine miRNA-disease associations are time consuming and expensive. Therefore, computational models to predict potential miRNA-disease associations are in urgent need. Though many miRNA-disease association prediction methods have been proposed, there is still a room to improve the prediction accuracy. In this paper, we propose a matrix completion model with bounded nuclear norm regularization to predict potential miRNA-disease associations, which is called BNNRMDA. BNNRMDA at first constructs a heterogeneous miRNA-disease network integrating the information of miRNA self-similarity, disease self-similarity, and the known miRNA-disease associations, which is represented by an adjacent matrix. Then, it models the miRNA-disease prediction as a relaxed matrix completion with error tolerance, value boundary and nuclear norm minimization. Finally it implements the alternating direction method to solve the matrix completion problem. BNNRMDA makes full use of available information of miRNAs and diseases, and can deals with the data containing noise. Compared with four state-of-the-art methods, the experimental results show BNNRMDA achieved the best performance in five-fold cross-validation and leave-one-out cross-validation. The case studies on two complex human diseases showed that 47 of the top 50 prediction results of BNNRMDA have been verified in the latest HMDD database.

Joint denoising of diffusion-weighted images via structured low-rank patch matrix approximation.

To develop a joint denoising method that effectively exploits natural information redundancy in MR DWIs via low-rank patch matrix approximation. A denoising method is introduced to jointly reduce noise in DWI dataset by exploiting nonlocal self-similarity as well as local anatomical/structural similarity within multiple 2D DWIs acquired with the same anatomical geometry but different diffusion directions. Specifically, for each small 3D reference patch sliding within 2D DWI, nonlocal but similar patches are searched by matching image contents within entire DWI dataset and then structured into a patch matrix. The resulting patch matrices are denoised by enforcing low-rankness via weighted nuclear norm minimization and finally are back-distributed to DWI space. The proposed procedure was evaluated with simulated and in vivo brain diffusion tensor imaging (DTI) datasets and then compared to existing Marchenko-Pastur principal component analysis denoising method. The proposed method achieved significant noise reduction while preserving structural details in all DWIs for both simulated and in vivo datasets. Quantitative evaluation of error maps demonstrated it consistently outperformed Marchenko-Pastur principal component analysis method. Further, the denoised DWIs led to substantially improved DTI parametric maps, exhibiting significantly less noise and revealing more microstructural details. The proposed method denoises DWI dataset by utilizing both nonlocal self-similarity and local structural similarity within DWI dataset. This weighted nuclear norm minimization-based low-rank patch matrix denoising approach is effective and highly applicable to various diffusion MRI applications, including DTI as a postprocessing procedure.

Low Tucker rank tensor completion using a symmetric block coordinate descent method

AbstractLow Tucker rank tensor completion has wide applications in science and engineering. Many existing approaches dealt with the Tucker rank by unfolding matrix rank. However, unfolding a tensor to a matrix would destroy the data's original multi‐way structure, resulting in vital information loss and degraded performance. In this article, we establish a relationship between the Tucker ranks and the ranks of the factor matrices in Tucker decomposition. Then, we reformulate the low Tucker rank tensor completion problem as a multilinear low rank matrix completion problem. For the reformulated problem, a symmetric block coordinate descent method is customized. For each matrix rank minimization subproblem, the classical truncated nuclear norm minimization is adopted. Furthermore, temporal characteristics in image and video data are introduced to such a model, which benefits the performance of the method. Numerical simulations illustrate the efficiency of our proposed models and methods.

Infrared Target-Background Separation Based on Weighted Nuclear Norm Minimization and Robust Principal Component Analysis

The target detection ability of an infrared small target detection (ISTD) system is advantageous in many applications. The highly varied nature of the background image and small target characteristics make the detection process extremely difficult. To address this issue, this study proposes an infrared patch model system using non-convex (IPNCWNNM) weighted nuclear norm minimization (WNNM) and robust principal component analysis (RPCA). As observed in the most advanced methods of infrared patch images (IPI), the edges, sometimes in a crowded background, can be detected as targets due to the extreme shrinking of singular values (SV). Therefore, a non-convex WNNM and RPCA have been utilized in this paper, where varying weights are assigned to the SV rather than the same weights for all SV in the existing nuclear norm minimization (NNM) of IPI-based methods. The alternate direction method of multiplier (ADMM) is also employed in the mathematical evaluation of the proposed work. The observed evaluations demonstrated that in terms of background suppression and target detection proficiency, the suggested technique performed better than the cited baseline methods.

A novel initialization method of fixed point continuation for recommendation systems

In recent years, the problem of matrix completion based on rank minimization has received widespread attention in machine learning. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Fixed point continuation (FPC), as a representative nuclear norm relaxation matrix completion algorithm, has been proven to perform well in theories and experiments. However, the traditional FPC algorithm initializes the matrix to be completed by zero, which does not make full use of the shrinkage characteristics of the singular value shrinkage operator on the matrix elements and the known field data information, then will lead to slow convergence and poor accuracy. Aiming at this problem, this paper analyzes the shrinkage properties of matrix elements in the iterative process of the FPC algorithm. Combined with the known rating information in the recommendation systems, a new initialization method of overestimation based on FPC is proposed, and it is applied to the rating prediction in the recommendation systems. The experimental results show that the initialization method proposed in this paper greatly improves the algorithm efficiency and prediction accuracy.

Selecting Regularization Parameters for Nuclear Norm--Type Minimization Problems

The reconstruction of a low-rank matrix from its noisy observation finds its usage in many applications. It can be reformulated into a constrained nuclear norm minimization problem, where the bound $\eta$ of the constraint is explicitly given or can be estimated by the probability distribution of the noise. When the Lagrangian method is applied to find the minimizer, the solution can be obtained by the singular value thresholding operator, where the thresholding parameter $\lambda$ is related to the Lagrangian multiplier. In this paper, we first show that the Frobenius norm of the discrepancy between the minimizer and the observed matrix is a strictly monotonically increasing function of $\lambda$. From that we derive a closed form solution for $\lambda$ in terms of $\eta$. Since $\lambda$ is the same as the regularization parameter for the unconstrained regularized problem, our results can be applied to automatically choosing a suitable regularization parameter for the nuclear norm--type regularized minimization problems using the discrepancy principle. The regularization parameters obtained are comparable to (and sometimes better than) those obtained by Stein's unbiased risk estimator approach, while the cost of solving the minimization problem can be reduced by 11--18 times. Numerical experiments with both synthetic data and real MRI data are performed to validate the proposed approach.