Nuclear Norm Minimization Problem Research Articles

HPETC: History Priority Enhanced Tensor Completion for Network Distance Measurement

In network distance measurement, how to estimate the whole network distance data from partially observed samples has attracted lots of attention because of its significance for network performance evaluation. Matrix completion becomes the most effective approach. However, the two-dimension matrix can only capture the spatial features in the network distance data while ignoring the temporal features. To conquer the problem, few recent studies begin to model the network distance data as a three-dimension tensor and propose tensor completion approaches for distance estimation. Although promising, existing tensor completion approaches still suffer the problem of low recovery accuracy and high measurement cost because they ignore the history priority information. To fully utilize both spatial and temporal features hidden in the distance data, this paper formulates a novel History Priority Enhanced Tensor Completion (HPETC) for distance estimation as a weighted tensor nuclear norm minimization problem where the weight is defined based on the history subspaces information. To solve the weighted tensor nuclear norm minimization problem, we firstly transform it into a factorization-based Frobenius norm minimization problem to avoid costly T-SVD computations, and then propose an iterative algorithm to solve the transformed problem. We further derive a theoretical sampling bound that is lower than the existing sampling bound, thus leads a lower measurement cost. We demonstrate the effectiveness of the proposed algorithm by conducting extensive experiments using two real network distance datasets. The result shows that the proposed algorithm can not only improve the estimation accuracy but also reduce the sampling complexity compared to the state-of-the-art approaches.

Semi-Proximal ADMM for Primal and Dual Robust Low-Rank Matrix Restoration from Corrupted Observations

The matrix nuclear norm minimization problem has been extensively researched in recent years due to its widespread applications in control design, signal and image restoration, machine learning, big data problems, and more. One popular model is nuclear norm minimization with the l2-norm fidelity term, but it is only effective for those problems with Gaussian noise. A nuclear norm minimization problem with the l1-norm fidelity term has been studied in this paper, which can deal with the problems with not only non-Gaussian noise but also Gaussian noise or their mixture. Moreover, it also keeps the efficiency for the noiseless case. Given the nonsmooth proposed model, we transform it into a separated form by introducing an auxiliary variable and solve it by the semi-proximal alternating direction method of multipliers (sPADMM). Furthermore, we first attempt to solve its dual problem by sPADMM. Then, the convergence guarantees for the aforementioned algorithms are given. Finally, some numerical studies are dedicated to show the robustness of the proposed model and the effectiveness of the presented algorithms.

A lagrange programming neural network approach for nuclear norm optimization.

This article proposes a continuous-time optimization approch instead of tranditional optimiztion methods to address the nuclear norm minimization (NNM) problem. Refomulating the NNM into a matrix form, we propose a Lagrangian programming neural network (LPNN) to solve the NNM. Moreover, the convergence condtions of LPNN are presented by the Lyapunov method. Convergence experiments are presented to demonstrate the convergence of LPNN. Compared with tranditional algorithms of NNM, the proposed algorithm outperforms in terms of image recovery.

Structural-Missing Tensor Completion for Robust DOA Estimation with Sensor Failure

Array sensor failure poses a serious challenge to robust direction-of-arrival (DOA) estimation in complicated environments. Although existing matrix completion methods can successfully recover the damaged signals of an impaired sensor array, they cannot preserve the multi-way signal characteristics as the dimension of arrays expands. In this paper, we propose a structural-missing tensor completion algorithm for robust DOA estimation with uniform rectangular array (URA), which exhibits a high robustness to non-ideal sensor failure conditions. Specifically, the signals received at the impaired URA are represented as a three-dimensional incomplete tensor, which contains whole fibers or slices of missing elements. Due to this structural-missing pattern, the conventional low-rank tensor completion becomes ineffective. To resolve this issue, a spatio-temporal dimension augmentation method is developed to transform the structural-missing tensor signal into a six-dimensional Hankel tensor with dispersed missing elements. The augmented Hankel tensor can then be completed with a low-rank regularization by solving a Hankel tensor nuclear norm minimization problem. As such, the inverse Hankelization on the completed Hankel tensor recovers the tensor signal of an unimpaired URA. Accordingly, a completed covariance tensor can be derived and decomposed for robust DOA estimation. Simulation results verify the effectiveness of the proposed algorithm.

Low-rank tensor recovery for 2-D DOA estimation using cylindrical nested conformal array

In recent years, nested conformal arrays have received considerable attention because they are capable of providing extended array aperture and increased degrees of freedom compared to uniform conformal arrays. In this article, we suggest a low-rank tensor recovery algorithm for two-dimensional (2-D) direction-of-arrival (DOA) estimation with cylindrical nested conformal array (CNCA). First of all, we derive the tensor form coarray output of the CNCA and use the coarray output to construct a third-order augmented tensor. Subsequently, we reconstruct a noise-free third-order augmented tensor by formulating a tensor nuclear norm (TNN) minimization problem. With the reconstructed tensor, we finally derive the closed-form expressions of 2-D DOA estimates. Compared with the existing approaches, the proposed approach achieves better performance by exploiting the multidimensional structure inherent in the coarray output signal. Numerical results illustrate the advantage of the proposed method over several existing techniques.

Non-uniformly sampled 2D NMR Spectroscopy reconstruction based on Low Rank Hankel Matrix Fast Tri-Factorization and Non-convex Factorization

Nuclear magnetic resonance (NMR) spectroscopy is widely used in chemistry and medicine to research the composition of matter and the spatial structure of proteins. However, NMR signals with larger sizes and higher dimensions are slower to acquire. Therefore, Non-Uniform Sampling (NUS) methods are used to speed up the acquisition of NMR signals, and some mathematical algorithms are used to recover the original NMR signals from the NUS data. In recent years, low-rank Hankel matrix completion is proved to recover NMR data with less error. However, it requires singular value decomposition (SVD) with high computational complexity in each iteration, which is not suitable for reconstructing signals with large sizes. In this paper, we propose a Fast Tri-Factorization (FTF) method to decompose the low-rank Hankel matrix into three small-scale matrices, and it convert the original problem into a matrix nuclear norm minimization problem on the small-scale matrix to mitigate high computation cost of multiple SVDs. Furthermore, the factor matrix size remains constant for different size of the Hankel matrices. We further replace the low-rank constraint with Non-convex Factorization (NF) to avoid the time-consuming SVD in each iteration. Numerical experiments show that the proposed method can significantly speed up the NMR reconstruction compared with several existing algorithms while ensuring the reconstruction accuracy.

Coarray Tensor Completion for DOA Estimation

Sparse array direction-of-arrival (DOA) estimation using tensor model has been developed to handle multi-dimensional sub-Nyquist sampled signals. Furthermore, augmented virtual arrays can be derived for Nyquist-matched coarray tensor processing. However, the partially augmentable sparse array corresponds to a discontinuous virtual array, whereas the existing methods can only utilize its continuous part. Conventional virtual linear array interpolation techniques complete coarray covariance matrices with dispersed missing elements, but fail to complete the coarray tensor with whole missing slices. In this paper, we propose a coarray tensor completion algorithm for two-dimensional DOA estimation, where the coarray tensor statistics can be entirely exploited. In particular, in order to impose an effective low-rank regularization on the slice-missing coarray tensor, we propose shift dimensional augmenting and coarray tensor reshaping approaches to reformulate a structured coarray tensor with sufficiently dispersed missing elements. Furthermore, the shape of the reformulated coarray tensor is optimized by maximizing the dispersion-to-percentage ratio of missing elements. As such, a coarray tensor nuclear norm minimization problem can be designed to optimize the completed coarray tensor corresponding to a filled virtual array, based on which the closed-form DOA estimation is achieved. Meanwhile, the global convergence of the coarray tensor completion is theoretically proved. Simulation results demonstrate the effectiveness of the proposed algorithm compared to other matrix-based and tensor-based methods.

DOA Estimation of Mixed Circular and Noncircular Sources Using Nonuniform Linear Array

In this article, an algorithm for accurately estimating the directions-of-arrival (DOAs) of mixed circular and noncircular signals with a nonuniform linear array is devised. Via applying coarray interpolation on the difference and sum coarrays, we first formulate two nuclear norm minimization problems to reconstruct an augmented covariance matrix and an augmented unconjugated covariance matrix, respectively. Based on the reconstructed matrices, we further build the extended covariance matrix in the coarray domain. Finally, we construct several spectral functions to successively estimate the DOAs of the nonstrictly noncircular, strictly noncircular, and circular signals. Unlike the existing schemes, the proposed algorithm can identify accurately circular, strictly noncircular, and nonstrictly noncircular signals. Numerical results demonstrate the superiority of the proposed algorithm over existing techniques.

Selecting Regularization Parameters for Nuclear Norm--Type Minimization Problems

The reconstruction of a low-rank matrix from its noisy observation finds its usage in many applications. It can be reformulated into a constrained nuclear norm minimization problem, where the bound $\eta$ of the constraint is explicitly given or can be estimated by the probability distribution of the noise. When the Lagrangian method is applied to find the minimizer, the solution can be obtained by the singular value thresholding operator, where the thresholding parameter $\lambda$ is related to the Lagrangian multiplier. In this paper, we first show that the Frobenius norm of the discrepancy between the minimizer and the observed matrix is a strictly monotonically increasing function of $\lambda$. From that we derive a closed form solution for $\lambda$ in terms of $\eta$. Since $\lambda$ is the same as the regularization parameter for the unconstrained regularized problem, our results can be applied to automatically choosing a suitable regularization parameter for the nuclear norm--type regularized minimization problems using the discrepancy principle. The regularization parameters obtained are comparable to (and sometimes better than) those obtained by Stein's unbiased risk estimator approach, while the cost of solving the minimization problem can be reduced by 11--18 times. Numerical experiments with both synthetic data and real MRI data are performed to validate the proposed approach.

Weighted truncated nuclear norm regularization for low-rank quaternion matrix completion

In recent years, quaternion matrix completion (QMC) based on low-rank regularization has been gradually used in image processing. Unlike low-rank matrix completion (LRMC) which handles RGB images by recovering each color channel separately, QMC models retain the connection of three channels and process them as a whole. Most of the existing quaternion-based methods formulate low-rank QMC (LRQMC) as a quaternion nuclear norm (a convex relaxation of the rank) minimization problem. The main limitation of these approaches is that they minimize the singular values simultaneously such that cannot approximate low-rank attributes efficiently. To achieve a more accurate low-rank approximation, we introduce a quaternion truncated nuclear norm (QTNN) for LRQMC and utilize the alternating direction method of multipliers (ADMM) to get the optimization in this paper. Further, we propose weights to the residual error quaternion matrix during the update process for accelerating the convergence of the QTNN method with admissible performance. The weighted method utilizes a concise gradient descent strategy which has a theoretical guarantee in optimization. The effectiveness of our method is illustrated by experiments on real visual data sets.

Medical Image Denoising Via Matrix Norm Minimization Problems

This paper presents the matrix completion problem for image denoising. Three problems based on matrix norm are performing: Spectral norm minimization problem (SNP), Nuclear norm minimization problem (NNP), and Weighted nuclear norm minimization problem (WNNP). In general, images representing by a matrix this matrix contains the information of the image, some information is irrelevant or unfavorable, so to overcome this unwanted information in the image matrix, information completion is used to comperes the matrix and remove this unwanted information. The unwanted information is handled by defining {0,1}-operator under some threshold. Applying this operator on a given matrix keeps the important information in the image and removing the unwanted information by solving the matrix completion problem that is defined by P. The quadratic programming use to solve the given three norm-based minimization problems. To improve the optimal solution a weighted exponential is used to compute the weighted vector of spectral that use to improve the threshold of optimal low rank that getting from solving the nuclear norm and spectral norm problems. The result of applying the proposed method on different types of images is given by adopting some metrics. The results showed the ability of the given methods.

Coarray Interpolation for DOA Estimation Using Coprime EMVS Array

In this letter, we develop a coarray interpolation method for direction-of-arrival (DOA) estimation using the coprime electromagnetic vector-sensor (EMVS) array. Firstly, we derive the coarray signal model of the coprime EMVS array, which can be viewed as a combination of multiple polarization components. Subsequently, we fill in zero elements in each polarization component and recover the corresponding low-rank covariance matrix by solving a nuclear norm minimization (NNM) problem. By exploiting the recovered covariance matrices, we eventually construct a larger covariance matrix to carry out DOA estimation. Numerical experiment results demonstrate the superiority of the proposed algorithm over conventional methods.

Labeled-Robust Regression: Simultaneous Data Recovery and Classification.

Rank minimization is widely used to extract low-dimensional subspaces. As a convex relaxation of the rank minimization, the problem of nuclear norm minimization has been attracting widespread attention. However, the standard nuclear norm minimization usually results in overcompression of data in all subspaces and eliminates the discrimination information between different categories of data. To overcome these drawbacks, in this article, we introduce the label information into the nuclear norm minimization problem and propose a labeled-robust principal component analysis (L-RPCA) to realize nuclear norm minimization on multisubspace data. Compared with the standard nuclear norm minimization, our method can effectively utilize the discriminant information in multisubspace rank minimization and avoid excessive elimination of local information and multisubspace characteristics of the data. Then, an effective labeled-robust regression (L-RR) method is proposed to simultaneously recover the data and labels of the observed data. Experiments on real datasets show that our proposed methods are superior to other state-of-the-art methods.

Recovery analysis of damped spectrally sparse signals and its relation to MUSIC

Abstract One of the classical approaches for estimating the frequencies and damping factors in a spectrally sparse signal is the MUltiple SIgnal Classification (MUSIC) algorithm, which exploits the low-rank structure of an autocorrelation matrix. Low-rank matrices have also received considerable attention recently in the context of optimization algorithms with partial observations, and nuclear norm minimization (NNM) has been widely used as a popular heuristic of rank minimization for low-rank matrix recovery problems. On the other hand, it has been shown that NNM can be viewed as a special case of atomic norm minimization (ANM), which has achieved great success in solving line spectrum estimation problems. However, as far as we know, the general ANM (not NNM) considered in many existing works can only handle frequency estimation in undamped sinusoids. In this work, we aim to fill this gap and deal with damped spectrally sparse signal recovery problems. In particular, inspired by the dual analysis used in ANM, we offer a novel optimization-based perspective on the classical MUSIC algorithm and propose an algorithm for spectral estimation that involves searching for the peaks of the dual polynomial corresponding to a certain NNM problem, and we show that this algorithm is in fact equivalent to MUSIC itself. Building on this connection, we also extend the classical MUSIC algorithm to the missing data case. We provide exact recovery guarantees for our proposed algorithms and quantify how the sample complexity depends on the true spectral parameters. In particular, we provide a parameter-specific recovery bound for low-rank matrix recovery of jointly sparse signals rather than use certain incoherence properties as in existing literature. Simulation results also indicate that the proposed algorithms significantly outperform some relevant existing methods (e.g., ANM) in frequency estimation of damped exponentials.

A Robust Pre-processing Approach of Array Samples with Hankel Matrix Completion

In this paper, we study the problem of recovering array samples when a few array samples are unavailable, and provide a robust pre-processing approach of recovering unavailable array samples with Hankel matrix completion. First, Alternating Direction Methods (ADM) for solving nuclear norm minimization problem with Hankel structure was discussed. Second, stability of Hankel matrix reconstruction is investigated in view of l*-Constrained Minimal Singular Value (l*-CMSV). Finally, simulation experiments illustrate that the proposed method's are robust for complex Gaussian noise.

Sparse and low-rank regularized deep subspace clustering

Subspace clustering aims at discovering the intrinsic structure of data in unsupervised fashion. As ever in most of approaches, an affinity matrix is constructed by learning from original data or the corresponding hand-crafted feature with some constraints on the self-expressive matrix (SEM), which is then followed by spectral clustering algorithm. Based on successful applications of deep technologies, it has become popular to simultaneously accomplish deep feature and self-representation learning for subspace clustering. However, deep feature and SEM in previous deep methods are lack of precise constraints, which is sub-optimal to conform with the linear subspace model. To address this, we propose an approach, namely sparse and low-rank regularized deep subspace clustering (SLR-DSC). In the proposed SLR-DSC, an end-to-end framework is proposed by introducing sparse and low-rank constraints on deep feature and SEM respectively. The sparse deep feature and low-rank regularized SEM implemented via fully-connected layers are encouraged to facilitate a more informative affinity matrix. In order to solve the nuclear norm minimization problem, a sub-gradient computation strategy is utilized to cater to the chain rule. Experiments on the data sets demonstrate that our method significantly outperforms the competitive unsupervised subspace clustering approaches.

Block-sparse recovery of semidefinite systems and generalized null space conditions

This article considers the recovery of low-rank matrices via a convex nuclear-norm minimization problem and presents two null space properties (NSP) which characterize uniform recovery for the case of block-diagonal matrices and block-diagonal positive semidefinite matrices. These null-space conditions turn out to be special cases of a new general setup, which allows to derive the mentioned NSPs and well-known NSPs from the literature. We discuss the relative strength of these conditions and also present a deterministic class of matrices that satisfies the block-diagonal semidefinite NSP.

Weighted Double-Low-Rank Decomposition With Application to Fabric Defect Detection

Recently, many methods based on low-rank representation have been proposed for fabric defect detection. Most of them relax the low-rank decomposition problem to a nuclear norm minimization (NNM) problem to pursue the convexity of the objective function. When solving the standard NNM problem, matrix singular values have to be treated equally. This, however, would be impractical in the scenario of fabric defect detection as the matrix singular values have clear physical meanings, and thus, they should be treated differently. In this article, we propose a weighted double-low-rank decomposition method (WDLRD) to treat the matrix singular values differently by assigning different weights. Thus, the most important/distinguishing characteristics of a fabric image can be preserved. Another difference between WDLRD and the other existing low-rank-based methods is that WDLRD considers a defective fabric image being decomposed to two low-rank matrices, i.e., low-rank defect-free matrix and low-rank defect matrix, as the defect-free and defective regions are usually composed of homogeneous objects that have a high correlation. Besides, WDLRD is more robust for defect detection in various situations by adding a noise term to avoid noise or other interference on the fabric surface. In addition, a defect prior is incorporated into the objective function of WDLRD to guide locating the defective regions. The proposed optimization problem can be easily solved by an iterative algorithm based on augmented Lagrange multipliers. Experimental results on TILDA, periodically patterned fabric, and Textile & Apparel Artificial Intelligence databases show that the proposed WDLRD obtains better performance than state-of-the-art methods in locating the defective regions on fabric images. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article is motivated by the problem that the performance of fabric defect detection in the textile industry is poor. It is necessary to develop an effective method to improve the defect detection accuracy and reduce overall manufacturing cost. Existing automatic defect detection approaches usually contain two stages: first, capture fabric images from the weaving machine and then use a defect detection algorithm in a host computer to conduct a real-time inspection and give an alarm if defects occur. This article focuses on locating defects for given defective images after the procedure of binary classification (which determines an image as defective or defect free). The article proposes an objective function to mathematically interpret the optimization problem between fabric images and predictive defects. The optimal solution can be obtained by employing the alternating direction method of multipliers (ADMMs). The proposed method is described as a new defect detection algorithm. Extensive experiments were conducted to evaluate the algorithm, and the experimental results indicate that the proposed method is superior to many existing fabric defect detection methods. Preliminary experiments suggest that this method is feasible but has not yet been really used in production. In future research, we will collect more fabric images from the textile industry and develop large-scale databases for verifying the proposed method for real-life applications.

Singular Value Thresholding Algorithm for Wireless Sensor Network Localization

Wireless Sensor Networks (WSN) are of great current interest in the proliferation of technologies. Since the location of the sensors is one of the most interesting issues in WSN, the process of node localization is crucial for any WSN-based applications. Subsequently, WSN’s node estimation deals with a low-rank matrix which gives rise to the application of the Nuclear Norm Minimization (NNM) method. This paper will focus on the localization of 2-dimensional WSN with objects (obstacles). Recent studies introduce Nuclear Norm Minimization (NNM) for node estimation instead of formulating the rank minimization problem. Common way to tackle this problem is by implementing the Semidefinite Programming (SDP). However, SDP can only handle matrices with a size of less than 100 × 100. Therefore, we introduce the method of Singular Value Thresholding (SVT) which is an iterative algorithm to solve the NNM problem that produces a sequence of matrices { X k , Y k } and executes a soft-thresholding operation on the singular value of the matrix Y k . This algorithm is a user-friendly algorithm which produces a low computational cost with low storage capacity required to give the lowest-rank minimum nuclear norm solution.

Tensorized Multi-view Subspace Representation Learning

Self-representation based subspace learning has shown its effectiveness in many applications. In this paper, we promote the traditional subspace representation learning by simultaneously taking advantages of multiple views and prior constraint. Accordingly, we establish a novel algorithm termed as Tensorized Multi-view Subspace Representation Learning. To exploit different views, the subspace representation matrices of different views are regarded as a low-rank tensor, which effectively models the high-order correlations of multi-view data. To incorporate prior information, a constraint matrix is devised to guide the subspace representation learning within a unified framework. The subspace representation tensor equipped with a low-rank constraint models elegantly the complementary information among different views, reduces redundancy of subspace representations, and then improves the accuracy of subsequent tasks. We formulate the model with a tensor nuclear norm minimization problem constrained with $$\ell _{2,1}$$ -norm and linear equalities. The minimization problem is efficiently solved by using an Augmented Lagrangian Alternating Direction Minimization method. Extensive experimental results on diverse multi-view datasets demonstrate the effectiveness of our algorithm.