Nuclear Norm Research Articles

Low-Rank Hankel Tensor Completion for Traffic Speed Estimation

This paper studies the traffic state estimation (TSE) problem using sparse observations from mobile sensors. Most existing TSE methods either rely on well-defined physical traffic flow models or require large amounts of simulation data as input to train learning algorithms. Different from previous studies, in this paper we propose a purely data-driven and model-free solution. We consider TSE as a spatiotemporal matrix completion/interpolation problem and apply spatiotemporal delay embedding to transform the original incomplete matrix into a fourth-order Hankel structured tensor. By imposing a low-rank assumption on this tensor structure, we can approximate and characterize both global patterns and local patterns in a data-driven manner. We use a truncated nuclear norm of a balanced spatiotemporal unfolding to approximate the tensor rank and develop an efficient algorithm based on the Alternating Direction Method of Multipliers (ADMM) to solve the problem. The proposed framework only involves two hyperparameters, spatial and temporal window lengths, which are easy to set given the degree of data sparsity. To validate the effectiveness of our proposed method, we conducted numerical experiments on real-world high-resolution trajectory data, which demonstrated its superiority in some challenging scenarios. The proposed method shows great potential for solving the TSE problem using sparse observations from mobile sensors and can be applied in various traffic applications.

The Proxy Step-Size Technique for Regularized Optimization on the Sphere Manifold.

We give an effective solution to the regularized optimization problem g (x) + h (x), where x is constrained on the unit sphere ||x ||2 = 1. Here g (·) is a smooth cost with Lipschitz continuous gradient within the unit ball {x : ||x ||2 ≤ 1 } whereas h (·) is typically non-smooth but convex and absolutely homogeneous, e.g., norm regularizers and their combinations. Our solution is based on the Riemannian proximal gradient, using an idea we call proxy step-size - a scalar variable which we prove is monotone with respect to the actual step-size within an interval. The proxy step-size exists ubiquitously for convex and absolutely homogeneous h(·), and decides the actual step-size and the tangent update in closed-form, thus the complete proximal gradient iteration. Based on these insights, we design a Riemannian proximal gradient method using the proxy step-size. We prove that our method converges to a critical point, guided by a line-search technique based on the g(·) cost only. The proposed method can be implemented in a couple of lines of code. We show its usefulness by applying nuclear norm, l1 norm, and nuclear-spectral norm regularization to three classical computer vision problems. The improvements are consistent and backed by numerical experiments. available at https://bitbucket.org/FangBai/proxystepsize-pgs.

Approximately low-rank recovery from noisy and local measurements by convex program

Abstract Low-rank matrix models have been universally useful for numerous applications, from classical system identification to more modern matrix completion in signal processing and statistics. The nuclear norm has been employed as a convex surrogate of the low-rankness since it induces a low-rank solution to inverse problems. While the nuclear norm for low rankness has an excellent analogy with the $\ell _1$ norm for sparsity through the singular value decomposition, other matrix norms also induce low-rankness. Particularly as one interprets a matrix as a linear operator between Banach spaces, various tensor product norms generalize the role of the nuclear norm. We provide a tensor-norm-constrained estimator for the recovery of approximately low-rank matrices from local measurements corrupted with noise. A tensor-norm regularizer is designed to adapt to the local structure. We derive statistical analysis of the estimator over matrix completion and decentralized sketching by applying Maurey’s empirical method to tensor products of Banach spaces. The estimator provides a near-optimal error bound in a minimax sense and admits a polynomial-time algorithm for these applications.

Separation-free super-resolution from compressed measurements is possible: an orthonormal atomic norm minimization approach

Abstract We consider the problem of recovering the superposition of $R$ distinct complex exponential functions from compressed non-uniform time-domain samples. Total variation (TV) minimization or atomic norm minimization was proposed in the literature to recover the $R$ frequencies or the missing data. However, it is known that in order for TV minimization and atomic norm minimization to recover the missing data or the frequencies, the underlying $R$ frequencies are required to be well separated, even when the measurements are noiseless. This paper shows that the Hankel matrix recovery approach can super-resolve the $R$ complex exponentials and their frequencies from compressed non-uniform measurements, regardless of how close their frequencies are to each other. We propose a new concept of orthonormal atomic norm minimization (OANM), and demonstrate that the success of Hankel matrix recovery in separation-free super-resolution comes from the fact that the nuclear norm of a Hankel matrix is an orthonormal atomic norm. More specifically, we show that, in traditional atomic norm minimization, the underlying parameter values must be well separated to achieve successful signal recovery, if the atoms are changing continuously with respect to the continuously valued parameter. In contrast, for the OANM, it is possible the OANM is successful even though the original atoms can be arbitrarily close. As a byproduct of this research, we provide one matrix-theoretic inequality of nuclear norm, and give its proof using the theory of compressed sensing.

Tensor completion via convolutional sparse coding with small samples-based training

Tensor data often suffer from missing value problems due to the complex high-dimensional structure while acquiring them. To complete the missing information, lots of Low-Rank Tensor Completion (LRTC) methods have been proposed, most of which depend on the low-rank property of tensor data. In this way, the low-rank component of the original data could be recovered roughly. However, the shortcoming is that the detailed information can not be fully restored, no matter the Sum of the Nuclear Norm (SNN) nor the Tensor Nuclear Norm (TNN) based methods. On the contrary, in the field of signal processing, Convolutional Sparse Coding (CSC) can provide a good representation of the high-frequency component of the image, which is generally associated with the detail component of the data. To this end, we propose two novel methods, LRTC-CSC-I and LRTC-CSC-II, which adopt CSC as a supplementary regularization for LRTC to capture the high-frequency components. Therefore, the LRTC-CSC methods can not only solve the missing value problem but also recover the details. Moreover, the regularizer CSC can be trained with small samples due to the sparsity characteristic. Extensive experiments show the effectiveness of LRTC-CSC methods, and quantitative evaluation indicates that the performance of our models are superior to state-of-the-art methods.

Autofocusing of Fresnel zone aperture lensless imaging for QR code recognition.

Fresnel zone aperture (FZA) lensless imaging encodes the incident light into a hologram-like pattern, so that the scene image can be numerically focused at a long imaging range by the back propagation method. However, the target distance is uncertain. The inaccurate distance causes blurs and artifacts in the reconstructed images. This brings difficulties for the target recognition applications, such as quick response code scanning. We propose an autofocusing method for FZA lensless imaging. By incorporating the image sharpness metrics into the back propagation reconstruction process, the method can acquire the desired focusing distance and reconstruct noise-free high-contrast images. By combining the Tamura of the gradient metrics and nuclear norm of gradient, the relative error of estimated object distance is only 0.95% in the experiment. The proposed reconstruction method significantly improves the mean recognition rate of QR code from 4.06% to 90.00%. It paves the way for designing intelligent integrated sensors.

Three dimensional seismic data reconstruction based on truncated nuclear norm

The rank reduction method is widely used to reconstruct three dimensional (3D) seismic data. The traditional multichannel singular spectrum analysis (MSSA) utilizes truncated singular value decomposition (TSVD) to approximately solve the rank function of the block Hankel structure matrix constructed by frequency slices of seismic data. However, the TSVD algorithm discards all nonzero singular values except a few largest singular values and ignores the useful seismic information in small nonzero singular values. To further utilize the seismic data information contained in these small singular values, this paper proposes a method to recover 3D seismic data with truncated nuclear norm (TNN). The proposed method imposes different constraints on large singular values and small singular values. Essentially, the TNN is closer to the rank function than the TSVD. Finally, the proposed model is addressed by the alternating direction method of multipliers, and closed-form solutions are used to update the optimization variables. The experimental results demonstrate that the proposed method achieves superior reconstruction results than the traditional MSSA method.

NUCLEAR NORM REGULARIZED QUANTILE REGRESSION WITH INTERACTIVE FIXED EFFECTS

This paper studies large N and large T conditional quantile panel data models with interactive fixed effects. We propose a nuclear norm penalized estimator of the coefficients on the covariates and the low-rank matrix formed by the interactive fixed effects. The estimator solves a convex minimization problem, not requiring pre-estimation of the (number of) interactive fixed effects. It also allows the number of covariates to grow slowly with N and T. We derive an error bound on the estimator that holds uniformly in the quantile level. The order of the bound implies uniform consistency of the estimator and is nearly optimal for the low-rank component. Given the error bound, we also propose a consistent estimator of the number of interactive fixed effects at any quantile level. We demonstrate the performance of the estimator via Monte Carlo simulations.

A new nonconvex low-rank tensor approximation method with applications to hyperspectral images denoising

Hyperspectral images (HSIs) are frequently corrupted by mixing noise during their acquisition and transmission. Such complicated noise may reduce the quality of the obtained HSIs and limit the accuracy of the subsequent processing. By using the low-rank prior of the tensor formed by spatial and spectral information and further exploring the intrinsic structure of the underlying HSI from noisy observations, in this paper, we propose a new nonconvex low-rank tensor approximation method including optimization model and efficient iterative algorithm to eliminate multiple types of noise. The proposed mathematical model consists of a nonconvex low-rank regularization term using the γ nuclear norm, which is nonconvex surrogate to Tucker rank, and two data fidelity terms representing sparse and Gaussian noise components, which are regularized by the -norm and the Frobenius norm, respectively. To solve this model, we propose an efficient augmented Lagrange multiplier algorithm. We also study the convergence and parameter setting of the algorithm. Extensive experimental results show that the proposed method has better denoising performance than the state-of-the-art competing methods for low-rank tensor approximation and noise modeling.

Image inpainting using non-convex low rank decomposition and multidirectional search

Low-rank (LR) and nonlocal self-similarity (NSS) are two important priors for image inpainting as a typical inverse problem. Nuclear norm minimization (NNM) is a widely used convex relaxation for relevant rank minimization problems. However, NNM regularizes each singular value equally and ignores the significance of bigger singular values. In this paper, we propose a non-convex low-rank decomposition (NC-LRD) model that is based on robust principal component analysis (RPCA) with a weighted L1 norm. Utilizing NSS prior for image inpainting we search similar patches by using a newly designed multidirectional search (MS) method, and apply the NC-LRD model to complete each corrupted patch matrix (low-rank decomposition with multidirectional search, MS-LRD). We focus on the spatial distribution of similar patches by restricting matched N patches to locate at N different directions relative to a target patch, while previous state-of-the-art methods do not consider the spatial distribution in similarity criterion. The MS method solves the problem that many patch-based inpainting algorithms fail to complete missing lines. Experimental results on line missing demonstrate that the proposed NC-LRD method has lower reconstruction error in matrix completion, and it converges faster than several state-of-the-art matrix completion algorithms. At the same time, the effectiveness and superiority of MS-LRD over other competitive inpainting algorithms are also verified.

TR-ReFloc: A TR-based Framework for Recovering Missed RSS for WiFi indoor positioning in the offline and online phase

The research on WiFi-based fingerprint positioning has become an important topic in the field of indoor positioning due to its low-cost deployment and widespread applications. However, received signal strength (RSS) from access points (APs) may be missed due to factors such as Green WiFi or AP malfunctions. Traditional approach often set the missed RSS to a predetermined default value, which may not be accurate. We propose a fingerprint recovery framework based on tensor-ring nuclear norm minimization (TRNNM) algorithm which uses the correlation of RSS to recover missed fingerprints in both offline and online phases, called TR-ReFloc. In the offline phase, the radio map in tensor form is recovered by tensor ring (TR) decomposition. In the online phase, depending on different user requirements, there are two stages to generate the positioning results. First, rapid results are generated in coarse positioning stage. Next, high-precision results generated in fine positioning stage by recovering the missed fingerprints using the correlation of test points (TPs) are closed to the current TP. Experiments demonstrate that the proposed framework is robust for recovering missed RSS and can maintain localization accuracy within 2 m even in cases of high missing rate, demonstrating significant improvement over traditional methods.

Hyper-Laplacian Regularized Multi-View Clustering with Exclusive L21 Regularization and Tensor Log-Determinant Minimization Approach

Multi-view clustering aims to capture the multiple views inherent information by identifying the data clustering that reflects distinct features of datasets. Since there is a consensus in literature that different views of a dataset share a common latent structure, most existing multi-view subspace learning methods rely on the nuclear norm to seek the low-rank representation of the underlying subspace. However, the nuclear norm often fails to distinguish the variance of features for each cluster due to its convex nature and data tends to fall in multiple non-linear subspaces for multi-dimensional datasets. To address these problems, we propose a new and novel multi-view clustering method (HL-L21-TLD-MSC) that unifies the Hyper-Laplacian (HL) and exclusive ℓ 2,1 (L21) regularization with the Tensor Log-Determinant Rank Minimization (TLD) setting. Specifically, the hyper-Laplacian regularization maintains the local geometrical structure that makes the estimation prune to nonlinearities, and the mixed ℓ 2,1 and ℓ 1,2 regularization provides the joint sparsity within-cluster as well as the exclusive sparsity between-cluster. Furthermore, a log-determinant function is used as a tighter tensor rank approximation to discriminate the dimension of features. An efficient alternating algorithm is then derived to optimize the proposed model, and the construction of a convergent sequence to the Karush-Kuhn-Tucker (KKT) critical point solution is mathematically validated in detail. Extensive experiments are conducted on ten well-known datasets to demonstrate that the proposed approach outperforms the existing state-of-the-art approaches with various scenarios, in which, six of them achieve perfect results under our framework developed in this article, demonstrating highl effectiveness for the proposed approach.

Probability-weighted tensor robust PCA with CP decomposition for hyperspectral image restoration

This paper presents a novel probability-weighted tensor robust principal component analysis (TRPCA) method based on CANDECOMP/PARAFAC decomposition (CPD) for hyperspectral image (HSI) restoration with low computational complexity from the coexistence of dense noise and sparse outliers. In the proposed method, via the CPD property, the tensor nuclear norm (TNN) optimization object is replaced as the CPD factor matrix of HSI with lower dimensions without losing low-rank property, so as to the computational complexity could be reduced during optimizing TNN in TRPCA model. To demonstrate the different effects of noise and outliers on HSI restoration, two weighted sets are defined with probability by the prior knowledge of the occurrence of outliers, which are used to analyze the probability whether the HSI is corrupted by dense noise or outliers. Via the probability theory, information entropy is employed to measure the uncertainty of noise and outlier occurrence in received data, so that to demonstrate their difference impacts on HSI restoration to improve the recovery accuracy. Theoretical analysis and simulation results show that the proposed method has lower computational complexity and better restoration performance compared with other related methods.

Novel Algorithms with Inertial Techniques for Solving Constrained Convex Minimization Problems and Applications to Image Inpainting

In this paper, we propose two novel inertial forward–backward splitting methods for solving the constrained convex minimization of the sum of two convex functions, φ1+φ2, in Hilbert spaces and analyze their convergence behavior under some conditions. For the first method (iFBS), we use the forward–backward operator. The step size of this method depends on a constant of the Lipschitz continuity of ∇φ1, hence a weak convergence result of the proposed method is established under some conditions based on a fixed point method. With the second method (iFBS-L), we modify the step size of the first method, which is independent of the Lipschitz constant of ∇φ1 by using a line search technique introduced by Cruz and Nghia. As an application of these methods, we compare the efficiency of the proposed methods with the inertial three-operator splitting (iTOS) method by using them to solve the constrained image inpainting problem with nuclear norm regularization. Moreover, we apply our methods to solve image restoration problems by using the least absolute shrinkage and selection operator (LASSO) model, and the results are compared with those of the forward–backward splitting method with line search (FBS-L) and the fast iterative shrinkage-thresholding method (FISTA).

A Fast Matrix Completion Method Based on Matrix Bifactorization and QR Decomposition

The problem of recovering the missing values in an incomplete matrix, i.e., matrix completion, has attracted a great deal of interests in the fields of machine learning and signal processing. A matrix bifactorization method, which is abbreviated as MBF, is a fast method of matrix completion that has a better speed than the traditional nuclear norm minimization methods. However, it may become inaccurate and slow when solving matrices of not low rank. In this paper, an improved fast and accurate MBF method based on Qatar Riyal (QR) decomposition is proposed, which can be called FMBF-QR. On one side, the optimization problem of MBF is improved to be an iteratively reweighted L 2 , 1 norm minimization problem to enhance the accuracy of MBF. On the other side, the minimization problem of FMBF-QR is optimized very efficiently by using QR decomposition for improving the speed of MBF. Sufficient experimental results verify that FMBF-QR can converge with a higher accuracy and a faster speed than the traditional matrix completion methods.

AMCSMMA: Predicting Small Molecule-miRNA Potential Associations Based on Accurate Matrix Completion.

Exploring potential associations between small molecule drugs (SMs) and microRNAs (miRNAs) is significant for drug development and disease treatment. Since biological experiments are expensive and time-consuming, we propose a computational model based on accurate matrix completion for predicting potential SM-miRNA associations (AMCSMMA). Initially, a heterogeneous SM-miRNA network is constructed, and its adjacency matrix is taken as the target matrix. An optimization framework is then proposed to recover the target matrix with the missing values by minimizing its truncated nuclear norm, an accurate, robust, and efficient approximation to the rank function. Finally, we design an effective two-step iterative algorithm to solve the optimization problem and obtain the prediction scores. After determining the optimal parameters, we conduct four kinds of cross-validation experiments based on two datasets, and the results demonstrate that AMCSMMA is superior to the state-of-the-art methods. In addition, we implement another validation experiment, in which more evaluation metrics in addition to the AUC are introduced and finally achieve great results. In two types of case studies, a large number of SM-miRNA pairs with high predictive scores are confirmed by the published experimental literature. In summary, AMCSMMA has superior performance in predicting potential SM-miRNA associations, which can provide guidance for biological experiments and accelerate the discovery of new SM-miRNA associations.

Nonconvex Tensor Relative Total Variation for Image Completion

Image completion, which falls to a special type of inverse problems, is an important but challenging task. The difficulties lie in that (i) the datasets usually appear to be multi-dimensional; (ii) the unavailable or corrupted data entries are randomly distributed. Recently, low-rank priors have gained importance in matrix completion problems and signal separation; however, due to the complexity of multi-dimensional data, using a low-rank prior by itself is often insufficient to achieve desirable completion, which requires a more comprehensive approach. In this paper, different from current available approaches, we develop a new approach, called relative total variation (TRTV), under the tensor framework, to effectively integrate the local and global image information for data processing. Based on our proposed framework, a completion model embedded with TRTV and tensor p-shrinkage nuclear norm minimization with suitable regularization is established. An alternating direction method of multiplier (ADMM)-based algorithm under our framework is presented. Extensive experiments in terms of denoising and completion tasks demonstrate our proposed method are not only effective but also superior to existing approaches in the literature.

A robust mixed error coding method based on nonconvex sparse representation

Linear representation based methods have been extensively applied in image recognition, especially for those with noise, illumination changes, and occlusions. However, most existing methods assume a specific distribution for image noise estimation, which is intractable to handle complex variations. Besides, they usually use convex norm to describe the noise sparse and low-rank property, and it is a biased approximation. To address these problems, we propose a novel nonconvex regularized robust mixed error coding (NRRM) method, which uses mixed norms from both 1D and 2D perspectives to model the complex image noise without convex relaxation. In specific, we use weighted ℓ2-norm based robust coding to characterize the sparse noise in images, and weighted matrix nuclear norm to characterize the low-rank noise. Compared with traditional regression approaches, our method can more fine-grained and accurate to capture noise and alleviate its negative influence for robust recognition. Besides, we constrain the representation component in a group-wise manner to weigh the roles of different classes. The NRRM model is solved efficiently by adopting an alternating direction method of multipliers (ADMM) algorithm. Comprehensive experiments on some benchmark face image databases validate the superiority of NRRM over several state-of-the-art linear representation based methods.

Invertible linear transforms based adaptive multi-view subspace clustering

Constructing tensor with low-rank prior is the crucial issue of tensor based multi-view subspace clustering methods, but there are still some shortcomings. First, they cannot adaptively allocate the contribution of different views, resulting in the learned tensor being suboptimal. Second, they pursue low-rank tensor for different data through Discrete Fourier Transform based tensor nuclear norm, which lacks generality. To overcome these problems, we propose an invertible linear transforms based adaptive multi-view subspace clustering method, named ILTMSC. Firstly, we mine the potential low-order representation of each view through self-representation subspace learning. Then, we capture high-order representation by integrating low-order representations with adaptive weights into a tensor and then rotated. This strategy can integrate tensor adaptively and handle the noise effectively. Finally, we approximate the low-rank tensor with a recently proposed invertible linear transforms based tensor nuclear norm. Such a new tensor nuclear norm makes our model more general because it can use different invertible linear transforms for different tensor data. Moreover, an adaptive weighted tensor singular value thresholding operator is proposed for capturing the new tensor nuclear norm. Our model could be solved by convex optimization efficiently. Extensive experiments on multi-view datasets validate the effectiveness and robustness of our method.

Flexible Hyperspectral Anomaly Detection Using Weighted Nuclear Norm

It has been demonstrated that nuclear-norm-based low-rank representation is capable of modeling cluttered backgrounds in hyperspectral images (HSIs) for robust anomaly detection. However, minimizing the nuclear norm regularizes each singular value equally during rank reduction, which restricts the capacity and flexibility of modeling the major structures of the background. To address this problem, we propose detection of anomaly pixels in HSIs using the weighted nuclear norm, which can preserve the major singular values during rank reduction. We present a down-up sampling scheme to remove plausible anomaly pixels from the image as much as possible and learn a robust principal component analysis (PCA) background dictionary. From a dictionary, we develop a weighted nuclear-norm minimization model to represent the background with a low-rank coefficients matrix that can be effectively optimized using the standard alternating direction method of multipliers (ADMM). Due to the flexible modeling capacity using the weighted nuclear norm, anomaly pixels can be distinguished from the background with the reconstruction error. The experimental results on two real HSIs datasets demonstrate the effectiveness of the proposed method for anomaly detection.