Low-rank Tensor Completion Problem Research Articles

Editorial: Tensor numerical methods and their application in scientific computing and data science

Editorial: Tensor numerical methods and their application in scientific computing and data science

Enhancing low-rank tensor completion via first-order and second-order total variation regularizations

<p style='text-indent:20px;'>Recently, low-rank tensor completion (LRTC) has attracted significant attention since it has been applied in a wide variety of practical areas. Due to the edge-preserving and noise removal properties, total variation (TV) has been extensively used in LRTC problems. However, first-order TV usually causes unexpected staircase effects. In this paper, we focus on the LRTC problem with various degradations, which aims to recover third-order tensors from partial observations corrupted by sparse noise and Gaussian noise. We use the transformed tensor nuclear norm to explore global low-rankness, and the combination of first-order and second-order TV regularizations to alleviate the staircase effects caused by first-order TV. Based on these, we propose a first-order and second-order TV regularizations (FSTV) model. In order to solve the proposed FSTV model, a symmetric Gauss-Seidel based alternating direction method of multipliers is adopted. We also establish its global convergence under very mild conditions. Finally, extensive experiments on different video and multispectral image datasets show the superiority of the proposed method compared with several state-of-the-art methods.</p>

Nonconvex Tensor Completion for 5-D Seismic Data Reconstruction

Multidimensional pre-stack seismic data reconstruction can be viewed as a low-rank tensor completion problem. Recently, the nuclear norm has been widely used as a convex surrogate of the tensor rank function for low-rank tensor recovery and has been successfully applied to 5D seismic data reconstruction. However, solving the nuclear norm-based relaxed convex problem typically leads to a suboptimal solution of the original rank minimization problem, often degrading the reconstruction performance. In this study, to seek solutions to the aforementioned problems, we established a non-convex logDet function as a smooth approximation for the tensor rank instead of the convex tensor nuclear norm and applied it to solve the 5D seismic data reconstruction problem. Thereafter, we propose solving the obtained non-convex relaxation problem using an alternating direction method of multipliers (ADMM) algorithm. Numerical experiments of our approach on synthetic 5D seismic data demonstrated remarkable reconstruction performance compared with the performances of HOSVD, nulear norm, and PMF methods in terms of visual examination and numerical test. We further illustrate the performance of the proposed method using a land data survey.

Multilayer Sparsity-Based Tensor Decomposition for Low-Rank Tensor Completion.

Existing methods for tensor completion (TC) have limited ability for characterizing low-rank (LR) structures. To depict the complex hierarchical knowledge with implicit sparsity attributes hidden in a tensor, we propose a new multilayer sparsity-based tensor decomposition (MLSTD) for the low-rank tensor completion (LRTC). The method encodes the structured sparsity of a tensor by the multiple-layer representation. Specifically, we use the CANDECOMP/PARAFAC (CP) model to decompose a tensor into an ensemble of the sum of rank-1 tensors, and the number of rank-1 components is easily interpreted as the first-layer sparsity measure. Presumably, the factor matrices are smooth since local piecewise property exists in within-mode correlation. In subspace, the local smoothness can be regarded as the second-layer sparsity. To describe the refined structures of factor/subspace sparsity, we introduce a new sparsity insight of subspace smoothness: a self-adaptive low-rank matrix factorization (LRMF) scheme, called the third-layer sparsity. By the progressive description of the sparsity structure, we formulate an MLSTD model and embed it into the LRTC problem. Then, an effective alternating direction method of multipliers (ADMM) algorithm is designed for the MLSTD minimization problem. Various experiments in RGB images, hyperspectral images (HSIs), and videos substantiate that the proposed LRTC methods are superior to state-of-the-art methods.

Online subspace learning and imputation by Tensor-Ring decomposition

This paper considers the completion problem of a partially observed high-order streaming data, which is cast as an online low-rank tensor completion problem. Though the online low-rank tensor completion problem has drawn lots of attention in recent years, most of them are designed based on the traditional decomposition method, such as CP and Tucker. Inspired by the advantages of Tensor Ring decomposition over the traditional decompositions in expressing high-order data and its superiority in missing values estimation, this paper proposes two online subspace learning and imputation methods based on Tensor Ring decomposition. Specifically, we first propose an online Tensor Ring subspace learning and imputation model by formulating an exponentially weighted least squares with Frobenium norm regularization of TR-cores. Then, two commonly used optimization algorithms, i.e. alternating recursive least squares and stochastic-gradient algorithms, are developed to solve the proposed model. Numerical experiments show that the proposed methods are more effective to exploit the time-varying subspace in comparison with the conventional Tensor Ring completion methods. Besides, the proposed methods are demonstrated to be superior to obtain better results than state-of-the-art online methods in streaming data completion under varying missing ratios and noise.

3-D Array Image Data Completion by Tensor Decomposition and Nonconvex Regularization Approach

Various image datasets appear naturally in the form of multi-dimensional arrays (hypermatrices), called tensors. Image with incomplete entries, which often can be formulated as the low-rank tensor completion problem, is practically important in order to process large size 3D array images both efficiently and effectively. To make the low-rank approximation being tractable, most current approaches make use of the lower convex envelope of tensor multi-rank function for the approximation. Due to the gap between the rank function and its lower convex envelope, the adoption of tensor nuclear norm may lead to the approximation of the corresponding tensor tubal-rank being insufficient. In this paper, we introduce a new nonconvex regularization approach, which can better capture the low-rank characteristics than the convex approach. By transforming the original problem to the Fourier domain, we formulate an equivalent optimization problem with more transparent tensor rank characteristics, whose explicit solution can be obtained under our framework. A minimization algorithm, associated with the augmented Lagrangian multipliers and the nonconvex regularizer, is established and is shown to be feasible. The constructed sequence converges to the desirable Karush-Kuhn-Tucker point, which is mathematically validated in detail. Extensive experimental results demonstrate that our proposed approach outperforms the existing state-of-the-art convex approaches consistently.

Deep Unrolled Low-Rank Tensor Completion for High Dynamic Range Imaging.

The major challenge in high dynamic range (HDR) imaging for dynamic scenes is suppressing ghosting artifacts caused by large object motions or poor exposures. Whereas recent deep learning-based approaches have shown significant synthesis performance, interpretation and analysis of their behaviors are difficult and their performance is affected by the diversity of training data. In contrast, traditional model-based approaches yield inferior synthesis performance to learning-based algorithms despite their theoretical thoroughness. In this paper, we propose an algorithm unrolling approach to ghost-free HDR image synthesis algorithm that unrolls an iterative low-rank tensor completion algorithm into deep neural networks to take advantage of the merits of both learning- and model-based approaches while overcoming their weaknesses. First, we formulate ghost-free HDR image synthesis as a low-rank tensor completion problem by assuming the low-rank structure of the tensor constructed from low dynamic range (LDR) images and linear dependency among LDR images. We also define two regularization functions to compensate for modeling inaccuracy by extracting hidden model information. Then, we solve the problem efficiently using an iterative optimization algorithm by reformulating it into a series of subproblems. Finally, we unroll the iterative algorithm into a series of blocks corresponding to each iteration, in which the optimization variables are updated by rigorous closed-form solutions and the regularizers are updated by learned deep neural networks. Experimental results on different datasets show that the proposed algorithm provides better HDR image synthesis performance with superior robustness compared with state-of-the-art algorithms, while using significantly fewer training samples.

Tensor Q-rank: new data dependent definition of tensor rank

Recently, the \({ Tensor}~{ Nuclear}~{ Norm}~{ (TNN)}\) regularization based on t-SVD has been widely used in various low tubal-rank tensor recovery tasks. However, these models usually require smooth change of data along the third dimension to ensure their low rank structures. In this paper, we propose a new definition of data dependent tensor rank named tensor Q-rank by a learnable orthogonal matrix \(\mathbf {Q}\), and further introduce a unified data dependent low rank tensor recovery model. According to the low rank hypothesis, we introduce two explainable selection methods of \(\mathbf {Q}\), under which the data tensor may have a more significant low tensor Q-rank structure than that of low tubal-rank structure. Specifically, maximizing the variance of singular value distribution leads to Variance Maximization Tensor Q-Nuclear norm (VMTQN), while minimizing the value of nuclear norm through manifold optimization leads to Manifold Optimization Tensor Q-Nuclear norm (MOTQN). Moreover, we apply these two models to the low rank tensor completion problem, and then give an effective algorithm and briefly analyze why our method works better than TNN based methods in the case of complex data with low sampling rate. Finally, experimental results on real-world datasets demonstrate the superiority of our proposed models in the tensor completion problem with respect to other tensor rank regularization models.

Robust Online Prediction of Spectrum Map With Incomplete and Corrupted Observations

Spectrum map is an essential tool for a range of emerging applications of 5G and 6G networks. Despite the great efforts that have been put on the construction of spectrum maps, access to accurate and valid spectrum data in dynamically changing environments emphasizes the need for more advanced solutions tailored to such rapidly varying scenarios. To this end, the idea of spectrum map prediction is introduced. In this paper, we address the problem of spectrum map prediction from historical spectrum observations in the dynamically changing environments. The problem is particularly challenging when the available historical spectrum observations are incomplete and corrupted by anomalies. We propose three techniques to solve the problem. First, we combine the spectrum map with prediction functionalities so as to offer a huge potential for efficient resource management and flexible sharing of resources in dynamically changing environments. Second, by fully exploiting the hidden spatial-temporal-spectral structures of the spectrum data and the sparsity of anomalies and missing data, we model the spectrum map as a 3rd-order spectrum tensor and formulate the spectrum map prediction problem as a low-rank tensor completion problem. Third, we design a robust online spectrum map prediction (ROSMP) algorithm based on the alternating direction minimization method, which derives the tensor decomposition factors for a new timeslot based on the update of existing ones rather than re-computing from the scratch. By gradually learning the hidden spatial-temporal-spectral structures of the spectrum data, ROSMP is able to predict and obtain the complete spectrum map with high accuracy. Finally, extensive numerical evaluations using a real spectrum measurement dataset confirm the efficacy and efficiency of ROSMP and show the superiority of ROSMP over the baselines.

Anomaly-Aware Network Traffic Estimation via Outlier-Robust Tensor Completion

Accurately estimating network traffic from the partial measurements plays a crucial role in network management. However, the potential anomaly existing in real networks usually makes this goal difficult to achieve. Existing network traffic estimation methods generally impute network traffic independent of anomaly detection, which incurs significant performance degradation with network anomaly. To address this issue in the realistic network scenario, we propose a novel anomaly-aware network traffic estimation method to recover network traffic data concurrently with network anomaly detection. Specifically, by exploiting the inherent spatio-temporal characteristics, we first formulate the network traffic estimation as a low-rank tensor completion problem. Then, an outlier-robust tensor completion (OrTC) model is constructed by introducing both L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm regularization and L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</sub> -norm regularization, which can not only well fit the intrinsic low-rank property of real traffic data, but also is robust against both the dense noise and the sparse anomaly. Furthermore, an effective optimization algorithm OrTC-AM is designed to solve the non-convex and non-smooth OrTC model based on the popular alternating minimization method. Finally, the extensive experiments performed on the public dataset demonstrate that our proposed OrTC-AM method outperforms the previously widely used network traffic estimation methods.

Nonlinear System Identification via Tensor Completion

Function approximation from input and output data pairs constitutes a fundamental problem in supervised learning. Deep neural networks are currently the most popular method for learning to mimic the input-output relationship of a general nonlinear system, as they have proven to be very effective in approximating complex highly nonlinear functions. In this work, we show that identifying a general nonlinear function y = ƒ(x1,…,xN) from input-output examples can be formulated as a tensor completion problem and under certain conditions provably correct nonlinear system identification is possible. Specifically, we model the interactions between the N input variables and the scalar output of a system by a single N-way tensor, and setup a weighted low-rank tensor completion problem with smoothness regularization which we tackle using a block coordinate descent algorithm. We extend our method to the multi-output setting and the case of partially observed data, which cannot be readily handled by neural networks. Finally, we demonstrate the effectiveness of the approach using several regression tasks including some standard benchmarks and a challenging student grade prediction task.

Two-dimensional seismic data reconstruction using patch tensor completion

Seismic data are often undersampled owing to physical or financial limitations. However, complete and regularly sampled data are becoming increasingly critical in seismic processing. In this paper, we present an efficient two-dimensional (2D) seismic data reconstruction method that works on texture-based patches. It performs completion on a patch tensor, which folds texture-based patches into a tensor. Reconstruction is performed by reducing the rank using tensor completion algorithms. This approach differs from past methods, which proceed by unfolding matrices into columns and then applying common matrix completion approaches to deal with 2D seismic data reconstruction. Here, we first re-arrange the seismic data matrix into a third-order patch tensor, by stacking texture-based patches that are divided from seismic data. Then, the seismic data reconstruction problem is formulated into a low-rank tensor completion problem. This formulation avoids destroying the spatial structure, and better extracts the underlying useful information. The proposed method is efficient and gives an improved performance compared with traditional approaches. The effectiveness of our patch tensor-based framework is validated using two classical tensor completion algorithms, low-rank tensor completion (LRTC), and the parallel matrix factorization algorithm (TMac), on both synthetic and field data experiments.

Tensor Factorization with Total Variation and Tikhonov Regularization for Low-Rank Tensor Completion in Imaging Data

The main aim of this paper is to study tensor factorization for low-rank tensor completion in imaging data. Due to the underlying redundancy of real-world imaging data, the low-tubal-rank tensor factorization (the tensor–tensor product of two factor tensors) can be used to approximate such tensor very well. Motivated by the spatial/temporal smoothness of factor tensors in real-world imaging data, we propose to incorporate a hybrid regularization combining total variation and Tikhonov regularization into low-tubal-rank tensor factorization model for low-rank tensor completion problem. We also develop an efficient proximal alternating minimization (PAM) algorithm to tackle the corresponding minimization problem and establish a global convergence of the PAM algorithm. Numerical results on color images, color videos, and multispectral images are reported to illustrate the superiority of the proposed method over competing methods.

Fast online low-rank tensor subspace tracking by CP decomposition using recursive least squares from incomplete observations

This paper considers the problem of online tensor subspace tracking of a partially observed high-dimensional data stream corrupted by noise, where we assume that the data lie in a low-dimensional linear subspace. This problem is cast as an online low-rank tensor completion problem. We propose a novel online tensor subspace tracking algorithm based on the CANDECOMP/PARAFAC (CP) decomposition, dubbed OnLine Low-rank Subspace tracking by TEnsor CP Decomposition (OLSTEC). The proposed algorithm specifically addresses the case in which data of interest are fed into the algorithm over time infinitely, and their subspace are dynamically time-varying. To this end, we build up our proposed algorithm exploiting the recursive least squares (RLS), which is a second-order gradient algorithm. Numerical evaluations on synthetic datasets and real-world datasets such as communication network traffic, environmental data, and surveillance videos, show that the proposed OLSTEC algorithm outperforms state-of-the-art online algorithms in terms of the convergence rate per iteration.

Accurate Recovery of Internet Traffic Data: A Sequential Tensor Completion Approach

The inference of traffic volume of the whole network from partial traffic measurements becomes increasingly critical for various network engineering tasks, such as capacity planning and anomaly detection. Previous studies indicate that the matrix completion is a possible solution for this problem. However, as a 2-D matrix cannot sufficiently capture the spatial-temporal features of traffic data, these approaches fail to work when the data missing ratio is high. To fully exploit hidden spatial-temporal structures of the traffic data, this paper models the traffic data as a 3-way traffic tensor and formulates the traffic data recovery problem as a low-rank tensor completion problem. However, the high computation complexity incurred by the conventional tensor completion algorithms prevents its practical application for the traffic data recovery. To reduce the computation cost, we propose a novel sequential tensor completion algorithm, which can efficiently exploit the tensor decomposition result based on the previous traffic data to derive the tensor decomposition upon arriving of new data. Furthermore, to better capture the changes of data correlation over time, we propose a dynamic sequential tensor completion algorithm. To the best of our knowledge, we are the first to propose sequential tensor completion algorithms to significantly speed up the traffic data recovery process. This facilitates the modeling of Internet traffic with the tensor to well exploit the hidden structures of traffic data for more accurate missing data inference. We have done extensive simulations with the real traffic trace as the input. The simulation results demonstrate that our algorithms can achieve significantly better performance compared with the literature tensor and matrix completion algorithms even when the data missing ratio is high.

A non-convex tensor rank approximation for tensor completion

• A non-convex approximation for tensors rank is proposed for low-rank tensor completion problem. • Develop an efficient ADMM algorithm for solving the proposed method. • The method can efficiently estimate the missing information, especially for the low sampling rates. Low-rankness has been widely exploited for the tensor completion problem. Recent advances have suggested that the tensor nuclear norm often leads to a promising approximation for the tensor rank. It treats the singular values equally to pursue the convexity of the objective function, while the singular values for the practical images have clear physical meanings with different importance and should be treated differently. In this paper, we propose a non-convex logDet function as a smooth approximation for tensor rank instead of the convex tensor nuclear norm and introduce it into the low-rank tensor completion problem. An alternating direction method of multiplier (ADMM)-based method is developed to solve the problem. Experimental results have shown that the proposed method can significantly outperform existing state-of-the-art nuclear norm-based methods for tensor completion.

Hankel Matrix Nuclear Norm Regularized Tensor Completion for $N$-dimensional Exponential Signals

Signals are generally modeled as a superposition of exponential functions in spectroscopy of chemistry, biology, and medical imaging. For fast data acquisition or other inevitable reasons, however, only a small amount of samples may be acquired, and thus, how to recover the full signal becomes an active research topic, but existing approaches cannot efficiently recover <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -dimensional exponential signals with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N\geq 3$</tex-math></inline-formula> . In this paper, we study the problem of recovering <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -dimensional (particularly <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N\geq 3$</tex-math></inline-formula> ) exponential signals from partial observations, and formulate this problem as a low-rank tensor completion problem with exponential factor vectors. The full signal is reconstructed by simultaneously exploiting the CANDECOMP/PARAFAC tensor structure and the exponential structure of the associated factor vectors. The latter is promoted by minimizing an objective function involving the nuclear norm of Hankel matrices. Experimental results on simulated and real magnetic resonance spectroscopy data show that the proposed approach can successfully recover full signals from very limited samples and is robust to the estimated tensor rank.

Hybrid Singular Value Thresholding for Tensor Completion

In this paper, we study the low-rank tensor completion problem, where a high-order tensor with missing entries is given and the goal is to complete the tensor. We propose to minimize a new convex objective function, based on log sum of exponentials of nuclear norms, that promotes the low-rankness of unfolding matrices of the completed tensor. We show for the first time that the proximal operator to this objective function is readily computable through a hybrid singular value thresholding scheme. This leads to a new solution to high-order (low-rank) tensor completion via convex relaxation. We show that this convex relaxation and the resulting solution are much more effective than existing tensor completion methods (including those also based on minimizing ranks of unfolding matrices). The hybrid singular value thresholding scheme can be applied to any problem where the goal is to minimize the maximum rank of a set of low-rank matrices.