Tensor Completion Model Research Articles

On the Synergy between Nonconvex Extensions of the Tensor Nuclear Norm for Tensor Recovery

Low-rank tensor recovery has attracted much attention among various tensor recovery approaches. A tensor rank has several definitions, unlike the matrix rank—e.g., the CP rank and the Tucker rank. Many low-rank tensor recovery methods are focused on the Tucker rank. Since the Tucker rank is nonconvex and discontinuous, many relaxations of the Tucker rank have been proposed, e.g., the sum of nuclear norm, weighted tensor nuclear norm, and weighted tensor schatten-p norm. In particular, the weighted tensor schatten-p norm has two parameters, the weight and p, and the sum of nuclear norm and weighted tensor nuclear norm are special cases of these parameters. However, there has been no detailed discussion of whether the effects of the weighting and p are synergistic. In this paper, we propose a novel low-rank tensor completion model using the weighted tensor schatten-p norm to reveal the relationships between the weight and p. To clarify whether complex methods such as the weighted tensor schatten-p norm are necessary, we compare them with a simple method using rank-constrained minimization. It was found that the simple methods did not outperform the complex methods unless the rank of the original tensor could be accurately known. If we can obtain the ideal weight, p=1 is sufficient, although it is necessary to set p<1 when using the weights obtained from observations. These results are consistent with existing reports.

Reshaped tensor nuclear norms for higher order tensor completion

We investigate optimal conditions for inducing low-rankness of higher order tensors by using convex tensor norms with reshaped tensors. We propose the reshaped tensor nuclear norm as a generalized approach to reshape tensors to be regularized by using the tensor nuclear norm. Furthermore, we propose the reshaped latent tensor nuclear norm to combine multiple reshaped tensors using the tensor nuclear norm. We analyze the generalization bounds for tensor completion models regularized by the proposed norms and show that the novel reshaping norms lead to lower Rademacher complexities. Through simulation and real-data experiments, we show that our proposed methods are favorably compared to existing tensor norms consolidating our theoretical claims.

A Novel Regularized Model for Third-Order Tensor Completion

Inspired by the accuracy and efficiency of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\gamma$</tex-math></inline-formula> -norm of a matrix, which is closer to the original rank minimization problem than nuclear norm minimization (NNM), we generalize the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\gamma$</tex-math></inline-formula> -norm of a matrix to tensors and propose a new tensor completion approach within the tensor singular value decomposition (t-svd) framework. An efficient algorithm, which combines the augmented Lagrange multiplier and closed-resolution of a cubic equation, was developed to solve the associated nonconvex tensor multi-rank minimization problem. Experimental results show that the proposed approach has an advantage over current state of the art algorithms in recovery accuracy.

Multi-Dimensional Visual Data Completion via Low-Rank Tensor Representation Under Coupled Transform.

This paper addresses the tensor completion problem, which aims to recover missing information of multi-dimensional images. How to represent a low-rank structure embedded in the underlying data is the key issue in tensor completion. In this work, we suggest a novel low-rank tensor representation based on coupled transform, which fully exploits the spatial multi-scale nature and redundancy in spatial and spectral/temporal dimensions, leading to a better low tensor multi-rank approximation. More precisely, this representation is achieved by using two-dimensional framelet transform for the two spatial dimensions, one/two-dimensional Fourier transform for the temporal/spectral dimension, and then Karhunen-Loéve transform (via singular value decomposition) for the transformed tensor. Based on this low-rank tensor representation, we formulate a novel low-rank tensor completion model for recovering missing information in multi-dimensional visual data, which leads to a convex optimization problem. To tackle the proposed model, we develop the alternating directional method of multipliers (ADMM) algorithm tailored for the structured optimization problem. Numerical examples on color images, multispectral images, and videos illustrate that the proposed method outperforms many state-of-the-art methods in qualitative and quantitative aspects.

Enhanced Sparsity Prior Model for Low-Rank Tensor Completion.

Conventional tensor completion (TC) methods generally assume that the sparsity of tensor-valued data lies in the global subspace. The so-called global sparsity prior is measured by the tensor nuclear norm. Such assumption is not reliable in recovering low-rank (LR) tensor data, especially when considerable elements of data are missing. To mitigate this weakness, this article presents an enhanced sparsity prior model for LRTC using both local and global sparsity information in a latent LR tensor. In specific, we adopt a doubly weighted strategy for nuclear norm along each mode to characterize global sparsity prior of tensor. Different from traditional tensor-based local sparsity description, the proposed factor gradient sparsity prior in the Tucker decomposition model describes the underlying subspace local smoothness in real-world tensor objects, which simultaneously characterizes local piecewise structure over all dimensions. Moreover, there is no need to minimize the rank of a tensor for the proposed local sparsity prior. Extensive experiments on synthetic data, real-world hyperspectral images, and face modeling data demonstrate that the proposed model outperforms state-of-the-art techniques in terms of prediction capability and efficiency.

Anomaly-Aware Network Traffic Estimation via Outlier-Robust Tensor Completion

Accurately estimating network traffic from the partial measurements plays a crucial role in network management. However, the potential anomaly existing in real networks usually makes this goal difficult to achieve. Existing network traffic estimation methods generally impute network traffic independent of anomaly detection, which incurs significant performance degradation with network anomaly. To address this issue in the realistic network scenario, we propose a novel anomaly-aware network traffic estimation method to recover network traffic data concurrently with network anomaly detection. Specifically, by exploiting the inherent spatio-temporal characteristics, we first formulate the network traffic estimation as a low-rank tensor completion problem. Then, an outlier-robust tensor completion (OrTC) model is constructed by introducing both L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm regularization and L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</sub> -norm regularization, which can not only well fit the intrinsic low-rank property of real traffic data, but also is robust against both the dense noise and the sparse anomaly. Furthermore, an effective optimization algorithm OrTC-AM is designed to solve the non-convex and non-smooth OrTC model based on the popular alternating minimization method. Finally, the extensive experiments performed on the public dataset demonstrate that our proposed OrTC-AM method outperforms the previously widely used network traffic estimation methods.

Adaptive total variation and second-order total variation-based model for low-rank tensor completion

Recently, low-rank regularization has achieved great success in tensor completion. However, only considering the global low-rankness is not sufficient, especially for a low sampling rate (SR). Total variation (TV) is introduced into low-rank tensor completion (LRTC) problem to promote the local smoothness by incorporating the first-order derivatives information. However, TV usually leads to undesirable staircase effects. To alleviate these staircase effects, we suggest a first- and second-order TV-based parallel matrix factorization model for LRTC problem, which integrates the local smoothness and global low-rankness by simultaneously exploiting the first- and second-order derivatives information. To solve the proposed model, an efficient proximal alternating optimization (PAO)-based algorithm is developed with theoretical guarantee. Moreover, we suggest a regularization parameter selection strategy to automatically update two regularization parameters, which is able to take advantage of the best properties of each of the two regularization terms. Extensive experiments on different tensor data show the superiority of the proposed method over other methods, particularly for extremely low SRs.

Optimised routing and compressive sensing based data communication in wireless sensor network

Wireless sensor networks (WSNs) play a prominent role in real-time applications, like traffic monitoring, tracking the intruder, and so on, which are duly based on the destination. Compressive sensing tracks a wide application in tracking the location, and this study uses the Kronecker basis representation-based tensor completion (KBR-TC) model for availing the data communication in WSNs. Additionally, this study proposes a fractional earthworm optimisation (FEWO) algorithm for finding the optimal cluster heads (CHs) in the network. Once the CHs are selected, the multipath is optimally finalised using the fit factor that depends on the energy, trust, distance and delay. The proposed FEWO is designed by modifying standard earthworm algorithm (EWA) with the fractional concept. Finally, the data is compressed using the KBR-TC model, for data transmission, and the data is sent to the destination, where the data is reconstructed. The analysis of the methods based on energy (J), throughput (%) and delay (ms) reveal that the proposed FEWO method outperformed the existing methods with the maximal energy, throughput and minimal delay of 0.1295 J, 0.1295 J and 88.5768%, respectively.

Deep plug-and-play prior for low-rank tensor completion

Multi-dimensional images, such as color images and multi-spectral images (MSIs), are highly correlated and contain abundant spatial and spectral information. However, real-world multi-dimensional images are usually corrupted by missing entries. By integrating deterministic low-rankness prior to the data-driven deep prior, we suggest a novel regularized tensor completion model for multi-dimensional image completion. In the objective function, we adopt the newly emerged tensor nuclear norm (TNN) to characterize the global low-rankness prior of multi-dimensional images. We also formulate an implicit regularizer by plugging a denoising neural network (termed as deep denoiser), which is convinced to express the deep image prior learned from a large number of natural images. The resulting model can be solved by the alternating directional method of multipliers algorithm under the plug-and-play (PnP) framework. Experimental results on color images, videos, and MSIs demonstrate that the proposed method can recover both the global structure and fine details very well and achieve superior performance over competing methods in terms of quality metrics and visual effects.

An efficient tensor completion method via truncated nuclear norm

Tensor completion aims to recover missing entries from partial observations for multi-dimensional data. Traditional tensor completion algorithms process the dimensional data by unfolding the tensor into matrices, which breaks the inherent correlation and dependencies in multiple channels and lead to critical information loss. In this paper, we propose a novel tensor completion model for visual multi-dimensional data completion under the tensor singular value decomposition (t-SVD) framework. In the proposed method, tensor is treated as a whole and a truncated nuclear norm regularization is employed to exploit the structural properties in a tensor and hidden information existing among the adjacent channels of a tensor. Besides, we introduce a weighted tensor to adjust the residual error of each frontal slices in consideration of their different recovery statistics. It does enhance the sparsity of all unfoldings of the tensor and accelerates the convergence of the proposed method. Experimental results on various visual datasets demonstrate the promising performance of the proposed method in comparison with the state-of-the-art tensor completion methods.

Tensor train rank minimization with hybrid smoothness regularization for visual data recovery

Recently, the tensor train (TT) rank has received much attention for tensor completion, due to its ability to explore the global low-rankness of tensors. However, existing methods still leave room for improvement, since the low-rankness itself is generally not sufficient to recover the underlying data. Inspired by this, we consider a novel tensor completion model by simultaneously exploiting the global low-rankness and local smoothness of visual data. In particular, we use low-rank matrix factorization to characterize the global TT low-rankness, and framelet and total variation regularization to enhance the local smoothness. We develop an efficient proximal alternating minimization algorithm to solve the proposed new model with guaranteed convergence. Extensive experiments on various data demonstrated that the proposed method outperforms compared methods in terms of visual and quantitative measures.

Robust Schatten-p Norm Based Approach for Tensor Completion

The matrix nuclear norm has been widely applied to approximate the matrix rank for low-rank tensor completion because of its convexity. However, this relaxation may make the solution seriously deviate from the original solution for real-world data recovery. In this paper, using a nonconvex approximation of rank, i.e., the Schatten-p norm, we propose a novel model for tensor completion. It’s hard to solve this model directly because the objective function of the model is nonconvex. To solve the model, we develop a variant of this model via the classical quadric penalty method, and propose an algorithm, i.e., SpBCD, based on the block coordinate descent method. Although the objective function of the variant is nonconvex, we show that the sequence generated by SpBCD is convergent to a critical point. Our numerical experiments on real-world data show that SpBCD delivers state-of-art performance in recovering missing data.

Robust Tensor Factorization for Color Image and Grayscale Video Recovery

Low-rank tensor completion (LRTC) plays an important role in many fields, such as machine learning, computer vision, image processing, and mathematical theory. Since rank minimization is an NP-hard problem, one strategy is that it is converted into a convex relaxation tensor nuclear norm (TNN) that requires the repeated calculation of time-consuming SVD, and the other is to convert it into some product of two smaller tensors that are easy to fall into the local minimum. In order to overcome the above shortcomings, we propose a robust tensor factorization (RTF) model for solving LRTC. In RTF, the noisy tensor data with missing entries is decomposed into low-rank tensor and noisy tensor, and then the low-rank tensor is equivalently decomposed into t-products (essentially vectors convolution) of two smaller tensors: orthogonal dictionary tensor and low-rank representation tensor. Meanwhile, the TNN of low-rank representation tensor is adopted to characterize the low-rank structure of the tensor data for preserving global information. Then, an effective iterative update algorithm based on the alternating direction method of multipliers (ADMM) is proposed to solve RTF. Finally, numerical experiments on image recovering and video completion tasks show the effectiveness of the proposed RTF model compared with several state-of-the-art tensor completion models.

Traffic Flow Prediction With Missing Data Imputed by Tensor Completion Methods

Missing data is inevitable and ubiquitous in intelligent transportation systems (ITSs). A handful of completion methods have been proposed, among which the tensor-based models have been shown to be the most advantageous for missing traffic data imputation. Despite their superior imputation accuracies, the adoption of these imputed data is not uniform in modern ITSs applications. The primary goal of this paper is to explore how to use tensor completion methods to support ITSs. In particular, we study how to improve traffic flow prediction accuracy under different missing scenarios. Specifically, three common missing scenarios including element-wise random missing, time-structured missing, and space-structured missing are considered. Four classical tensor completion models including Smooth PARAFAC Decomposition based Completion (SPC), CP Decomposition-based (CP-WOPT) Completion, Tucker Decomposition-based Completion (TDI), and High-accuracy Low-rank Tensor Completion (HaLRTC) are used to impute the missing data. Four well-known prediction methods including Support Vector Regression (SVR), K-nearest Neighbor (KNN), Gradient Boost Regression Tree (GBRT), and Long Short-term Memory (LSTM) are tested. The simple mean value interpolation completed traffic data is regarded as the baseline data. The extensive experiments show that improvements of traffic flow prediction can be achieved by increasing missing traffic data imputation accuracy at most cases. Interestingly we find that prediction accuracy cannot be improved by an imputation model when the sparsely observed training datasets already provide sufficient training samples.

Optimality Verification of Tensor Completion Model via Self-Validation

Tensor completion (TC) attempts to estimate the missing tensor entries from partial observations. Over the past decade, a tremendous amount of work has focused on this problem, which has given rise to a few excellent TC methods. Thus, a critical question is how to verify the optimality of TC models, especially when the underlying tensor is unknown, which is a common scenario in practice. The only existing method available—data validation—addresses the problem based on the validation set. While straightforward, this method may lead to a nonoptimal model, especially in scenarios with high missing ratios, and noise. In this work, we propose a self-validation method, which accounts for a metric named the identifiability measure , defined based on the generalization of isomeric conditions in the tensor case. It is noteworthy that the identifiability measure can verify the optimality of TC models without the use of any validation set. Extensive numerical experiments conducted on both synthetic, and real-world datasets empirically validate the superiority of the proposed method over the data validation approach.

Accurate Tensor Completion via Adaptive Low-Rank Representation.

Low-rank representation-based approaches that assume low-rank tensors and exploit their low-rank structure with appropriate prior models have underpinned much of the recent progress in tensor completion. However, real tensor data only approximately comply with the low-rank requirement in most cases, viz., the tensor consists of low-rank (e.g., principle part) as well as non-low-rank (e.g., details) structures, which limit the completion accuracy of these approaches. To address this problem, we propose an adaptive low-rank representation model for tensor completion that represents low-rank and non-low-rank structures of a latent tensor separately in a Bayesian framework. Specifically, we reformulate the CANDECOMP/PARAFAC (CP) tensor rank and develop a sparsity-induced prior for the low-rank structure that can be used to determine tensor rank automatically. Then, the non-low-rank structure is modeled using a mixture of Gaussians prior that is shown to be sufficiently flexible and powerful to inform the completion process for a variety of real tensor data. With these two priors, we develop a Bayesian minimum mean-squared error estimate framework for inference. The developed framework can capture the important distinctions between low-rank and non-low-rank structures, thereby enabling more accurate model, and ultimately, completion. For various applications, compared with the state-of-the-art methods, the proposed model yields more accurate completion results.

Accurate Recovery of Missing Network Measurement Data With Localized Tensor Completion

The inference of the network traffic data from partial measurements data becomes increasingly critical for various network engineering tasks. By exploiting the multi-dimensional data structure, tensor completion is a promising technique for more accurate missing data inference. However, existing tensor completion algorithms generally have the strong assumption that the tensor data have a global low-rank structure, and try to find a single and global model to fit the data of the whole tensor. In a practical network system, a subset of data may have stronger correlation. In this work, we propose a novel localized tensor completion model (LTC) to increase the data recovery accuracy by taking advantage of the stronger local correlation of data to form and recover sub-tensors each with a lower rank. Despite that it is promising to use local tensors, the finding of correlated entries faces two challenges, the data with adjacent indexes are not ones with higher correlation and it is difficult to find the similarity of data with missing tensor entries. To conquer the challenges, we propose several novel techniques: efficiently calculating the candidate anchor points based on locality-sensitive hash (LSH), building sub-tensors around properly selected anchor points, encoding factor matrices to facilitate the finding of similarity with missing entries, and similarity-aware local tensor completion and data fusion. We have done extensive experiments using real traffic traces. Our results demonstrate that LTC is very effective in increasing the tensor recovery accuracy without depending on specific tensor completion algorithms.

Rank minimization on tensor ring: an efficient approach for tensor decomposition and completion

In recent studies, tensor ring decomposition (TRD) has become a promising model for tensor completion. However, TRD suffers from the rank selection problem due to the undetermined multilinear rank. For tensor decomposition with missing entries, the sub-optimal rank selection of traditional methods leads to the overfitting/underfitting problem. In this paper, we first explore the latent space of the TRD and theoretically prove the relationship between the TR-rank and the rank of the tensor unfoldings. Then, we propose two tensor completion models by imposing the different low-rank regularizations on the TR-factors, by which the TR-rank of the underlying tensor is minimized and the low-rank structures of the underlying tensor are exploited. By employing the alternating direction method of multipliers scheme, our algorithms obtain the TR factors and the underlying tensor simultaneously. In experiments of tensor completion tasks, our algorithms show robustness to rank selection and high computation efficiency, in comparison to traditional low-rank approximation algorithms.

Low-rank tensor completion via combined non-local self-similarity and low-rank regularization

Global low-rank methods have achieved great successes in tensor completion. However, these methods neglected the abundant non-local self-similarities, which exist in a wide range of multi-dimensional imaging data. To integrate the global and non-local property of the underlying tensor, we propose a novel low-rank tensor completion model via combined non-local self-similarity and low-rank regularization, which is named as NLS-LR. We adopt the parallel low-rank matrix factorization to guarantee the global low-rankness while plugging in non-local based denoisers to promote the non-local self-similarity instead of tailoring regularizers. To tackle the proposed model, we develop an efficient block successive upper-bound minimization (BSUM) based algorithm. Numerical experiment results demonstrate that the proposed method outperforms many state-of-the-art tensor completion methods in terms of quality metrics and visual effects.

Low-rank tensor completion via smooth matrix factorization

• The proposed model exploits the low-rankness prior by matrix factorization . • The proposed model exploits the piecewise smoothness prior by introducing smoothness constraints on the factor matrices . • An efficient block successive upper-bound minimization based algorithm is developed to solve the proposed model. • Extensive experimental results demonstrate the superiority of the proposed model over the compared ones. Low-rank modeling has achieved great success in tensor completion. However, the low-rank prior is not sufficient for the recovery of the underlying tensor, especially when the sampling rate (SR) is extremely low. Fortunately, many real world data exhibit the piecewise smoothness prior along both the spatial and the third modes (e.g., the temporal mode in video data and the spectral mode in hyperspectral data). Motivated by this observation, we propose a novel low-rank tensor completion model using smooth matrix factorization (SMF-LRTC), which exploits the piecewise smoothness prior along all modes of the underlying tensor by introducing smoothness constraints on the factor matrices. An efficient block successive upper-bound minimization (BSUM)-based algorithm is developed to solve the proposed model. The developed algorithm converges to the set of the coordinate-wise minimizers under some mild conditions. Extensive experimental results demonstrate the superiority of the proposed method over the compared ones.