Tensor Completion Methods Research Articles

CP decomposition-based algorithms for completion problem of motion capture data

Motion capture (MoCap) technology is an essential tool for recording and analyzing movements of objects or humans. However, MoCap systems frequently encounter the challenge of missing data, stemming from mismatched markers, occlusion, or equipment limitations. Recovery of these missing data is imperative to maintain the reliability and integrity of MoCap recordings. This paper introduces a novel application of the tensor framework for MoCap data completion. We propose three completion algorithms based on the canonical polyadic (CP) decomposition of tensors. The first algorithm utilizes CP decomposition to capture the low-rank structure of the tensor. However, relying only on low-rank assumptions may be insufficient to deal with complex motion data. Thus, we propose two modified CP decompositions that incorporate additional information, SmoothCP and SparseCP decompositions. SmoothCP integrates piecewise smoothness prior, while SparseCP incorporates sparsity prior, each aiming to improve the accuracy and robustness of MoCap data recovery. To compare and evaluate the merit of the proposed algorithms over other tensor completion methods in terms of several evaluation metrics, we conduct numerical experiments with different MoCap sequences from the CMU motion capture dataset.

“Sparse + Low-Rank” tensor completion approach for recovering images and videos

Recovering color images and videos from highly undersampled data is a fundamental and challenging task in face recognition and computer vision. By the multi-dimensional nature of color images and videos, in this paper, we propose a novel tensor completion approach, which is able to efficiently explore the sparsity of tensor data under the discrete cosine transform (DCT). Specifically, we introduce two “sparse + low-rank” tensor completion models as well as two implementable algorithms for finding their solutions. The first one is a DCT-based sparse plus weighted nuclear norm induced low-rank minimization model. The second one is a DCT-based sparse plus p-shrinking mapping induced low-rank optimization model. Moreover, we accordingly propose two implementable augmented Lagrangian-based algorithms for solving the underlying optimization models. A series of numerical experiments including color image inpainting and video data recovery demonstrate that our proposed approach performs better than many existing state-of-the-art tensor completion methods, especially for the case when the ratio of missing data is high.

Smooth Tensor Product for Tensor Completion.

Low-rank tensor completion (LRTC) has shown promise in processing incomplete visual data, yet it often overlooks the inherent local smooth structures in images and videos. Recent advances in LRTC, integrating total variation regularization to capitalize on the local smoothness, have yielded notable improvements. Nonetheless, these methods are limited to exploiting local smoothness within the original data space, neglecting the latent factor space of tensors. More seriously, there is a lack of theoretical backing for the role of local smoothness in enhancing recovery performance. In response, this paper introduces an innovative tensor completion model that concurrently leverages the global low-rank structure of the original tensor and the local smooth structure of its factor tensors. Our objective is to learn a low-rank tensor that decomposes into two factor tensors, each exhibiting sufficient local smoothness. We propose an efficient alternating direction method of multipliers to optimize our model. Further, we establish generalization error bounds for smooth factor-based tensor completion methods across various decomposition frameworks. These bounds are significantly tighter than existing baselines. We conduct extensive inpainting experiments on color images, multispectral images, and videos, which demonstrate the efficacy and superiority of our method. Additionally, our approach shows a low sensitivity to hyper-parameter settings, enhancing its convenience and reliability for practical applications.

Tensor completion via convolutional sparse coding with small samples-based training

Tensor data often suffer from missing value problems due to the complex high-dimensional structure while acquiring them. To complete the missing information, lots of Low-Rank Tensor Completion (LRTC) methods have been proposed, most of which depend on the low-rank property of tensor data. In this way, the low-rank component of the original data could be recovered roughly. However, the shortcoming is that the detailed information can not be fully restored, no matter the Sum of the Nuclear Norm (SNN) nor the Tensor Nuclear Norm (TNN) based methods. On the contrary, in the field of signal processing, Convolutional Sparse Coding (CSC) can provide a good representation of the high-frequency component of the image, which is generally associated with the detail component of the data. To this end, we propose two novel methods, LRTC-CSC-I and LRTC-CSC-II, which adopt CSC as a supplementary regularization for LRTC to capture the high-frequency components. Therefore, the LRTC-CSC methods can not only solve the missing value problem but also recover the details. Moreover, the regularizer CSC can be trained with small samples due to the sparsity characteristic. Extensive experiments show the effectiveness of LRTC-CSC methods, and quantitative evaluation indicates that the performance of our models are superior to state-of-the-art methods.

Predicting Temperature‐Dependent Activity Coefficients at Infinite Dilution Using Tensor Completion

AbstractKnowledge of thermodynamic properties of mixtures is essential in many fields of science and engineering. However, the experimental data is usually scarce, so prediction methods are needed. Matrix completion methods have proven to be very successful in predicting thermodynamic properties of binary mixtures. In this approach, the experimental data is organized in a matrix whose rows and columns correspond to the two components, and whose entries indicate the value of the studied thermodynamic property at fixed conditions. In the present work, we extend the concept to tensor completion methods (TCMs). This allows to account for the variation of the studied property depending on the chosen conditions. The feasibility is demonstrated by applying a TCM to predict activity coefficients at infinite dilution. The third dimension of the tensor is used to describe the influence of the temperature. The TCM is shown to yield better predictions than the well‐established UNIFAC method. Furthermore, the proposed TCM is able to learn and unveil the physical law describing the temperature dependence of activity coefficients from the scarce experimental mixture data only.

Two new low rank tensor completion methods based on sum nuclear norm

The low rank tensor completion (LRTC) problem has attracted great attention in computer vision and signal processing. How to acquire high quality image recovery effect is still an urgent task to be solved at present. This paper firstly proposes a new tensor L2,1 norm minimization model (TLNM) that integrates sum nuclear norm (SNN) method, differing from the classical tensor nuclear norm (TNN)-based tensor completion method, with L2,1 norm and QR decomposition for solving the LRTC problem. To improve the utilization rate of the local prior information of the image, a total variation (TV) regularization term is introduced, resulting in another new class of tensor L2,1 norm minimization with total variation model (TLNMTV). Both proposed models are convex and therefore have global optimal solutions. Moreover, we adopt the Alternating Direction Multiplier Method (ADMM) to obtain the closed-form solution of each variable, which makes the algorithm be able to be efficiently implemented. Numerical experiments show that the two proposed algorithms are convergent and outperform compared methods. In particular, our method significantly outperforms the compared methods when the sampling rate is 2.5% for hyperspectral images.

Dictionary Learning With Low-Rank Coding Coefficients for Tensor Completion.

In this article, we propose a novel tensor learning and coding model for third-order data completion. The aim of our model is to learn a data-adaptive dictionary from given observations and determine the coding coefficients of third-order tensor tubes. In the completion process, we minimize the low-rankness of each tensor slice containing the coding coefficients. By comparison with the traditional predefined transform basis, the advantages of the proposed model are that: 1) the dictionary can be learned based on the given data observations so that the basis can be more adaptively and accurately constructed and 2) the low-rankness of the coding coefficients can allow the linear combination of dictionary features more effectively. Also we develop a multiblock proximal alternating minimization algorithm for solving such tensor learning and coding model and show that the sequence generated by the algorithm can globally converge to a critical point. Extensive experimental results for real datasets such as videos, hyperspectral images, and traffic data are reported to demonstrate these advantages and show that the performance of the proposed tensor learning and coding method is significantly better than the other tensor completion methods in terms of several evaluation metrics.

Jointly Low-Rank Tensor Completion for Estimating Missing Spatiotemporal Values in Logistics Systems

With the deepening of industry 4.0 paradigm in logistics systems, artificial intelligent has been widely used to improve the quality of logistics services. Considering that data collected in comprehensive logistics service system usually integrate multistage complex information such as traffic flow records and spatiotemporal trajectory, it is inevitable that the data are incomplete and partially missing due to equipment failures, communication interruptions, etc. As an effective spatiotemporal completion tool in logistics systems, low-rank tensor completion has aroused extensive research interest thanks to its excellent performance on data recovery. Although existing tensor completion methods effectively capture the complex associations/dependencies of multidimensional inputs, they fail to exploit the potential characteristics of spatiotemporal data, such as the periodicity. In this article, we propose a jointly low-rank tensor completion method for logistics data completion, which constructs multiple periodic subtensors by setting an appropriate time window, then performs jointly low-rank completion and imputation. In addition, we also provide an optimization algorithm based on Alternating Direction Method of Multiplier framework for the proposed problem. Experimental results on four logistics-related datasets have further demonstrated the promising performance of the proposed method compared with other state-of-the-art competitors. We believe that the proposed approach not only effectively maintains the advantages of classical completion methods, but also fully excavates the multidimensional correlation and hidden patterns behind records, and further provides a novel and effective strategy for data completion and imputation in logistics systems.

Rank-Adaptive Tensor Completion Based on Tucker Decomposition

Tensor completion is a fundamental tool to estimate unknown information from observed data, which is widely used in many areas, including image and video recovery, traffic data completion and the multi-input multi-output problems in information theory. Based on Tucker decomposition, this paper proposes a new algorithm to complete tensors with missing data. In decomposition-based tensor completion methods, underestimation or overestimation of tensor ranks can lead to inaccurate results. To tackle this problem, we design an alternative iterating method that breaks the original problem into several matrix completion subproblems and adaptively adjusts the multilinear rank of the model during optimization procedures. Through numerical experiments on synthetic data and authentic images, we show that the proposed method can effectively estimate the tensor ranks and predict the missing entries.

DOA Estimation via Tensor Truncated Convolution Nuclear Norm Minimization in MIMO Radar Under Array Sensor Failures

Exploring the inherent multidimensional structures has great potential to obtain more latent information for reconstructing the missing data resulting from array sensor failures in multiple-input–multiple-output (MIMO) radar. Most tensor completion (TC) methods based on the minimization of tensor nuclear norms are adopted to fill randomly distributed missing entries. However, the randomness of missing entries is violated in MIMO radar under sensor failures, where some slices are missing entirely. Thus, these existing methods become ineffective when faced with such type of structurally missing entries. To remedy this, we propose an improved TC method based on tensor truncated convolution nuclear norm minimization (CNNM) to reconstruct the failed sensor data in MIMO radar in order to handle sensor failure in direction-of-arrival (DOA) estimation. First, we formulate a new tensor by convoluting the third-order tensor of MIMO radar with a kernel tensor in a truncated manner and establish a low-rank TC model equipped with the truncated CNNM. Then, we introduce a tractable relaxation of our minimization function by deducing the equality relation between the truncated convolution nuclear norm of a tensor and the nuclear norm of the truncated convolution matrix of the same tensor. Finally, we derive an efficient algorithm implementation based on alternating direction method of multipliers (ADMM) to tackle our problem. Simulation results show that the proposed method is quite effective in completing tensors with structurally missing entries and facilitating more accurate data recovery by exploiting the inherent multidimensional nature. This provides superior DOA estimation performance in MIMO radar.

Patch Tracking-Based Streaming Tensor Ring Completion for Visual Data Recovery

Tensor completion aims to recover the missing entries of a partially observed tensor by exploiting its low-rank structure, and has been applied to visual data recovery. In applications where the data arrives sequentially such as streaming video completion, the missing entries of the tensor need to be dynamically recovered in a streaming fashion. Traditional streaming tensor completion algorithms treat the entire visual data as a tensor, which may not work satisfactorily when there is a big change in the tensor subspace along the temporal dimension, such as due to strong motion across the video frames. In this paper, we develop a novel patch tracking-based streaming tensor ring completion framework for visual data recovery. Given a newly incoming frame, small patches are tracked from the previous frame. Meanwhile, for each tracked patch, a patch tensor is constructed by stacking similar patches from the new frame. Patch tensors are then completed using a streaming tensor ring completion algorithm, and the incoming frame is recovered using the completed patch tensors. We propose a new patch tracking strategy that can accurately and efficiently track the patches with missing data. Further, a new streaming tensor ring completion algorithm is proposed which can efficiently and accurately update the latent core tensors and complete the missing entries of the patch tensors. Extensive experimental results demonstrate the superior performance of the proposed algorithms compared with both batch and streaming state-of-the-art tensor completion methods.

Effective Tensor Completion via Element-Wise Weighted Low-Rank Tensor Train With Overlapping Ket Augmentation

Tensor completion methods based on the tensor train (TT) have the issues of inaccurate weight assignment and ineffective tensor augmentation pre-processing. In this work, we propose a novel tensor completion approach via the element-wise weighted technique. Accordingly, a novel formulation for tensor completion and an effective optimization algorithm, called tensor completion by parallel weighted matrix factorization via tensor train (TWMac-TT), is proposed. In addition, we specifically consider the recovery quality of edge elements from adjacent blocks. Different from traditional reshaping and ket augmentation, we utilize a new tensor augmentation technique called overlapping ket augmentation, which can further avoid blocking artifacts. We then conduct extensive performance evaluations on synthetic data and several real image data sets. Our experimental results demonstrate that the proposed algorithm TWMac-TT outperforms several other competing tensor completion methods. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yzcv/</uri> TWMac-TT-OKA

Imbalanced low-rank tensor completion via latent matrix factorization

Tensor completion has been widely used in computer vision and machine learning. Most existing tensor completion methods empirically assume the intrinsic tensor is simultaneous low-rank in all over modes. However, tensor data recorded from real-world applications may conflict with these assumptions, e.g., face images taken from different subjects often lie in a union of low-rank subspaces, which may result in a quite high rank or even full rank structure in its sample mode. To this aim, in this paper, we propose an imbalanced low-rank tensor completion method, which can flexibly estimate the low-rank incomplete tensor via decomposing it into a mixture of multiple latent tensor ring (TR) rank components. Specifically, each latent component is approximated using low-rank matrix factorization based on TR unfolding matrix. In addition, an effective proximal alternating minimization algorithm is developed and theoretically proved to maintain the global convergence property, that is, the whole sequence of iterates is convergent and converges to a critical point. Extensive experiments on both synthetic and real-world tensor data demonstrate that the proposed method achieves more favorable completion results with less computational cost when compared to the state-of-the-art tensor completion methods.

Distributed-memory tensor completion for generalized loss functions in python using new sparse tensor kernels

Tensor computations are increasingly prevalent numerical techniques in data science, but pose unique challenges for high-performance implementation. We provide novel algorithms and systems infrastructure which enable efficient parallel implementation of algorithms for tensor completion with generalized loss functions. Specifically, we consider alternating minimization, coordinate minimization, and a quasi-Newton (generalized Gauss-Newton) method. By extending the Cyclops library, we implement all of these methods in high-level Python syntax. To make possible tensor completion for very sparse tensors, we introduce new multi-tensor primitives, for which we provide specialized parallel implementations. We compare these routines to pairwise contraction of sparse tensors by reduction to hypersparse matrix formats, and find that the multi-tensor routines are more efficient in theoretical cost and execution time in experiments. We provide microbenchmarking results on the Stampede2 supercomputer to demonstrate the efficiency of the new primitives and Cyclops functionality. We then study the performance of the tensor completion methods for a synthetic tensor with 10 billion nonzeros and the Netflix dataset, considering both least squares and Poisson loss functions.

Contour information regularized tensor ring completion for realistic image restoration

Tensor completion has gained considerable research interest in recent years and has been frequently applied to image restoration. This type of method basically employs the low-rank nature of images, implicitly requiring that the whole picture is of globally consistent features. As a result, existing tensor completion algorithms often give reasonably good performance if the target image has only random pixel-level missing. Unfortunately, pixel-level missing is very rare in practice and it is often wanted to restore an image with irregular hole-shaped missing, such as removing electricity poles from landscape photos or irrelevant people from tourist photos. This task is extremely difficult for traditional low-rank based tensor completion methods. To overcome this drawback, a Contour Information regularized Tensor RIng Completion (CITRIC) method is proposed for practical image restoration. Meanwhile, the contour information regularization is used to capture significant local features, whereas the low-rank tensor ring structure is utilized to capture as much global information as possible. The alternating direction method of multipliers (ADMM) is adopted to optimize the cost function. Extensive experimental results using real-world images show that CITRIC is more practical than existing methods and can restore real-world images with irregular hole-shaped missing.

An Efficient Transform based Low Rank Tensor Completion to Extreme Visual Recovery

Objective: To propose an optimization approach in recovering of the corrupted tensors in the high dimensional real time data. Methods: The recovering of corrupted tensors is performed by low-rank tensor completion methods. The tensor decomposition methods are used in tensor completion methods. These Tensor decomposition methods; candecomp / parafac (CP), tucker and higher-order Singular Value Decomposition (HoSVD) are used to minimize the rank of a tensor data. The limitations are in finding the rank of a tensor. Findings: The recovered data using the lifting transform induced tensor- Singular Value Decomposition (t-SVD) technique were assessed utilizing the Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), Naturalness Image Quality Evaluator (NIQE), and Perceptual Image Quality Evaluator (PIQE). When compared to state-of-the-art approaches, the low rank assumption condition with the lifting transform consideration gave good data recovery for every missing ratio. Novelty: The missing data is calculated by lifting polyphase structures by utilizing the available data. The polyphase structures are splitting the value into equivalent multiple triangular matrices, these are processed with the t-SVD to have the better approximation tensor rank. Keywords: Tensor Completion; Transformbased Optimization; 5/3 Lifting Wavelet Transform; Lowrank tensor completion; tSVD

Shuffled rolling shutter for snapshot temporal imaging.

We propose a modification to the rolling shutter mechanism found in CMOS detectors by shuffling the pixels in every scanline. This potential hardware modification improves the sampling of the space-time datacube, allowing the recovery of high-speed videos from a single image using either tensor completion methods or reconstruction algorithms often used for compressive temporal video. We also present a design methodology for optimal sampling schemes and compare them to random shuffling. Simulations, and experimental results obtained by optically emulating the hardware, demonstrate the ability of the shuffled rolling shutter to capture images that allow reconstructing videos, which would otherwise be impossible when using the traditional rolling shutter mechanism.

Complex Video Completion Fusing Low-Rank Background and Deep Foreground Priors

Recently, low-rank prior (LRP) has achieved success in tensor completion, thanks to its ability in capturing global correlations. However, since the presence of dynamic foreground breaks the low-rank assumption, LRP sometimes fails to work for complex videos. How to effectively capture the implicit low-rankness is a tricky challenge in complex video completion. To address this challenge, we propose a video completion model, which organically combines the respective merits of deterministic LRP and deep image prior. Specifically, we first decompose a complex video into the dynamic foreground and static background, and then characterize each component individually. For the dynamic foreground, instead of the hand-crafted prior, we consider the deep foreground prior expressed by U-Net architecture in an unsupervised manner, while we consider the tensor low-rank prior for the static background. Moreover, we develop an efficient alternating direction method of multipliers algorithm under the plug-and-play framework to tackle the resulting model. Different from the classic plug-and-play framework with a pre-trained and fixed network, our algorithm can adaptively update the parameters of network during iterations, which enables the network to have a more flexible expression ability. Experimental results verify that the proposed method can do better than some state-of-the-art tensor completion methods in complex video completion.

Trainable Subspaces for Low Rank Tensor Completion: Model and Analysis

With the help of auxiliary data, tensor completion may better recover a low rank multidimensional array from limited observation entries. Most existing methods, including coupled matrix-tensor factorization and coupled tensor rank minimization, mainly focus on how to extract and incorporate subspace or directly use auxiliary data for tensor completion. They are either sensitive to a given rank or lack of physical interpretations of subspace information. In addition, the shared subspace information receives little attention in current tensor completion methods, especially there is no analysis of its impact on sample complexity. In this paper, we propose to extract shared subspace information from auxiliary data and incorporate the learned subspace for tensor completion. Specifically, dictionary learning is employed to get the useful subspace from auxiliary data in the first step. Then an optimization model for low rank tensor completion incorporates the learned subspace by assuming that the recovered tensor is composed of two low rank components, where one shares the subspace information with auxiliary data and the other is outside the shared space. Based on this optimization model, we make a quantitative analysis to illustrate the effect of subspace information on sample complexity, and provide theoretical insights into the usefulness of subspace information. Finally, experiments on simulated data are conducted to validate the theoretical analysis on the impact of subspace information. Experiments in two real-world applications including color image and multispectral image recovery show that the proposed method outperforms state-of-the-art ones in terms of prediction accuracy and CPU time.

Low-rank tensor completion using nonconvex total variation

In this work, we study the tensor completion problem in which the main point is to predict the missing values in visual data. To greatly benefit from the smoothness structure and edge-preserving property in visual images, we suggest a tensor completion model that seeks gradient sparsity via the l0-norm. The proposal combines the low-rank matrix factorization which guarantees the low-rankness property and the nonconvex total variation (TV). We present several experiments to demonstrate the performance of our model compared with popular tensor completion methods in terms of visual and quantitative measures.