Tucker Rank Research Articles

Nonnegative low multi‐rank third‐order tensor approximation via transformation

AbstractThe main aim of this paper is to develop a new algorithm for computing a nonnegative low multi‐rank tensor approximation for a nonnegative tensor. In the literature, there are several nonnegative tensor factorizations or decompositions, and their approaches are to enforce the nonnegativity constraints in the factors of tensor factorizations or decompositions. In this paper, we study nonnegativity constraints in tensor entries directly, and a low rank approximation for the transformed tensor by using discrete Fourier transformation matrix, discrete cosine transformation matrix, or unitary transformation matrix. This strategy is particularly useful in imaging science as nonnegative pixels appear in tensor entries and a low rank structure can be obtained in the transformation domain. We propose an alternating projections algorithm for computing such a nonnegative low multi‐rank tensor approximation. The convergence of the proposed projection method is established. Numerical examples for multidimensional images are presented to demonstrate that the performance of the proposed method is better than that of nonnegative low Tucker rank tensor approximation and the other nonnegative tensor factorizations and decompositions.

Balanced Unfolding Induced Tensor Nuclear Norms for High-Order Tensor Completion.

The recently proposed tensor tubal rank has been witnessed to obtain extraordinary success in real-world tensor data completion. However, existing works usually fix the transform orientation along the third mode and may fail to turn multidimensional low-tubal-rank structure into account. To alleviate these bottlenecks, we introduce two unfolding induced tensor nuclear norms (TNNs) for the tensor completion (TC) problem, which naturally extends tensor tubal rank to high-order data. Specifically, we show how multidimensional low-tubal-rank structure can be captured by utilizing a novel balanced unfolding strategy, upon which two TNNs, namely, overlapped TNN (OTNN) and latent TNN (LTNN), are developed. We also show the immediate relationship between the tubal rank of unfolding tensor and the existing tensor network (TN) rank, e.g., CANDECOMP/PARAFAC (CP) rank, Tucker rank, and tensor ring (TR) rank, to demonstrate its efficiency and practicality. Two efficient TC models are then proposed with theoretical guarantees by analyzing a unified nonasymptotic upper bound. To solve optimization problems, we develop two alternating direction methods of multipliers (ADMM) based algorithms. The proposed models have been demonstrated to exhibit superior performance based on experimental findings involving synthetic and real-world tensors, including facial images, light field images, and video sequences.

Accurate regularized Tucker decomposition for image restoration

We propose a new accurate regularized Tucker decomposition (ARTD) method for image restoration (IR), which considers global low-rankness and local similarity of intrinsic image characteristics. Specifically, global low-rankness is represented by a sparse Tucker core tensor, whereas the local similarity is captured using nonnegative factor matrices and manifold regularization terms. Sparse nonnegative Tucker decomposition (SNTD) and graph nonnegative Tucker decomposition (GNTD) can be considered a special case of ARTD. We propose and implement an effective Alternating Proximal Gradient (APG) based algorithm to solve the ARTD model and deduce the closed-form updating rules. Notably, ARTD does not need to tune the Tucker rank and provides an initialization strategy and a first-order feedback control rule to accelerate its convergence. Experiments on real IR problems show that our method outperforms some existing state-of-the-art methods.

Multilinear multitask learning by transformed tensor singular value decomposition

In this paper, we study the problem of multilinear multitask learning (MLMTL), in which all tasks are stacked into a third-order tensor for consideration. In contrast to conventional multitask learning, MLMTL can explore inherent correlations among multiple tasks in a better manner by utilizing multilinear low rank structure. Existing approaches about MLMTL are mainly based on the sum of singular values for approximating low rank matrices obtained by matricizing the third-order tensor. However, these methods are suboptimal in the Tucker rank approximation. In order to elucidate intrinsic correlations among multiple tasks, we present a new approach by the use of transformed tensor nuclear norm (TTNN) constraint in the objective function. The main advantage of the proposed approach is that it can acquire a low transformed multi-rank structure in a transformed tensor by applying suitable unitary transformations which is helpful to determine principal components in grouping multiple tasks for describing their intrinsic correlations more precisely. Furthermore, we establish an excess risk bound of the minimizer of the proposed TTNN approach. Experimental results including synthetic problems and real-world images, show that the mean-square errors of the proposed method is lower than those of the existing methods for different number of tasks and training samples in MLMTL.

Algorithm 1036: ATC, An Advanced Tucker Compression Library for Multidimensional Data

We present ATC, a C++ library for advanced Tucker-based lossy compression of dense multidimensional numerical data in a shared-memory parallel setting, based on the sequentially truncated higher-order singular value decomposition (ST-HOSVD) and bit plane truncation. Several techniques are proposed to improve speed, memory usage, error control and compression rate. First, a hybrid truncation scheme is described which combines Tucker rank truncation and TTHRESH quantization. We derive a novel expression to approximate the error of truncated Tucker decompositions in the case of core and factor perturbations. We parallelize the quantization and encoding scheme and adjust this phase to improve error control. Implementation aspects are described, such as an ST-HOSVD procedure using only a single transposition. We also discuss several usability features of ATC, including the presence of multiple interfaces, extensive data type support, and integrated downsampling of the decompressed data. Numerical results show that ATC maintains state-of-the-art Tucker compression rates while providing average speed-up factors of 2.2 to 3.5 and halving memory usage. Our compressor provides precise error control, deviating only 1.4% from the requested error on average. Finally, ATC often achieves higher compression than non-Tucker-based compressors in the high-error domain.

Hybrid Low-Rank Tensor CP and Tucker Decomposition with Total Variation Regularization for HSI Noise Removal

The acquired hyperspectral images (HSIs) are affected by a mixture of several types of noise, which often suffer from information missing. Corrupted HSIs limit the precision of the subsequent processing. In this paper, the weighted Total Variation-regularized Hybrid Model of CP and Tucker (WTV-HMCT) is proposed to accurately identify the intrinsic structures of the clean HSIs. By jointly minimizing CP rank and Tucker rank in the low-rank tensor approximation, WTV-HMCT fully exploits the high-dimensional structure correlations of HSI. To ensure the piecewise smoothness of the recovered image, the hybrid low-rank tensor decomposition approach integrates the weighted spatial spectral total variation regularization for the separation of the noise-free HSI and mixed noise. By the Alternating Direction Method of Multipliers (ADMM), the optimization model is transformed into two subproblems. Finally, an efficient proximal alternating minimization algorithm is developed to optimize the proposed hybrid low-rank tensor decomposition efficiently. The experimental results show that the proposed model effectively handles Gauss noise, striping noise, and mixed noise and that it outperforms the most advanced methods in terms of evaluation metrics and visual evaluation.

A new nonconvex low-rank tensor approximation method with applications to hyperspectral images denoising

Hyperspectral images (HSIs) are frequently corrupted by mixing noise during their acquisition and transmission. Such complicated noise may reduce the quality of the obtained HSIs and limit the accuracy of the subsequent processing. By using the low-rank prior of the tensor formed by spatial and spectral information and further exploring the intrinsic structure of the underlying HSI from noisy observations, in this paper, we propose a new nonconvex low-rank tensor approximation method including optimization model and efficient iterative algorithm to eliminate multiple types of noise. The proposed mathematical model consists of a nonconvex low-rank regularization term using the γ nuclear norm, which is nonconvex surrogate to Tucker rank, and two data fidelity terms representing sparse and Gaussian noise components, which are regularized by the -norm and the Frobenius norm, respectively. To solve this model, we propose an efficient augmented Lagrange multiplier algorithm. We also study the convergence and parameter setting of the algorithm. Extensive experimental results show that the proposed method has better denoising performance than the state-of-the-art competing methods for low-rank tensor approximation and noise modeling.

Tensor Robust Principal Component Analysis via Tensor Fibered Rank and \({\boldsymbol{{l_p}}}\) Minimization

Tensor robust principal component analysis (TRPCA) is an important method to handle high-dimensional data and has been widely used in many areas. In this paper, we mainly focus on the TRPCA problem based on tensor fibered rank for sparse noise removal, which aims to recover the low-fibered-rank tensor from grossly corrupted observations. Usually, the -norm is used as a convex approximation of tensor rank, but it is essentially biased and fails to achieve the best estimation performance. Therefore, we first propose a novel nonconvex model named , in which the norm is adopted to approximate tensor fibered rank and measure sparsity. Then, an error bound of the estimator of is established and this error bound can be better than those of similar models based on Tucker rank or tubal rank. Further, we use the alternating direction method of multipliers to solve and provide convergence guarantee. Finally, extensive experiments on color images, videos, and hyperspectral images demonstrate the effectiveness of the proposed method.

Editorial: Tensor numerical methods and their application in scientific computing and data science

Editorial: Tensor numerical methods and their application in scientific computing and data science

Low Tucker rank tensor completion using a symmetric block coordinate descent method

AbstractLow Tucker rank tensor completion has wide applications in science and engineering. Many existing approaches dealt with the Tucker rank by unfolding matrix rank. However, unfolding a tensor to a matrix would destroy the data's original multi‐way structure, resulting in vital information loss and degraded performance. In this article, we establish a relationship between the Tucker ranks and the ranks of the factor matrices in Tucker decomposition. Then, we reformulate the low Tucker rank tensor completion problem as a multilinear low rank matrix completion problem. For the reformulated problem, a symmetric block coordinate descent method is customized. For each matrix rank minimization subproblem, the classical truncated nuclear norm minimization is adopted. Furthermore, temporal characteristics in image and video data are introduced to such a model, which benefits the performance of the method. Numerical simulations illustrate the efficiency of our proposed models and methods.

Iterative hard thresholding for low CP-rank tensor models

Recovery of low-rank matrices from a small number of linear measurements is now well-known to be possible under various model assumptions on the measurements. Such results demonstrate robustness and are backed with provable theoretical guarantees. However, extensions to tensor recovery have only recently began to be studied and developed, despite an abundance of practical tensor applications. Recently, a tensor variant of the Iterative Hard Thresholding method was proposed and theoretical results were obtained that exact guarantee recovery of tensors with low Tucker rank. In this paper, we utilize and prove a similar tensor version of the Restricted Isometry Property (RIP) to extend these results for tensors with low CANDECOMP/PARAFAC (CP) rank. In doing so, we leverage recent results on efficient approximations of CP decompositions that remove the need for challenging assumptions in prior works. We complement our theoretical findings with empirical results that showcase the potential of the approach.

Tensor Manifold with Tucker Rank Constraints

Low-rank tensor approximation plays a crucial role in various tensor analysis tasks ranging from science to engineering applications. There are several important problems facing low-rank tensor approximation. First, the rank of an approximating tensor is given without checking feasibility. Second, even such approximating tensors exist, however, current proposed algorithms cannot provide global optimality guarantees. In this work, we define the low-rank tensor set (LRTS) for Tucker rank which is a union of manifolds of tensors with specific Tucker rank. We propose a procedure to describe LRTS semi-algebraically and characterize the properties of this LRTS, e.g., feasibility of tensors manifold, the equations/inequations size of LRTS, algebraic dimensions, etc. Furthermore, if the cost function for tensor approximation is polynomial type, e.g., Frobenius norm, we propose an algorithm to approximate a given tensor with Tucker rank constraints and prove the global optimality of the proposed algorithm through critical sets determined by the semi-algebraic characterization of LRTS.

A novel non-convex low-rank tensor approximation model for hyperspectral image restoration

Remote sensing hyperspectral images (HSIs) are inevitably corrupted by several types of noise in the process of acquisition and transmission. In this paper, we propose a non-convex low-rank tensor approximation (NonLRTA) model for mixed noise removal, which can estimate the intrinsic structure of the underlying HSI from its noisy observation. The clean HSI component is characterized by the ϵ-norm, which is a non-convex surrogate to Tucker rank. The mixed noise is modeled as the sum of sparse and Gaussian components, which are regularized by the l1-norm and the Frobenius norm, respectively. An efficient augmented Lagrange multiplier (ALM) algorithm is developed to solve the proposed model. Experiments implemented on simulated and real HSIs validate the superiority of the proposed method, as compared to the state-of-the-art matrix-based and tensor-based methods.

Tensor-structured algorithm for reduced-order scaling large-scale Kohn\u2013Sham density functional theory calculations

We present a tensor-structured algorithm for efficient large-scale density functional theory (DFT) calculations by constructing a Tucker tensor basis that is adapted to the Kohn–Sham Hamiltonian and localized in real-space. The proposed approach uses an additive separable approximation to the Kohn–Sham Hamiltonian and an L1 localization technique to generate the 1-D localized functions that constitute the Tucker tensor basis. Numerical results show that the resulting Tucker tensor basis exhibits exponential convergence in the ground-state energy with increasing Tucker rank. Further, the proposed tensor-structured algorithm demonstrated sub-quadratic scaling with system-size for both systems with and without a gap, and involving many thousands of atoms. This reduced-order scaling has also resulted in the proposed approach outperforming plane-wave DFT implementation for systems beyond 2000 electrons.

On the Synergy between Nonconvex Extensions of the Tensor Nuclear Norm for Tensor Recovery

Low-rank tensor recovery has attracted much attention among various tensor recovery approaches. A tensor rank has several definitions, unlike the matrix rank—e.g., the CP rank and the Tucker rank. Many low-rank tensor recovery methods are focused on the Tucker rank. Since the Tucker rank is nonconvex and discontinuous, many relaxations of the Tucker rank have been proposed, e.g., the sum of nuclear norm, weighted tensor nuclear norm, and weighted tensor schatten-p norm. In particular, the weighted tensor schatten-p norm has two parameters, the weight and p, and the sum of nuclear norm and weighted tensor nuclear norm are special cases of these parameters. However, there has been no detailed discussion of whether the effects of the weighting and p are synergistic. In this paper, we propose a novel low-rank tensor completion model using the weighted tensor schatten-p norm to reveal the relationships between the weight and p. To clarify whether complex methods such as the weighted tensor schatten-p norm are necessary, we compare them with a simple method using rank-constrained minimization. It was found that the simple methods did not outperform the complex methods unless the rank of the original tensor could be accurately known. If we can obtain the ideal weight, p=1 is sufficient, although it is necessary to set p<1 when using the weights obtained from observations. These results are consistent with existing reports.

Robust Tensor Recovery with Fiber Outliers for Traffic Events

Event detection is gaining increasing attention in smart cities research. Large-scale mobility data serves as an important tool to uncover the dynamics of urban transportation systems, and more often than not the dataset is incomplete. In this article, we develop a method to detect extreme events in large traffic datasets, and to impute missing data during regular conditions. Specifically, we propose a robust tensor recovery problem to recover low-rank tensors under fiber-sparse corruptions with partial observations, and use it to identify events, and impute missing data under typical conditions. Our approach is scalable to large urban areas, taking full advantage of the spatio-temporal correlations in traffic patterns. We develop an efficient algorithm to solve the tensor recovery problem based on the alternating direction method of multipliers (ADMM) framework. Compared with existing l 1 norm regularized tensor decomposition methods, our algorithm can exactly recover the values of uncorrupted fibers of a low-rank tensor and find the positions of corrupted fibers under mild conditions. Numerical experiments illustrate that our algorithm can achieve exact recovery and outlier detection even with missing data rates as high as 40% under 5% gross corruption, depending on the tensor size and the Tucker rank of the low rank tensor. Finally, we apply our method on a real traffic dataset corresponding to downtown Nashville, TN and successfully detect the events like severe car crashes, construction lane closures, and other large events that cause significant traffic disruptions.

Low-rank quaternion tensor completion for recovering color videos and images

Low-rank quaternion tensor completion method, a novel approach to recovery color videos and images, is proposed in this paper. We respectively reconstruct a color image and a color video as a quaternion matrix (second-order tensor) and a third-order quaternion tensor by encoding the red, green, and blue channel pixel values on the three imaginary parts of a quaternion. Different from some traditional models which treat color pixel as a scalar and represent color channels separately, whereas, during the quaternion-based reconstruction, it is significant that the inherent color structures of color images and color videos can be completely preserved. Under the definition of Tucker rank, the global low-rank prior to quaternion tensor is encoded as the nuclear norm of unfolding quaternion matrices. Then, by applying the ADMM framework, we provide the tensor completion algorithm for any order ( ≥ 2) quaternion tensors, which theoretically can be well used to recover missing entries of any multidimensional data with color structures. Simulation results for color videos and color images recovery show the superior performance and efficiency of the proposed method over some state-of-the-art existing ones.

Low‐rank tensor completion for visual data recovery via the tensor train rank‐1 decomposition

In this study, the authors study the problem of tensor completion, in particular for three-dimensional arrays such as visual data. Previous works have shown that the low-rank constraint can produce impressive performances for tensor completion. These works are often solved by means of Tucker rank. However, Tucker rank does not capture the intrinsic correlation of the tensor entries. Therefore, the authors propose a new proximal operator for the approximation of tensor nuclear norms based on tensor-train rank-1 decomposition via the singular value decomposition. The proximal operator will perform a soft-thresholding operation on tensor singular values. In addition, the low-rank constraint can capture the global structure of data well, but it does not exploit local smooth of visual data. Therefore, they integrate total variation as a regularisation term into low-rank tensor completion. Finally, they use a primal–dual splitting to achieve optimisation. Experimental results have shown that the proposed method, can preserve the multi-dimensional nature inherent in the data, and thus provide superior results over many state-of-the-art tensor completion techniques.

Tensor Denoising Using Low-Rank Tensor Train Decomposition

Exploiting the latent low-rankness of tensors is crucial in tensor denoising. Classically, many methods use the Tucker model to find the low-rank structure of a tensor. Recently, the tensor train (TT) model has drawn wide attention owing to its powerful representation ability, and well-balanced matricization scheme for a tensor, and it has been successfully applied to various problems in signal processing, and machine learning applications. In this letter, we propose a tensor denoising method using the TT singular value decomposition, and information criteria, where we leverage the minimum description length to automatically estimate the TT rank. Furthermore, we establish the relationship between Tucker decomposition, and TT decomposition. In specific, the low Tucker rank of a tensor is the sufficient but unnecessary condition to the low TT rank. It unveils in theory the potential advantages of the TT model in characterizing the latent low-rankness of tensor. Denoising experiments on both synthetic data, and real HSI dataset demonstrate its superiority against Tucker-based methods.

Deterministic and Probabilistic Conditions for Finite Completability of Low-Tucker-Rank Tensor

We investigate the fundamental conditions on the sampling pattern, i.e., locations of the sampled entries, for finite completability of a low-rank tensor given some components of its Tucker rank. In order to find the deterministic necessary and sufficient conditions, we propose an algebraic geometric analysis on the Tucker manifold, which allows us to incorporate multiple rank components in the proposed analysis in contrast with the conventional geometric approaches on the Grassmannian manifold. This analysis characterizes the algebraic independence of a set of polynomials defined based on the sampling pattern, which is closely related to finite completion. Probabilistic conditions are then studied and a lower bound on the sampling probability is given, which guarantees that the proposed deterministic conditions on the sampling patterns for finite completability hold with high probability. Furthermore, using the proposed geometric approach for finite completability, we propose a sufficient condition on the sampling pattern that ensures there exists exactly one completion for the sampled tensor.