High-order tensor estimation via trains of coupled third-order CP and Tucker decompositions

Abstract

In this work, equivalence relations between a Tensor Train (TT) decomposition and the Canonical Polyadic Decomposition (CPD)/Tucker Decomposition (TD) are investigated. It is shown that a Q-order tensor following a CPD/TD with Q>3 can be written using the graph-based formalism as a train of Q tensors of order at most 3 following the same decomposition as the initial Q-order tensor. This means that for any practical problem of interest involving the CPD/TD, there exists an equivalent TT-based formulation. This equivalence allows us to overcome the curse of dimensionality when dealing with the big data tensors. In this paper, it is shown that the native difficult optimization problems for CPD/TD of Q-order tensors can be efficiently solved using the TT decomposition according to flexible strategies that involve Q−2 optimization problems with 3-order tensors. This methodology hence involves a number of free parameters linear with Q, and thus allows to mitigate the exponential growth of parameters for Q-order tensors. Then, by capitalizing on the TT decomposition, we also formulate several robust and fast algorithms to accomplish Joint dImensionality Reduction And Factors rEtrieval (JIRAFE) for the CPD/TD. In particular, based on the TT-SVD algorithm, we show how to exploit the existing coupling between two successive TT-cores in the graph-based formalism. The advantages of the proposed solutions in terms of storage cost, computational complexity and factor estimation accuracy are also discussed.