Order Tensor Research Articles

Tensor Principal Component Analysis in High Dimensional CP Models

The CP decomposition for high dimensional non-orthogonal spiked tensors is an important problem with broad applications across many disciplines. However, previous works with theoretical guarantee typically assume restrictive incoherence conditions on the basis vectors for the CP components. In this paper, we propose new computationally efficient composite PCA and concurrent orthogonalization algorithms for tensor CP decomposition with theoretical guarantees under mild incoherence conditions. The composite PCA applies the principal component or singular value decompositions twice, first to a matrix unfolding of the tensor data to obtain singular vectors and then to the matrix folding of the singular vectors obtained in the first step. It can be used as an initialization for any iterative optimization schemes for the tensor CP decomposition. The concurrent orthogonalization algorithm iteratively estimates the basis vector in each mode of the tensor by simultaneously applying projections to the orthogonal complements of the spaces generated by other CP components in other modes. It is designed to improve the alternating least squares estimator and other forms of the high order orthogonal iteration for tensors with low or moderately high CP ranks, and it is guaranteed to have second or higher order convergence when the error of any given initial estimator is bounded by a small constant. Our theoretical investigation provides estimation accuracy and convergence rates for the two proposed algorithms. Both proposed algorithms are applicable to deterministic tensor, its noisy version, and the order- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2K$ </tex-math></inline-formula> covariance tensor of order- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> tensor data in a factor model with uncorrelated factors. Simulation experiments demonstrate significant practical superiority of our approach over existing methods.

On the Phase Space of Fourth-Order Fiber-Orientation Tensors

Fiber-orientation tensors describe the relevant features of the fiber-orientation distribution compactly and are thus ubiquitous in injection-molding simulations and subsequent mechanical analyses. In engineering applications to date, the second-order fiber-orientation tensor is the basic quantity of interest, and the fourth-order fiber-orientation tensor is obtained via a closure approximation. Unfortunately, such a description limits the predictive capabilities of the modeling process significantly, because the wealth of possible fourth-order fiber-orientation tensors is not exploited by such closures, and the restriction to second-order fiber-orientation tensors implies artifacts. Closures based on the second-order fiber-orientation tensor face a fundamental problem – which fourth-order fiber-orientation tensors can be realized? In the literature, only necessary conditions for a fiber-orientation tensor to be connected to a fiber-orientation distribution are found. In this article, we show that the typically considered necessary conditions, positive semidefiniteness and a trace condition, are also sufficient for being a fourth-order fiber-orientation tensor in the physically relevant case of two and three spatial dimensions. Moreover, we show that these conditions are not sufficient in higher dimensions. The argument is based on convex duality and a celebrated theorem of D. Hilbert (1888) on the decomposability of positive and homogeneous polynomials of degree four. The result has numerous implications for modeling the flow and the resulting microstructures of fiber-reinforced composites, in particular for the effective elastic constants of such materials. Based on our findings, we show how to connect optimization problems on fourth-order fiber-orientation tensors to semi-definite programming. The proposed formulation permits to encode symmetries of the fiber-orientation tensor naturally. As an application, we look at the differences between orthotropic and general, i.e., triclinic, fiber-orientation tensors of fourth order in two and three spatial dimensions, revealing the severe limitations inherent to orthotropic closure approximations.

Monotone Inclusions, Acceleration, and Closed-Loop Control

We propose and analyze a new dynamical system with a closed-loop control law in a Hilbert space [Formula: see text], aiming to shed light on the acceleration phenomenon for monotone inclusion problems, which unifies a broad class of optimization, saddle point, and variational inequality (VI) problems under a single framework. Given an operator [Formula: see text] that is maximal monotone, we propose a closed-loop control system that is governed by the operator [Formula: see text], where a feedback law [Formula: see text] is tuned by the resolution of the algebraic equation [Formula: see text] for some [Formula: see text]. Our first contribution is to prove the existence and uniqueness of a global solution via the Cauchy–Lipschitz theorem. We present a simple Lyapunov function for establishing the weak convergence of trajectories via the Opial lemma and strong convergence results under additional conditions. We then prove a global ergodic convergence rate of [Formula: see text] in terms of a gap function and a global pointwise convergence rate of [Formula: see text] in terms of a residue function. Local linear convergence is established in terms of a distance function under an error bound condition. Further, we provide an algorithmic framework based on the implicit discretization of our system in a Euclidean setting, generalizing the large-step hybrid proximal extragradient framework. Even though the discrete-time analysis is a simplification and generalization of existing analyses for a bounded domain, it is largely motivated by the aforementioned continuous-time analysis, illustrating the fundamental role that the closed-loop control plays in acceleration in monotone inclusion. A highlight of our analysis is a new result concerning [Formula: see text]-order tensor algorithms for monotone inclusion problems, complementing the recent analysis for saddle point and VI problems. Funding: This work was supported in part by the Mathematical Data Science Program of the Office of Naval Research [Grant N00014-18-1-2764] and by the Vannevar Bush Faculty Fellowship Program [Grant N00014-21-1-2941].

Perturbation bounds for (nearly) orthogonally decomposable tensors with statistical applications

Abstract We develop deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors, in a spirit similar to classical results for matrices such as those due to Weyl, Davis, Kahan and Wedin. Our bounds demonstrate intriguing differences between matrices and higher order tensors. Most notably, they indicate that for higher order tensors perturbation affects each essential singular value/vector in isolation, and its effect on an essential singular vector does not depend on the multiplicity of its corresponding singular value or its distance from other singular values. Our results can be readily applied and provide a unified treatment to many different problems involving higher order orthogonally decomposable tensors. In particular, we illustrate the implications of our bounds through connected yet seemingly different high-dimensional data analysis tasks: the unsupervised learning scenario of tensor SVD and the supervised task of tensor regression, leading to new insights in both of these settings.

The C-eigenvalue of third order tensors and its application in crystals

&lt;p style="text-indent:20px;"&gt;In crystallography, piezoelectric tensors of various crystals play a crucial role in piezoelectric effect and converse piezoelectric effect. Generally, a third order real tensor is called a piezoelectric-type tensor if it is partially symmetric with respect to its last two indices. The piezoelectric tensor is a piezoelectric-type tensor of dimension three. We introduce C-eigenvalues and C-eigenvectors for piezoelectric-type tensors. Here, &amp;quot;C'' names after Curie brothers, who first discovered the piezoelectric effect. We show that C-eigenvalues always exist, they are invariant under orthogonal transformations, and for a piezoelectric-type tensor, the largest C-eigenvalue and its C-eigenvectors form the best rank-one piezoelectric-type approximation of that tensor. This means that for the piezoelectric tensor, its largest C-eigenvalue determines the highest piezoelectric coupling constant. We further show that for the piezoelectric tensor, the largest C-eigenvalue corresponds to the electric displacement vector with the largest 2-norm in the piezoelectric effect under unit uniaxial stress, and the strain tensor with the largest 2-norm in the converse piezoelectric effect under unit electric field vector. Thus, C-eigenvalues and C-eigenvectors have concrete physical meanings in piezoelectric effect and converse piezoelectric effect. Finally, by numerical experiments, we report C-eigenvalues and associated C-eigenvectors for piezoelectric tensors corresponding to several piezoelectric crystals.&lt;/p&gt;

Multiplex Transformed Tensor Decomposition for Multidimensional Image Recovery.

Low-rank tensor completion aims to recover the missing entries of multi-way data, which has become popular and vital in many fields such as signal processing and computer vision. It varies with different tensor decomposition frameworks. Compared with matrix SVD, recently emerging transform t-SVD can better characterize the low-rank structure of order-3 data. However, it suffers from rotation sensitivity, and dimensional limitation (i.e. only effective for order-3 tensors). To alleviate these deficiencies, we develop a novel multiplex transformed tensor decomposition (MTTD) framework, which can characterize the global low-rank structure along all modes for any order-N tensor. Based on MTTD, we propose a related multi-dimensional square model for low-rank tensor completion. Besides, a total variation term is also introduced to utilize the local piecewise smoothness of the tensor data. The classic alternating direction method of multipliers is used to solve the convex optimization problems. For performance testing, we choose three linear invertible transforms including FFT, DCT, and a group of unitary transform matrices for our proposed methods. The simulated and real-data experiments demonstrate the superior recovery accuracy and computational efficiency of our method compared with state-of-the-art ones.

Global rate optimality of integral curve estimators in high order tensor models

Вдохновленные применениями в нейровизуализации, мы рассматриваем проблему установления глобальной минимаксной нижней границы в модели тензора высокого порядка. В частности, описываемая нами методология позволяет получить глобальную минимаксную границу для оценок интегральных кривых, предложенных в работе О. Кармайкла и второго автора 2015 г., при полупараметрической постановке задачи. Теоретические результаты настоящей работы гарантируют, что оценки, используемые для отслеживания сложной структуры волокон внутри живого человеческого мозга и построенные по данным, полученным из диффузионной тензорной МРТ с высоким угловым разрешением (HARDI), оптимальны не только локально, но и глобально. Таким образом, глобальная минимаксная граница асимптотического риска оценок предоставит квантификацию неопределенности для метода оценки во всей области изображения. В дополнение к теоретическим результатам проводится подробное эмпирическое исследование с целью определить оптимальное число градиентных направлений для протоколов нейровизуализации, которые мы далее иллюстрируем анализом сканирования мозга живого человека по реальным данным.

Critical behaviors of Ising model in a fractal lattice

Fractal lattices are a special kind of lattice: they have non-integer Hausdorff dimensions and break the translation invariance. Studying these lattices can help us understand the influence of non-integer dimensions and lacking of translational symmetry on critical behaviors. We study the Ising model in a fractal lattice with a non-integer dimension of &lt;inline-formula&gt;&lt;tex-math id="M1"&gt;\begin{document}$\log_4(12)\approx 1.7925$\end{document}&lt;/tex-math&gt;&lt;alternatives&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="8-20222432_M1.jpg"/&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="8-20222432_M1.png"/&gt;&lt;/alternatives&gt;&lt;/inline-formula&gt;by using the higher order tensor network renormalization group (HOTRG) algorithm. The partition function is represented in terms by a tensor network, and is finally calculated by a coarse graining process based on higher order singular value decomposition. When the truncation length and the time of coarse graining increase, the results are found convergent. Magnetic moment, internal energy and correlation properties are calculated by inserting impurity tensors into the tensor network at different temperatures and in different external magnetic fields. The magnetic susceptibility is obtained by differentiating the magnetic moment with respect to the magnetic field, and the capacity is calculated by differentiating the internal energy with respect to the temperature. Our numerical results show that there is a continuous order-disorder phase transition in this system, and the critical temperature is found to be &lt;inline-formula&gt;&lt;tex-math id="M2"&gt;\begin{document}$T_{\rm{c}}/J = 1.317188$\end{document}&lt;/tex-math&gt;&lt;alternatives&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="8-20222432_M2.jpg"/&gt;&lt;graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="8-20222432_M2.png"/&gt;&lt;/alternatives&gt;&lt;/inline-formula&gt;. Physical quantities show singular behaviours around the critical point, and the correlation length is found to be divergent at the critical point, which is consistent with the result of the renormalization group theory. The corresponding critical exponent is obtained by fitting the numerical data around the critical point. We also calculate the critical exponents at different positions by inserting impurity tensors into different places of the lattice. Owing to the lack of translational symmetry, it is found that the critical exponents &lt;i&gt;α&lt;/i&gt;, &lt;i&gt;β&lt;/i&gt;, &lt;i&gt;δ&lt;/i&gt; fitted at different positions vary, but the critical exponent &lt;i&gt;γ&lt;/i&gt; remains almost the same. From the scaling hypothesis, it can be deduced that the critical exponents satisfy the hyperscaling relations which contain the dimension of the lattice. Our numerical results show that all of the hyperscaling relations are satisfied when the fractional dimension and the critical exponents we have obtained are substituted into them on some sites of the fractal lattice, but only two of the four hyperscaling relations are satisfied on other sites.

Capacity Performance of Tensor Multi-Domain Communication Systems With Discrete Signalling Constellations

Modern communication systems employ multi-domain modulation and coding techniques for effectively exploiting all available resources. Hence in such systems the transmit and receive signals have an inherent multi-domain structure which can be represented using tensors. This work considers the capacity of higher order tensor channels associated with such multi-domain communication systems when the elements of the input tensor are constrained to be drawn from discrete signalling constellations. We establish a relationship between the tensor gradient of the mutual information and the error covariance tensor associated with the minimum mean squared error estimator at the receiver. This relation is used to iteratively find a multi-linear precoder at the input which achieves capacity of the tensor channel under the signalling constellation constraints. Through numerical examples, we show the convergence behavior of the proposed precoder, and compare the capacity achieved under different constellations with the capacity when the input is Gaussian. Further, we exploit the tensor formulation of the problem to find the channel capacity under a variety of different power constraints spanning across several domains. At high SNR, the constellation constraints saturates the capacity while at low SNRs, the constellation constraints are not too relevant, and the power constraints dominate and limit the performance. The capacity saturation level depends on the input order and distribution.

Classification of second orders symmetric tensors on manifolds through an associated fourth order tensor

For a manifold M admitting a metric 1 and given a second order symmetric tensor T on M one can construct from 1 and (the trace-free part of) T a fourth order tensor E on M which is related in a one-to-one way with T and from which T may be readily obtained algebraically. In the case when dimM = 4 this leads to an interesting relationship between the Jordan-Segre algebraic classification of T, viewed as a linear map on the tangent space to M with respect to 1, and the Jordan-Segre classification of E, viewed as a linear map on the 6?dimensional vector space of 2?forms to itself (with respect to the usual metric on 2?forms). This paper explores this relationship for each of the three possible signatures for 1.

Geometrically Non-Linear Dynamic Behavior of Simply Supported Rectangular Plates Carrying a Concentrated Mass

Simply supported plates carrying an added point mass are encountered in many engineering fields, like circuit boards or slabs carrying machines at different locations. Determination of the plate modified dynamic characteristics is a quite laborious task, especially in the non-linear regime, which is rarely treated in the literature. The added mass effect on the plate linear parameters was first examined using Hamilton’s principle and spectral analysis. The modified plate's non-linear fundamental mode was then calculated and its non-linear response to high levels of harmonic excitation was determined. The non-linear formulation, involving a fourth order tensor due to the membrane forces induced in the plate mid-plane by large vibration amplitudes, led to a non-linear algebraic amplitude equation. The iterative solution gave the free vibration case a better qualitative understanding and a quantitative evaluation of the effect of the added mass. The non-linear forced response of the modified plate, examined for a wide frequency range, shows that the added eccentric mass induces changes in the area between the mass location and the simple supports and decreases the non-linear hardening effect. The numerical results, covering new situations, are expected to be useful in engineering applications necessitating for some reason the addition to the plate of a point mass or an adaptation of the plate frequencies in order to avoid the occurrence of undesirable resonances.

Causal Inference for Social and Engineering Systems

What will happen to Y if we do A? A variety of meaningful social and engineering questions can be formulated this way: What will happen to a patient's health if they are given a new therapy? What will happen to a country's economy if policy-makers legislate a new tax? What will happen to a data center's latency if a new congestion control protocol is used? We explore how to answer such counterfactual questions using observational data-which is increasingly available due to digitization and pervasive sensors-and/or very limited experimental data. The two key challenges are: (i) counterfactual prediction in the presence of latent confounders; (ii) estimation with modern datasets which are high-dimensional, noisy, and sparse. The key framework we introduce is connecting causal inference with tensor completion. In particular, we represent the various potential outcomes (i.e., counterfactuals) of interest through an order-3 tensor. The key theoretical results presented are: (i) Formal identification results establishing under what missingness patterns, latent confounding, and structure on the tensor is recovery of unobserved potential outcomes possible. (ii) Introducing novel estimators to recover these unobserved potential outcomes and proving they are finite-sample consistent and asymptotically normal. Finally, we discuss connections between matrix/tensor completion and time series analysis and reinforcement learning; we believe this could serve as a basis to do counterfactual forecasting, and building data-driven simulators for reinforcement learning.

Decomposing Euler–Poincaré Flow on the Space of Hamiltonian Vector Fields

The main result of this paper is a matched-pair decomposition of the space of symmetric contravariant tensors TQ. From this procedure two complementary Lie subalgebras of TQ under mutual interaction arise. Introducing a lift operator, the matched pair decomposition of the space of Hamiltonian vector fields is determined. According to this realization, the Euler–Poincaré flows on such spaces are decomposed into two subdynamics: one is the Euler–Poincaré formulation of isentropic fluid flows, and the other one corresponds with Euler–Poincaré equations on contravariant tensors of order n⩾2.

Trace maximization algorithm for the approximate tensor diagonalization

In this paper, we develop a Jacobi-type algorithm for the approximate diagonalization of tensors of order via tensor trace maximization. For a general tensor, this is an alternating least squares algorithm and the rotation matrices are chosen in each mode one-by-one to maximize the tensor trace. On the other hand, for symmetric tensors, we discuss a structure-preserving variant of this algorithm where in each iteration the same rotation is applied in all modes. We show that both versions of the algorithm converge to the stationary points of the corresponding objective functions.

Efficient Construction of Canonical Polyadic Approximations of Tensor Networks.

We consider the problem of constructing a canonical polyadic (CP) decomposition for a tensor network, rather than a single tensor. We illustrate how it is possible to reduce the complexity of constructing an approximate CP representation of the network by leveraging its structure in the course of the CP factor optimization. The utility of this technique is demonstrated for the order-4 Coulomb interaction tensor approximated by two order-3 tensors via an approximate generalized square-root (SQ) factorization, such as density fitting or (pivoted) Cholesky. The complexity of constructing a four-way CP decomposition is reduced from (for the nonapproximated Coulomb tensor) to (for the SQ-factorized Coulomb tensor), where n and RCP are the basis and CP ranks, respectively. This reduces the cost of constructing the CP approximation of two-body interaction tensors of relevance to accurate many-body electronic structure by up to 2 orders of magnitude for systems with up to 36 atoms studied here. The full four-way CP approximation of the Coulomb interaction tensor is shown to be more accurate than the known approaches which utilize CP-factorizations of the SQ factors (which are also constructed with an cost), such as the algebraic pseudospectral and tensor hypercontraction approaches. The CP-decomposed SQ factors can also serve as a robust initial guess for the four-way CP factors.

On Estimating Rank-One Spiked Tensors in the Presence of Heavy Tailed Errors

In this paper, we study the estimation of a rank-one spiked tensor in the presence of heavy tailed noise. Our results highlight some of the fundamental similarities and differences in the tradeoff between statistical and computational efficiencies under heavy tailed and Gaussian noise. In particular, we show that, for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> th order tensors, the tradeoff manifests in an identical fashion as the Gaussian case when the noise has finite <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4(p-1)$ </tex-math></inline-formula> th moment. The difference in signal strength requirements, with or without computational constraints, for us to estimate the singular vectors at the optimal rate, interestingly, narrows for noise with heavier tails and vanishes when the noise only has finite fourth moment. Moreover, if the noise has less than fourth moment, tensor SVD, perhaps the most natural approach, is suboptimal even though it is computationally intractable. Our analysis exploits a close connection between estimating the rank-one spikes and the spectral norm of a random tensor with iid entries. In particular, we show that the order of the spectral norm of a random tensor can be precisely characterized by the moment of its entries, generalizing classical results for random matrices. In addition to the theoretical guarantees, we propose estimation procedures for the heavy tailed regime, which are easy to implement and efficient to run. Numerical experiments are presented to demonstrate their practical merits.

Generalized eigenvalue for even order tensors via Einstein product and its applications in multilinear control systems

Generalized eigenvalue for even order tensors via Einstein product and its applications in multilinear control systems

Properties of generalized magnetic polarizability tensors

We present new alternative complete asymptotic expansions for the time harmonic low‐frequency magnetic field perturbation caused by the presence of a conducting permeable object as its size tends to zero for the eddy current approximation of the Maxwell system. Our new alternative formulations enable a natural extension of the well‐known rank 2 magnetic polarizability tensor (MPT) object characterization to higher order tensor descriptions by introducing generalized MPTs (GMPTs) using multi‐indices. In particular, we identify the magnetostatic contribution, provide new results on the symmetries of GMPTs, derive explicit formulae for the real and imaginary parts of GMPT coefficients and also describe the spectral behavior of GMPT coefficients. We also introduce the concept of harmonic GMPTs (HGMPTs) that have fewer coefficients than other GMPT descriptions of the same order. We describe the scaling, translation and rotational properties of HGMPTs and describe an approach for obtaining those HGMPT coefficients that are invariant under the action of a symmetry group. Such an approach is one candidate for selecting features in object classification for hidden object identification using HGMPTs.

Hot-SVD: higher order t-singular value decomposition for tensors based on tensor–tensor product

This paper considers a way of generalizing the t-SVD of third-order tensors (regarded as tubal matrices) to tensors of arbitrary order \(N\) [which can be similarly regarded as tubal tensors of order \((N-1)\)]. Such a generalization is different from the t-SVD for tensors of order greater than three (Martin et al. in SIAM J Sci Comput 35(1):A474–A490, 2013). The decomposition is called Hot-SVD, since it can be recognized as a tensor–tensor product version of the celebrated higher order SVD (HOSVD). The existence of Hot-SVD is proved. To this end, the “small-t” transpose for third-order tensors is introduced. This transpose is crucial in the verification of Hot-SVD, since it serves as a bridge between tubal tensors and their unfolding tubal matrices. We establish some properties of Hot-SVD, analogous to those of HOSVD. The truncated Hot-SVD and sequentially truncated Hot-SVD are then introduced, with \(\sqrt{N}\)-error bounds established for an \((N+1)\)-th-order tensor. We provide numerical examples to validate Hot-SVD, truncated Hot-SVD, and sequentially truncated Hot-SVD.

Geometry Does Impact on the Plane Strain Directions of the Human Left Ventricle, Irrespective of Disease.

The directions of primary strain lines of local deformation in Epicardial and Endocardial layers have been the subject of debate in recent years. Different methods led to different conclusions and a complete assessment of strain direction patterns in large and variable (in terms of pathology) cohorts of healthy and diseased patients is still lacking. Here, we use local deformation tensors in order to evaluate the angle of strain lines with respect to the horizontal circumferential direction in both Epi- and Endo-layers. We evaluated this on a large group of 193 subjects including 82 healthy control and 111 patients belonging to a great variety of pathological conditions. We found that Epicardial strain lines obliquely directed while those of Endocardium are almost circumferential. This result occurs irrespective of pathological condition. We propose that the geometric vinculum characterizing Endocardium and Epicardium in terms of different lever arm length and orientation of muscular fibers during contraction inescapably requires Endocardial strain lines to be circumferentially oriented and this is corroborated by experimental results. Further investigations on transmural structure of myocytes could couple results presented here in order to furnish additional experimental explanations.