Higher-order Tensors Research Articles

Isogeometric finite element-based simulation of the aortic heart valve: Integration of neural network structural material model and structural tensor fiber architecture representations.

The functional complexity of native and replacement aortic heart valves (AVs) is well known, incorporating such physical phenomenons as time-varying non-linear anisotropic soft tissue mechanical behavior, geometric non-linearity, complex multi-surface time varying contact, and fluid-structure interactions to name a few. It is thus clear that computational simulations are critical in understanding AV function and for the rational basis for design of their replacements. However, such approaches continued to be limited by ad-hoc approaches for incorporating tissue fibrous structure, high-fidelity material models, and valve geometry. To this end, we developed an integrated tri-leaflet valve pipeline built upon an isogeometric analysis framework. A high-order structural tensor (HOST)-based method was developed for efficient storage and mapping the two-dimensional fiber structural data onto the valvular 3D geometry. We then developed a neural network (NN) material model that learned the responses of a detailed meso-structural model for exogenously cross-linked planar soft tissues. The NN material model not only reproduced the full anisotropic mechanical responses but also demonstrated a considerable efficiency improvement, as it was trained over a range of realizable fibrous structures. Results of parametric simulations were then performed, as well as population-based bicuspid AV fiber structure, that demonstrated the efficiency and robustness of the present approach. In summary, the present approach that integrates HOST and NN material model provides an efficient computational analysis framework with increased physical and functional realism for the simulation of native and replacement tri-leaflet heart valves.

Tensor-Based Recurrent Neural Network and Multi-Modal Prediction With Its Applications in Traffic Network Management

Predicting the future traffic flows by applying deep learning methods has become an alternative way for transportation network management. Combining recurrent neural networks (RNNs) with tensor to implement accurate predictions has drawn intensive attention. However, traditional RNNs cannot deal with the high-order traffic flow data and capture their inherent structural relationship to provide accurate multi-modal prediction services. Therefore, this article focuses on proposing a series of tensor-based RNNs (T-RNNs) and a T-RNNs based multi-modal prediction approach (TMMP) to provide accurate prediction services. First, we propose three types of T-RNNs including tensor-based vanilla RNN, tensor-based long short-term memory (T-LSTM) and tensor-based gated recurrent unit (T-GRU), in which the input, output and weights are arbitrary high-order tensors. Then, to compress the weight parameters, we further propose two compact TT-based GRU (TT-GRU) and Tucker-based GRU (Tucker-GRU) models by applying tensor train (TT) and Tucker decompositions to T-GRU model. Afterwards, based on the high-order output tensor generated by T-RNNs, a TMMP approach is proposed to achieve the accurate predictions under various scenarios. Extensive experimental results on the metro traffic flow dataset demonstrate that the proposed TMMP approach can improve the traffic flow prediction accuracy by at most 25.29 percentage compared with the traditional MSE-based approaches. Meanwhile, compared with the T-GRU model, the TT-GRU model can compress the number of parameters by 200~780 times. The proposed T-RNNs and TMMP approach can adapt to different application scenarios and can be used to improve the efficiency of transportation management.

Tensor-Train-Based High-Order Dominant Eigen Decomposition for Multimodal Prediction Services

By leveraging neoteric analytical techniques associated with big data, numerous new data-focused computation and service models have flourished in service computing systems. Accurate future predictions based on tensor-based multivariate Markov models can vigorously support enterprise decisions. However, the computation efficiency and quick response of tensor-based multimodal prediction approach are seriously restricted by the curse of dimensionality arising from high-order tensor. Therefore, to alleviate the problem, this paper focuses on proposing a tensor-train (TT)-based computation approach with its scalable implementation for high-order dominant eigen decomposition (HODED) in multivariate Markov models. First, we present a TT-based Einstein product directly based on decomposed TT cores and guarantee that the result remains TT format. Then, we put forward a scalable implementation for TT-based Einstein product in a distributed or parallel manner. Afterwards, we propose a scalable TT-based HODED (TT-HODED) algorithm and a multimodal accurate prediction algorithm. Furthermore, a TT-based big data processing and services framework is presented to provide accurate proactive services. Experimental results based on real-world GPS trajectory dataset demonstrate that TT-HODED algorithm can significantly improve the computation efficiency and reduce the running memory on the premise of guaranteeing the almost consistent prediction accuracy compared to the original HODED algorithm.

Global optimization of tensor renormalization group using the corner transfer matrix

A tensor network renormalization algorithm with global optimization based on the corner transfer matrix is proposed. Since the environment is updated by the corner transfer matrix renormalization group method, the forward-backward iteration is unnecessary, which is a time-consuming part of other methods with global optimization. In addition, a further approximation reducing the order of the computational cost of contraction for the calculation of the coarse-grained tensor is proposed. The computational time of our algorithm in two dimensions scales as the sixth power of the bond dimension while the higher-order tensor renormalization group and the higher-order second renormalization group methods have the seventh power. We perform benchmark calculations in the Ising model on the square lattice and show that the time-to-solution of the proposed algorithm is faster than that of other methods.

A CLT in Stein’s Distance for Generalized Wishart Matrices and Higher-Order Tensors

AbstractWe study the a central limit theorem for sums of independent tensor powers, $\frac{1}{\sqrt{d}}\sum \limits _{i=1}^d X_i^{\otimes p}$. We focus on the high-dimensional regime where $X_i \in{\mathbb{R}}^n$ and $n$ may scale with $d$. Our main result is a proposed threshold for convergence. Specifically, we show that, under some regularity assumption, if $n^{2p-1}\ll d$, then the normalized sum converges to a Gaussian. The results apply, among others, to symmetric uniform log-concave measures and to product measures. This generalizes several results found in the literature. Our main technique is a novel application of optimal transport to Stein’s method, which accounts for the low-dimensional structure, which is inherent in $X_i^{\otimes p}$.

Flexoelectric effect in dielectrics under a dynamic load

Health monitoring of structures can be improved if a more universal two-way electromechanical coupling, the flexoelectric effect, is utilized. It is a coupling between the electric polarization and the strain gradients. In the direct flexoelectricity, the electric polarization is induced by strain gradients, which yield also a finite higher-order stress tensor in the considered phenomenological theory. Because of the size-effect in higher-grade theories of continua, the polarization in piezoelectric solids under a non-uniform strains in nano-sized structures is significantly influenced by flexoelectricity. This effect is substantially enhanced near the crack defects, in regions with large strain gradients. The mixed finite element method (FEM) is developed from the variational formulation of flexoelectric boundary value problems. The C0 continuous approximation is applied independently for both the displacements and displacement gradients. The kinematic constraints between them are satisfied by collocation at some internal points of elements. The developed computational method is applied to general 2D boundary value problems with cracks under a dynamic load. The influence of flexoelectricity on the induced electric potential and the crack opening displacement is investigated.

Local convergence of tensor methods

In this paper, we study local convergence of high-order Tensor Methods for solving convex optimization problems with composite objective. We justify local superlinear convergence under the assumption of uniform convexity of the smooth component, having Lipschitz-continuous high-order derivative. The convergence both in function value and in the norm of minimal subgradient is established. Global complexity bounds for the Composite Tensor Method in convex and uniformly convex cases are also discussed. Lastly, we show how local convergence of the methods can be globalized using the inexact proximal iterations.

Development of Online Learning Material for Data Science Programming Using 3D Puzzle

In multidimensional data processing, one of the important data structures is a higher-order tensor or a multidimensional array. In general, the processing related to the higher-order tensor is so complicated that we have been developing understanding support tools for it using 3D puzzles from the viewpoint of making students interested. However, although these tools have been tried by students in graduation studies and other some occasions, their introduction into lectures was one issue. Therefore, in this study, we developed a new programming exercise material for the higher-order tensor, which is supposed to be used in data science subjects, by using a 3D puzzle. This learning material is also composed of Microsoft Teams, and students can access the material remotely to learn programming. In this paper, several students actually tried this material. As a result, it was found that the students themselves could create and submit assignment reports by viewing explanatory videos and performing exercises. From this, it is expected that this material will be able to introduce to data science courses, including online use.

Deep Polynomial Neural Networks.

Deep convolutional neural networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their building blocks (e.g., residual blocks, rectifiers, sophisticated normalization schemes, to mention but a few). In this paper, we propose Π-Nets, a new class of function approximators based on polynomial expansions. Π-Nets are polynomial neural networks, i.e., the output is a high-order polynomial of the input. The unknown parameters, which are naturally represented by high-order tensors, are estimated through a collective tensor factorization with factors sharing. We introduce three tensor decompositions that significantly reduce the number of parameters and show how they can be efficiently implemented by hierarchical neural networks. We empirically demonstrate that Π-Nets are very expressive and they even produce good results without the use of non-linear activation functions in a large battery of tasks and signals, i.e., images, graphs, and audio. When used in conjunction with activation functions, Π-Nets produce state-of-the-art results in three challenging tasks, i.e., image generation, face verification and 3D mesh representation learning. The source code is available at https://github.com/grigorisg9gr/polynomial_nets.

Fast Circulant Tensor Power Method for High-Order Principal Component Analysis

To understand high-order intrinsic key patterns in high-dimensional data, tensor decomposition is a more versatile tool for data analysis than standard flat-view matrix models. Several existing tensor models aim to achieve rapid computation of high-order principal components based on the tensor power method. However, since a tensor power method does not enforce orthogonality in subsequently calculated decomposition components, it causes far more challenges on principal component analysis of high-order tensors. To address this problem, several tensor power method variant algorithms incorporating sparsity into decomposition factors have been proposed. However, because these variant algorithms require additional procedures based on data-driven hyper-parameter optimization algorithms, a trade-off between computational cost and convergence exists. In this paper, a novel tensor power method called the fast circulant tensor power method is proposed. The proposed algorithm combines tensor-train decomposition and the power method. Tensor-train decomposition is a high-order tensor decomposition method based on auxiliary unfolding matrix decomposition. Thus, the power method can be embedded into our methodology without any additional processes. Notably, a simple combination of these two methods may cause a local optima problem because the power method only guarantees convergence on each unfolding matrix in tensor-train decomposition. To solve this problem, the circulant updating method is proposed, which globally optimizes all factor vectors by reordering some steps of the factor vector updates. It is experimentally demonstrated that, compared to state-of-the-art tensor power method variant methodologies, the proposed algorithm achieves the lowest computational complexity and quantitatively good performance in various applications including large-scale color image decomposition and convolutional neural network compression.

Multi-Trek Separation in Linear Structural Equation Models

Building on the theory of causal discovery from observational data, we study interactions between multiple (sets of) random variables in a linear structural equation model with non-Gaussian error terms. We give a correspondence between structure in the higher-order cumulants and combinatorial structure in the causal graph. It has previously been shown that low rank of the covariance matrix corresponds to trek separation in the graph. Generalizing this criterion to multiple sets of vertices, we characterize when determinants of subtensors of the higher-order cumulant tensors vanish. This criterion applies when hidden variables are present as well. For instance, it allows us to identify the presence of a hidden common cause of $k$ of the observed variables.

Enhanced Nonconvex Low-Rank Approximation of Tensor Multi-Modes for Tensor Completion

Higher-order low-rank tensor arises in many data processing applications and has attracted great interests. Inspired by low-rank approximation theory, researchers have proposed a series of effective tensor completion methods. However, most of these methods directly consider the global low-rankness of underlying tensors, which is not sufficient for a low sampling rate; in addition, the single nuclear norm or its relaxation is usually adopted to approximate the rank function, which would lead to suboptimal solution deviated from the original one. To alleviate the above problems, in this paper, we propose a novel low-rank approximation of tensor multi-modes (LRATM), in which a double nonconvex L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">γ</sub> norm is designed to represent the underlying joint-manifold drawn from the factorization factors of each mode in the underlying tensor. A block successive upper-bound minimization method-based algorithm is designed to efficiently solve the proposed model, and it can be demonstrated that our numerical scheme converges to the coordinate-wise minimizers. Numerical results on three types of public multi-dimensional datasets have tested and shown that our algorithm can recover a variety of low-rank tensors with significantly fewer samples than the compared methods.

Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification

An increasing number of emerging applications in data science and engineering are based on multidimensional and structurally rich data. The irregularities, however, of high-dimensional data often compromise the effectiveness of standard machine learning algorithms. We hereby propose the Rank- $R$ Feedforward Neural Network (FNN), a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters, thereby offering two core advantages compared to typical machine learning methods. First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension. Moreover, the number of the model’s trainable parameters is substantially reduced, making it very efficient for small sample setting problems. We establish the universal approximation and learnability properties of Rank- $R$ FNN, and we validate its performance on real-world hyperspectral datasets. Experimental evaluations show that Rank- $R$ FNN is a computationally inexpensive alternative of ordinary FNN that achieves state-of-the-art performance on higher-order tensor data.

From Computation to Comparison of Tensor Decompositions

Decompositions of higher-order tensors into sums of simple terms are ubiquitous. We show that in order to verify that two tensors are generated by the same (possibly scaled) terms it is not necessary to compute the individual decompositions. In general the explicit computation of such a decomposition may have high complexity and can be ill-conditioned. We now show that under some assumptions the verification can be reduced to a comparison of both the column and row spaces of the corresponding matrix representations of the tensors. We consider rank-1 terms as well as low multilinear rank terms (also known as block terms) and show that the number of the terms and their multilinear rank can be inferred as well. The comparison relies only on numerical linear algebra and can be done in a numerically reliable way. We also illustrate how our results can be applied to solve a multilabel classification problem that appears in the context of blind source separation.

Systems of Polynomial Equations, Higher-order Tensor Decompositions, and Multidimensional Harmonic Retrieval: A Unifying Framework. Part I: The Canonical Polyadic Decomposition

We propose a multilinear algebra framework to solve systems of polynomial equations with simple roots. We translate connections between univariate polynomial root-finding, eigenvalue decompositions...

Hierarchical Factorization Strategy for High-Order Tensor and Application to Data Completion

Low-rank tensor completion (LRTC) aims to impute the missing entries from partially observed tensor data, among which low-rankness is of vital importance to get satisfactory results. In this letter, we propose a hierarchical low-rank factorization framework for high-order tensors. For the first layer, the low TR rank is exploited, and for the second layer the low-rankness of each TR core is further considered. With the hierarchical model, the low-rankness of the original tensor can be fully utilized and thus achieving better completion performance. Experimental results on synthetic data and on inpainting tasks using various datasets demonstrate the superior performance and efficiency of our proposed method as compared to the state-of-the-art algorithms.

SGD_Tucker: A Novel Stochastic Optimization Strategy for Scalable Parallel Sparse Tucker Decomposition

Sparse Tucker Decomposition (STD) algorithms learn a core tensor and a group of factor matrices to obtain an optimal low-rank representation feature for the High-Order, High-Dimension, and Sparse Tensor (HOHDST). However, existing STD algorithms face the problem of intermediate variables explosion which results from the fact that the formation of those variables, i.e., matrices Khatri-Rao product, Kronecker product, and matrix-matrix multiplication, follows the whole elements in sparse tensor. The above problems prevent deep fusion of efficient computation and big data platforms. To overcome the bottleneck, a novel stochastic optimization strategy (SGD Tucker) is proposed for STD which can automatically divide the high-dimension intermediate variables into small batches of intermediate matrices. Specifically, SGD Tucker only follows the randomly selected small samples rather than the whole elements, while maintaining the overall accuracy and convergence rate. In practice, SGD Tucker features the two distinct advancements over the state of the art. First, SGD Tucker can prune the communication overhead for the core tensor in distributed settings. Second, the low data-dependence of SGD Tucker enables fine-grained parallelization, which makes SGD Tucker obtaining lower computational overheads with the same accuracy. Experimental results show that SGD Tucker runs at least 2X faster than the state of the art.

Diagonalization of High-order Second Quantization Form

In this letter, by introducing high order tensor representation, the method to diagonalizing second quantization quadratic form is generalized to any orders operators. By some simple calculation and comparison to classic exact diagonalization, this new technique can be verified since has the same result. Moreover, this procedure could also be applied to some analytical computation, which may give some useful information in many-body interacting systems.

Explicit harmonic structure of bidimensional linear strain-gradient elasticity

In the perspective of homogenization theory, strain-gradient elasticity is a strategy to describe the overall behavior of materials with coarse mesostructure. In this approach, the effect of the mesostructure is described by the use of three elasticity tensors whose orders vary from 4 to 6. Higher-order constitutive tensors make it possible to describe rich physical phenomena. However, these objects have intricate algebraic structures that prevent us from having a clear picture of their modeling capabilities. The harmonic decomposition is a fundamental tool to investigate the anisotropic properties of constitutive tensor spaces. For higher-order tensors (i.e. tensors of order n≥3), their determination is generally a difficult task. In this paper, a novel procedure to obtain this decomposition is introduced. This method, which we have called the Clebsch–Gordan Harmonic Algorithm, allows one to obtain explicit harmonic decompositions satisfying good properties such as orthogonality and uniqueness. The elements of the decomposition also have a precise geometrical meaning simplifying their physical interpretation. This new algorithm is here developed in the specific case of 2D space and applied to Mindlin’s Strain-Gradient Elasticity. We provide, for the first time, the harmonic decompositions of the fifth- and sixth-order elasticity tensors involved in this constitutive law. The Clebsch–Gordan Harmonic Algorithm is not restricted to strain-gradient elasticity and may find interesting applications in different fields of mechanics which involve higher-order tensors.

Tensor train rank minimization with nonlocal self-similarity for tensor completion

<p style='text-indent:20px;'>The tensor train (TT) rank has received increasing attention in tensor completion due to its ability to capture the global correlation of high-order tensors (<inline-formula><tex-math id="M1">\begin{document}$ \rm{order} &gt;3 $\end{document}</tex-math></inline-formula>). For third order visual data, direct TT rank minimization has not exploited the potential of TT rank for high-order tensors. The TT rank minimization accompany with <i>ket augmentation</i>, which transforms a lower-order tensor (e.g., visual data) into a higher-order tensor, suffers from serious block-artifacts. To tackle this issue, we suggest the TT rank minimization with nonlocal self-similarity for tensor completion by simultaneously exploring the spatial, temporal/spectral, and nonlocal redundancy in visual data. More precisely, the TT rank minimization is performed on a formed higher-order tensor called group by stacking similar cubes, which naturally and fully takes advantage of the ability of TT rank for high-order tensors. Moreover, the perturbation analysis for the TT low-rankness of each group is established. We develop the alternating direction method of multipliers tailored for the specific structure to solve the proposed model. Extensive experiments demonstrate that the proposed method is superior to several existing state-of-the-art methods in terms of both qualitative and quantitative measures.