Tensor Format Research Articles

Tensor-Based Parameter Reduction of Dynamic Load Models With Variable Frequency Drive

Accurate load modeling is critical for credible power system stability analysis. Because of the increasing complexity in modern system load, establishing a more comprehensive load model and performing accurate parameter estimation are the two biggest challenges in composite load modeling. In this paper, an induction motor with variable frequency drive (IM-VFD) together with the traditional ZIP+IM model is utilized to describe the dynamic characteristics of the system load. We estimate the time-varying probabilistic distributions of all states and parameters to describe their stochastic characteristics, thereby representing the real-time varying features of an actual load. The Fokker-Planck operator is applied to linearize the updated load model in tensor format. We then use tensor decomposition to reduce the computational and storage costs for calculating load model parameters. A sensitivity indicator is used to analyze the sensitivities of the estimated parameter distributions for subsequent model reduction. Simulations demonstrate that the updated load model has better performance than the traditional ZIP+IM model.

Compression of volume-surface integral equation matrices via Tucker decomposition for magnetic resonance applications.

In this work, we propose a method for the compression of the coupling matrix in volume-surface integral equation (VSIE) formulations. VSIE methods are used for electromagnetic analysis in magnetic resonance imaging (MRI) applications, for which the coupling matrix models the interactions between the coil and the body. We showed that these effects can be represented as independent interactions between remote elements in 3D tensor formats, and subsequently decomposed with the Tucker model. Our method can work in tandem with the adaptive cross approximation technique to provide fast solutions of VSIE problems. We demonstrated that our compression approaches can enable the use of VSIE matrices of prohibitive memory requirements, by allowing the effective use of modern graphical processing units (GPUs) to accelerate the arising matrix-vector products. This is critical to enable numerical MRI simulations at clinical voxel resolutions in a feasible computation time. In this paper, we demonstrate that the VSIE matrix-vector products needed to calculate the electromagnetic field produced by an MRI coil inside a numerical body model with 1 mm3 voxel resolution, could be performed in ~ 33 seconds in a GPU, after compressing the associated coupling matrix from ~ 80 TB to ~ 43 MB.

LEARNING HIGH-DIMENSIONAL PROBABILITY DISTRIBUTIONS USING TREE TENSOR NETWORKS

We consider the problem of the estimation of a high-dimensional probability distribution from i.i.d. samples of the distribution using model classes of functions in tree-based tensor formats, a particular case of tensor networks associated with a dimension partition tree. The distribution is assumed to admit a density with respect to a product measure, possibly discrete for handling the case of discrete random variables. After discussing the representation of classical model classes in tree-based tensor formats, we present learning algorithms based on empirical risk minimization using a $L^2$ contrast. These algorithms exploit the multilinear parametrization of the formats to recast the nonlinear minimization problem into a sequence of empirical risk minimization problems with linear models. A suitable parametrization of the tensor in tree-based tensor format allows to obtain a linear model with orthogonal bases, so that each problem admits an explicit expression of the solution and cross-validation risk estimates. These estimations of the risk enable the model selection, for instance when exploiting sparsity in the coefficients of the representation. A strategy for the adaptation of the tensor format (dimension tree and tree-based ranks) is provided, which allows to discover and exploit some specific structures of high-dimensional probability distributions such as independence or conditional independence. We illustrate the performances of the proposed algorithms for the approximation of classical probabilistic models (such as Gaussian distribution, graphical models, Markov chain).

Computing vibrational energy levels by solving linear equations using a tensor method with an imposed rank.

Present day computers do not have enough memory to store the high-dimensional tensors required when using a direct product basis to compute vibrational energy levels of a polyatomic molecule with more than about five atoms. One way to deal with this problem is to represent tensors using a tensor format. In this paper, we use the canonical polyadic (CP) format. Energy levels are computed by building a basis from vectors obtained by solving linear equations. The method can be thought of as a CP realization of a block inverse iteration method with multiple shifts. The CP rank of the tensors is fixed, and the linear equations are solved with an method. There is no need for rank reduction and no need for orthogonalization, and tensors with a rank larger than the fixed rank used to solve the linear equations are never generated. The ideas are tested by computing vibrational energy levels of a 64-D bilinearly coupled model Hamiltonian and of acetonitrile (12-D).

Some iterative approaches for Sylvester tensor equations, Part I: A tensor format of truncated Loose Simpler GMRES

In recent years, Krylov subspace methods based on the tensor format have demonstrated their superiority over classical ones for handling various kinds of tensor equations. In this paper, according to the efficient computational cost of the classical Simpler GMRES method (SGMRES), we adopt the tensor format of this method, which is called SGMRES−BTF, for solving the Sylvester tensor equations. This paper is divided into two parts describing the tensor format of two types of acceleration approaches to tackle some serious problems of the restarted methods GMRES−BTF and SGMRES−BTF, such as the “stalling” and “alternating phenomenon”. In the first part of this two-part work, the truncated Loose Simpler GMRES based on the tensor format (LSGMRES−BTF) is introduced, which is an acceleration approach that aims to construct an augmented tensor Krylov subspace by means of approximation error tensors. Moreover, a detailed study is carried out on the connection between the values of sequential and skip angles and the convergence behavior of both SGMRES−BTF and truncated LSGMRES−BTF. In the second part of this paper, an acceleration approach based on the idea of inner-outer iteration in the truncated version of the generalized conjugate residual with inner orthogonalization (GCRO) method is developed. In this method, SGMRES−BTF is applied in the inner iteration, and the generalized conjugate residual based on the tensor format (GCR−BTF) method is used in the outer iteration. Numerical experiments show that SGMRES−BTF achieves an appropriate performance compared with GMRES−BTF. In addition, the numerical results reveal the high potential of the presented accelerating strategies to deal with Sylvester tensor equations.

Some iterative approaches for Sylvester tensor equations, Part II: A tensor format of Simpler variant of GCRO-based methods

In the second part of this two-part work, another accelerator method based on the tensor format is established for solving the Sylvester tensor equations. This acceleration approach, which is called SGCRO−BTF, is based on the idea of inner-outer iteration used in the generalized conjugate residual with inner orthogonalization (GCRO) method. In SGCRO−BTF, the Simpler GMRES method based on the tensor format (SGMRES−BTF) is applied to the inner iteration, and the generalized conjugate residual based on the tensor format (GCR−BTF) method is used in the outer iteration. Furthermore, SGCRO−BTF seeks an approximate solution over a tensor subspace spanned by the approximation error tensors produced during the previous outer iterations of SGCRO−BTF and a tensor Krylov subspace constructed by the inner iteration. In order to reduce the computational storage in the outer iteration, the truncated version of SGCRO−BTF is presented, in which only some of the last approximation error tensors are kept and will be then added to the tensor Krylov subspace in order to obtain a new search subspace. Finally, the proposed methods are tested on a set of experiments and compared to some conventional and state-of-the-art Krylov subspace methods based on the tensor format. Experimental results indicate that the truncated version of SGCRO−BTF is particularly effective for solving the Sylvester tensor equations.

Analysis of Tensor Approximation Schemes for Continuous Functions

In this article, we analyze tensor approximation schemes for continuous functions. We assume that the function to be approximated lies in an isotropic Sobolev space and discuss the cost when approximating this function in the continuous analogue of the Tucker tensor format or of the tensor train format. We especially show that the cost of both approximations are dimension-robust when the Sobolev space under consideration provides appropriate dimension weights.

A generalized Fourier transform by means of change of variables within multilinear approximation

The paper deals with approximations of periodic functions that play a significant role in harmonic analysis. The approach revisits the trigonometric polynomials, seen as combinations of functions, and proposes to extend the class of models of the combined functions to a wider class of functions. The key here is to use structured functions, that have low complexity, with suitable functional representation and adapted parametrizations for the approximation. Such representation enables to approximate multivariate functions with few eventually random samples. The new parametrization is determined automatically with a greedy procedure, and a low rank format is used for the approximation associated with each new parametrization. A supervised learning algorithm is used for the approximation of a function of multiple random variables in tree-based tensor format, here the particular Tensor Train format. Adaptive strategies using statistical error estimates are proposed for the selection of the underlying tensor bases and the ranks for the Tensor-Train format. The method is applied for the estimation of the wall pressure for a flow over a cylinder for a range of low to medium Reynolds numbers for which we observe two flow regimes: a laminar flow with periodic vortex shedding and a laminar boundary layer with a turbulent wake (sub-critic regime). The automatic re-parametrization enables here to take into account the specific periodic feature of the pressure.

Prospects of Tensor-Based Numerical Modeling of the Collective Electrostatics in Many-Particle Systems

Recently the rank-structured tensor approach suggested a progress in the numerical treatment of the long-range electrostatic potentials in many-particle systems and the respective interaction energy and forces [39,40,2]. In this paper, we outline the prospects for tensor-based numerical modeling of the collective electrostatic potential on lattices and in many-particle systems of general type. We generalize the approach initially introduced for the rank-structured grid-based calculation of the collective potentials on 3D lattices [39] to the case of many-particle systems with variable charges placed on $L^{\otimes d}$ lattices and discretized on fine $n^{\otimes d}$ Cartesian grids for arbitrary dimension $d$. As result, the interaction potential is represented in a parametric low-rank canonical format in $O(d L n)$ complexity. The energy is then calculated in $O(d L)$ operations. Electrostatics in large biomolecules is modeled by using the novel range-separated (RS) tensor format [2], which maintains the long-range part of the 3D collective potential of the many-body system represented on $n\times n \times n$ grid in a parametric low-rank form in $O(n)$-complexity. We show that the force field can be easily recovered by using the already precomputed electric field in the low-rank RS format. The RS tensor representation of the discretized Dirac delta [45] enables the efficient energy preserving regularization scheme for solving the 3D elliptic PDEs with strongly singular right-hand side arising in bio-sciences. We conclude that the rank-structured tensor-based approximation techniques provide the promising numerical tools for applications to many-body dynamics, protein docking and classification problems and for low-parametric interpolation of scattered data in data science.

Solving the circuit equations using tensor decompositions

Tensor decomposition methods are widely used for solving high-dimensional and multi-dimensional problems. These approaches have a huge potential for reducing computational costs in the simulation of electronic circuits and in the CAD systems for electronics. Simulation of modern radio-frequency (RF) circuits, especially for RF integrated circuits, is one of the areas of extensive scientific research and development. The main problem of analysis methods for circuit design of RF electronics is the significant computational costs associated with the high dimensions to be solved. This paper shows that the computational costs of circuit design are largely determined by the costs of solving a system of linear algebraic equations (SLAE). The problems of the solving of SLAE are indicated in the paper. Further development of methods for solving and reducing computational costs requires the use of new approaches, one of which is proposed in this paper. The article presents the developed algorithm for converting a second-order matrix into a higher-order tensor. In this paper algorithms and software tools for tensor decompositions, as well as methods for solving SLAE in the representation of equations in tensor format are investigated. The results of numerical experiments and the results of comparison with traditional methods are presented.

Bayesian inversion for electromyography using low-rank tensor formats

The reconstruction of the structure of biological tissue using electromyographic (EMG) data is a non-invasive imaging method with diverse medical applications. Mathematically, this process is an inverse problem. Furthermore, EMG data are highly sensitive to changes in the electrical conductivity that describes the structure of the tissue. Modeling the inevitable measurement error as a stochastic quantity leads to a Bayesian approach. Solving the discretized Bayesian inverse problem means drawing samples from the posterior distribution of parameters, e.g., the conductivity, given measurement data. Using, e.g., a Metropolis–Hastings algorithm for this purpose involves solving the forward problem for different parameter combinations which requires a high computational effort. Low-rank tensor formats can reduce this effort by providing a data-sparse representation of all occurring linear systems of equations simultaneously and allow for their efficient solution. The application of Bayes’ theorem proves the well-posedness of the Bayesian inverse problem. The derivation and proof of a low-rank representation of the forward problem allow for the precomputation of all solutions of this problem under certain assumptions, resulting in an efficient and theory-based sampling algorithm. Numerical experiments support the theoretical results, but also indicate that a high number of samples is needed to obtain reliable estimates for the parameters. The Metropolis–Hastings sampling algorithm, using the precomputed forward solution in a tensor format, draws this high number of samples and therefore enables solving problems which are infeasible using classical methods.

Minimal divergence for border rank-2 tensor approximation

ABSTRACT A tensor is the sum of at least elementary tensors. In addition, a ‘border rank’ is defined: holds if r is the minimum integer such that is a limit of rank-r tensors. Usually, the set of rank-r tensors is not closed, i.e. tensors with may exist. It is easy to see that in such a case the representation of rank-r tensors contains diverging elementary tensors as approaches In a first part, we recall results about the uniform strength of the divergence in the case of general nonclosed tensor formats (restricted to finite dimensions). The second part discusses the r-term format for infinite-dimensional tensor spaces. It is shown that the general situation is very similar to the behaviour of finite-dimensional model spaces. The third part contains the main result: it is proved that in the case of the divergence strength is , i.e. if and the parameters of increase at least proportionally to

Regularization of Poisson--Boltzmann Type Equations with Singular Source Terms Using the Range-Separated Tensor Format

In this paper, we present a new regularization scheme for the linearized Poisson--Boltzmann equation (PBE) which models the electrostatic potential of biomolecules in a solvent. This scheme is base...

Robust Alternating Direction Implicit Solver in Quantized Tensor Formats for a Three-Dimensional Elliptic PDE

The aim of this paper is to propose a robust numerical solver, which is capable of efficiently solving a three-dimensional elliptic problem in a data-sparse quantized tensor format. In particular, ...

On the Compressibility of Tensors

Tensors are often compressed by expressing them in data-sparse tensor formats, where storage costs in such formats are less than those in the original structure. In this paper, we develop three met...

Tensor product method for fast solution of optimal control problems with fractional multidimensional Laplacian in constraints

We introduce the tensor numerical method for solution of the d-dimensional optimal control problems (d=2,3) with spectral fractional Laplacian type operators in constraints discretized on large n⊗d tensor-product Cartesian grids. The approach is based on the rank-structured approximation of the matrix valued functions of the corresponding fractional finite difference Laplacian. We solve the equation for the control function, where the system matrix includes the sum of the spectral fractional d-dimensional Laplacian and its inverse. The matrix valued functions of discrete Laplace operator on a tensor grid are diagonalized by using the fast Fourier transform (FFT). Then the low rank approximation of the d-dimensional tensors obtained by folding of the corresponding large diagonal matrices of eigenvalues are computed, which allows to solve the governing equation for the control function in a tensor-structured format. The existence of low rank canonical approximation to the class of matrix valued functions involved is justified by using the sinc quadrature approximation method applied to the Laplace transform of the generating function. The linear system of equations for the control function is solved by the PCG iterative method with the rank truncation at each iteration step, where the low Kronecker rank preconditioner is pre-computed. The right-hand side, the solution vector, and the governing system matrix are maintained in the rank-structured tensor format which beneficially reduces the numerical cost to O(nlog⁡n), outperforming the standard FFT based methods of complexity O(n3log⁡n) for 3D case. Numerical tests for the 2D and 3D control problems confirm the linear complexity scaling of the method in the univariate grid size n.

HT-AWGM: a hierarchical Tucker–adaptive wavelet Galerkin method for high-dimensional elliptic problems

This paper is concerned with the construction, analysis, and realization of a numerical method to approximate the solution of high-dimensional elliptic partial differential equations. We propose a new combination of an adaptive wavelet Galerkin method (AWGM) and the well-known hierarchical tensor (HT) format. The arising HT-AWGM is adaptive both in the wavelet representation of the low-dimensional factors and in the tensor rank of the HT representation. The point of departure is an adaptive wavelet method for the HT format using approximate Richardson iterations and an AWGM for elliptic problems. HT-AWGM performs a sequence of Galerkin solves based upon a truncated preconditioned conjugate gradient (PCG) algorithm in combination with a tensor-based preconditioner. Our analysis starts by showing convergence of the truncated conjugate gradient method. The next step is to add routines realizing the adaptive refinement. The resulting HT-AWGM is analyzed concerning convergence and complexity. We show that the performance of the scheme asymptotically depends only on the desired tolerance with convergence rates depending on the Besov regularity of low-dimensional quantities and the low-rank tensor structure of the solution. The complexity in the ranks is algebraic with powers of four stemming from the complexity of the tensor truncation. Numerical experiments show the quantitative performance.

Adaptive stochastic Galerkin FEM for lognormal coefficients in hierarchical tensor representations

Stochastic Galerkin methods for non-affine coefficient representations are known to cause major difficulties from theoretical and numerical points of view. In this work, an adaptive Galerkin FE method for linear parametric PDEs with lognormal coefficients discretized in Hermite chaos polynomials is derived. It employs problem-adapted function spaces to ensure solvability of the variational formulation. The inherently high computational complexity of the parametric operator is made tractable by using hierarchical tensor representations. For this, a new tensor train format of the lognormal coefficient is derived and verified numerically. The central novelty is the derivation of a reliable residual-based a posteriori error estimator. This can be regarded as a unique feature of stochastic Galerkin methods. It allows for an adaptive algorithm to steer the refinements of the physical mesh and the anisotropic Wiener chaos polynomial degrees. For the evaluation of the error estimator to become feasible, a numerically efficient tensor format discretization is developed. Benchmark examples with unbounded lognormal coefficient fields illustrate the performance of the proposed Galerkin discretization and the fully adaptive algorithm.

Discovering dynamic task-modulated functional networks with specific spectral modes using MEG

Efficient neuronal communication between brain regions through oscillatory synchronization at certain frequencies is necessary for cognition. Such synchronized networks are transient and dynamic, established on the timescale of milliseconds in order to support ongoing cognitive operations. However, few studies characterizing dynamic electrophysiological brain networks have simultaneously accounted for temporal non-stationarity, spectral structure, and spatial properties. Here, we propose an analysis framework for characterizing the large-scale phase-coupling network dynamics during task performance using magnetoencephalography (MEG). We exploit the high spatiotemporal resolution of MEG to measure time-frequency dynamics of connectivity between parcellated brain regions, yielding data in tensor format. We then use a tensor component analysis (TCA)-based procedure to identify the spatio-temporal-spectral modes of covariation among separate regions in the human brain. We validate our pipeline using MEG data recorded during a hand movement task, extracting a transient motor network with beta-dominant spectral mode, which is significantly modulated by the movement task. Next, we apply the proposed pipeline to explore brain networks that support cognitive operations during a working memory task. The derived results demonstrate the temporal formation and dissolution of multiple phase-coupled networks with specific spectral modes, which are associated with face recognition, vision, and movement. The proposed pipeline can characterize the spectro-temporal dynamics of functional connectivity in the brain on the subsecond timescale, commensurate with that of cognitive performance.

The maximum rank of 2 × ⋯ × 2 tensors over 𝔽2

We determine that the maximum rank of an order-n tensor with format over the finite field is for even n, and for odd n. Since tensor rank is non-increasing upon taking field extensions, gives the largest rank attainable for this tensor format. We also determine a maximum rank canonical form and compute its orbit under the action of the symmetry group , and prove that this is the unique maximum rank canonical form, for even .