Tensor Train Representation Research Articles

Simulation of Spin Chains with Off-Diagonal Coupling Using the Inchworm Method.

We study the dynamical simulation of an open quantum spin chain with nearest neighboring coupling, where each spin in the chain is associated with a harmonic bath. This is an extension of our previous work (Wang, G.; Cai, Z. J. Chem. Theory Comput. 2023, 19, 8523-8540) by generalizing the application of the inchworm method and the technique of modular path integrals from diagonally coupled cases to off-diagonally coupled cases. Additionally, to reduce computational and memory cost in long time simulation, we apply tensor-train representation to efficiently represent the reduced density matrix of the spin chains, and employ the transfer tensor method (TTM) to avoid exponential growth of computational cost with respect to time. Abundant numerical experiments are performed to validate our method.

A quantum inspired approach to learning dynamical laws from data—block-sparsity and gauge-mediated weight sharing

Recent years have witnessed an increased interest in recovering dynamical laws of complex systems in a largely data-driven fashion under meaningful hypotheses. In this work, we propose a scalable and numerically robust method for this task, utilizing efficient block-sparse tensor train representations of dynamical laws, inspired by similar approaches in quantum many-body systems. Low-rank tensor train representations have been previously derived for dynamical laws of one-dimensional systems. We extend this result to efficient representations of systems with K-mode interactions and controlled approximations of systems with decaying interactions. We further argue that natural structure assumptions on dynamical laws, such as bounded polynomial degrees, can be exploited in the form of block-sparse support patterns of tensor-train cores. Additional structural similarities between interactions of certain modes can be accounted for by weight sharing within the ansatz. To make use of these structure assumptions, we propose a novel optimization algorithm, block-sparsity restricted alternating least squares with gauge-mediated weight sharing. The algorithm is inspired by similar notions in machine learning and achieves a significant improvement in performance over previous approaches. We demonstrate the performance of the method numerically on three one-dimensional systems—the Fermi–Pasta–Ulam–Tsingou system, rotating magnetic dipoles and point particles interacting via modified Lennard–Jones potentials, observing a highly accurate and noise-robust recovery.

Tensor networks for solving the time-independent Boltzmann neutron transport equation

Tensor network techniques, known for their low-rank approximation ability that breaks the curse of dimensionality, are emerging as a foundation of new mathematical methods for ultra-fast numerical solutions of high-dimensional Partial Differential Equations (PDEs). Here, we present a mixed Tensor Train (TT)/Quantized Tensor Train (QTT) approach for the numerical solution of time-independent Boltzmann Neutron Transport equations (BNTEs) in Cartesian geometry. Discretizing a realistic three-dimensional (3D) BNTE by (i) diamond differencing in space, (ii) multigroup-in-energy, and (iii) discrete ordinates collocation in angle leads to large generalized eigenvalue problems that generally require a matrix-free approach and large computer clusters. Starting from this discretization, we construct a TT representation of the PDE fields and discrete operators, followed by a QTT representation of the TT cores. We then solve the tensorized generalized eigenvalue problem using a fixed-point scheme with tensor network optimization techniques. We validate our approach by applying the method to two examples of 3D neutron transport problems, currently solved by the Los Alamos National Laboratory PARallel TIme-dependent SN (PARTISN) solver.1 We demonstrate that our TT/QTT method, executed on a standard desktop computer, leads to large compression. This allows for the storage of terrabyte-sized neutron angular flux eigenvectors in megabytes. Additionally, we create megabyte-sized full access TT representations of yottabyte-sized transport matrix operators. By leveraging the TT operators and solution methods, we obtain a 7500 times speedup when compared to the PARTISN solution time with an error of less than 10−5.

Active Fault Detection Based on Tensor Train Decomposition

The paper deals with active fault detection of stochastic systems based on tensor train representation of the Bellman function. The faulty and faulty-free behavior of the system is represented using multiple models. The active fault detection problem is treated as an optimal design problem similar to optimal stochastic control. The original problem is reformulated as a perfect state information problem by introducing an information state that contains statistics computed by a state estimator. The Bellman function is computed using the value iteration algorithm over a rectilinear grid set up in the information state space. Within the value iteration algorithm, the Bellman function is represented using the tensor train decomposition, and considerable attention is devoted to designing a rectilinear grid that respects the constraints placed on the elements of the information state.

Challenging the Curse of Dimensionality in Multidimensional Numerical Integration by Using a Low-Rank Tensor-Train Format

Numerical integration is a basic step in the implementation of more complex numerical algorithms suitable, for example, to solve ordinary and partial differential equations. The straightforward extension of a one-dimensional integration rule to a multidimensional grid by the tensor product of the spatial directions is deemed to be practically infeasible beyond a relatively small number of dimensions, e.g., three or four. In fact, the computational burden in terms of storage and floating point operations scales exponentially with the number of dimensions. This phenomenon is known as the curse of dimensionality and motivated the development of alternative methods such as the Monte Carlo method. The tensor product approach can be very effective for high-dimensional numerical integration if we can resort to an accurate low-rank tensor-train representation of the integrand function. In this work, we discuss this approach and present numerical evidence showing that it is very competitive with the Monte Carlo method in terms of accuracy and computational costs up to several hundredths of dimensions if the integrand function is regular enough and a sufficiently accurate low-rank approximation is available.

EFFICIENT APPROXIMATION OF HIGH-DIMENSIONAL EXPONENTIALS BY TENSOR NETWORKS

In this work a general approach to compute a compressed representation of the exponential $\exp(h)$ of a high-dimensional function $h$ is presented. Such exponential functions play an important role in several problems in Uncertainty Quantification, e.g. the approximation of log-normal random fields or the evaluation of Bayesian posterior measures. Usually, these high-dimensional objects are intractable numerically and can only be accessed pointwise in sampling methods. In contrast, the proposed method constructs a functional representation of the exponential by exploiting its nature as a solution of an ordinary differential equation. The application of a Petrov--Galerkin scheme to this equation provides a tensor train representation of the solution for which we derive an efficient and reliable a posteriori error estimator. Numerical experiments with a log-normal random field and a Bayesian likelihood illustrate the performance of the approach in comparison to other recent low-rank representations for the respective applications. Although the present work considers only a specific differential equation, the presented method can be applied in a more general setting. We show that the composition of a generic holonomic function and a high-dimensional function corresponds to a differential equation that can be used in our method. Moreover, the differential equation can be modified to adapt the norm in the a posteriori error estimates to the problem at hand.

Laser-controlled electronic symmetry breaking in a phenylene ethynylene dimer: Simulation by the hierarchical equations of motion and optimal control

The ability to prepare a specific superposition of electronic excited states leading to a transitory symmetry breaking of the electronic density in complex systems remains a challenging concern. We investigate how an initial coherence can be controlled by laser fields. The selected molecular system is a symmetric dimer of phenylene ethynylene presenting different interesting properties: Two bright nearly degenerate excited states coupled through a conical intersection are addressable by orthogonal transition dipole moments and are well separated from neighboring states. Creating a superposed state with equal weights corresponds to a right or left electronic localization as in the double-well system followed by a transitory oscillation between the two wells. To ensure a spectral bandwidth typically smaller than 0.25 eV, the pulse duration is in the tens of femtoseconds range, so nuclear motion cannot be neglected. Optimal control theory (OCT) is applied with guess fields that effectively create the target coherence in the absence of dephasing due to the vibrational baths. We analyze the field reshaping proposed by the control and we further fit a sequence of pulses on the optimal field. The overall result is efficient and robust disymmetry control over reasonable timescales of few tens of femtoseconds, exceeding the pulse duration. The monotonically convergent algorithm is combined with the hierarchical equations of motion (HEOM) able to treat strongly coupled non-Markovian dynamics. We also check the implementation of the combined OCT-HEOM approach in the tensor-train representation with propagation using the time-dependent variational method.

Density matrix reconstruction using non-negative matrix product states

Quantum state tomography is a key technique for quantum information processing but is challenging due to the exponential growth of its complexity with the system size. In this work we propose an algorithm which iteratively finds the best non-negative matrix product state approximation based on a set of measurement outcomes whose size does not necessarily grow exponentially. Compared to the tomography method based on neural network states, our scheme utilizes a so-called tensor train representation that allows straightforward recovery of the unknown density matrix in the matrix product operator form. As applications, the effectiveness of our algorithm is numerically demonstrated to reconstruct the ground state of the XXZ spin chain under depolarizing noise.

Rank Bounds for Approximating Gaussian Densities in the Tensor-Train Format

Low-rank tensor approximations have shown great potential for uncertainty quantification in high dimensions, for example, to build surrogate models that can be used to speed up large-scale inference problems (Eigel et al., Inverse Problems 34, 2018; Dolgov et al., Statistics & Computing 30, 2020). The feasibility and efficiency of such approaches depends critically on the rank that is necessary to represent or approximate the underlying distribution. In this paper, a-priori rank bounds for approximations in the functional tensor-train representation for the case of Gaussian models are developed. It is shown that under suitable conditions on the precision matrix, the Gaussian density can be approximated to high accuracy without suffering from an exponential growth of complexity as the dimension increases. These results provide a rigorous justification of the suitability and the limitations of low-rank tensor methods in a simple but important model case. Numerical experiments confirm that the rank bounds capture the qualitative behavior of the rank structure when varying the parameters of the precision matrix and the accuracy of the approximation. Finally, the practical relevance of the theoretical results is demonstrated in the context of a Bayesian filtering problem.

Tensor-Train Split-Operator KSL (TT-SOKSL) Method for Quantum Dynamics Simulations.

Numerically exact simulations of quantum reaction dynamics, including nonadiabatic effects in excited electronic states, are essential to gain fundamental insights into ultrafast chemical reactivity and rigorous interpretations of molecular spectroscopy. Here, we introduce the tensor-train split-operator KSL (TT-SOKSL) method for quantum simulations in tensor-train (TT)/matrix product state (MPS) representations. TT-SOKSL propagates the quantum state as a tensor train using the Trotter expansion of the time-evolution operator, as in the tensor-train split-operator Fourier transform (TT-SOFT) method. However, the exponential operators of the Trotter expansion are applied using a rank-adaptive TT-KSL scheme instead of using the scaling and squaring approach as in TT-SOFT. We demonstrate the accuracy and efficiency of TT-SOKSL as applied to simulations of the photoisomerization of the retinal chromophore in rhodopsin, including nonadiabatic dynamics at a conical intersection of potential energy surfaces. The quantum evolution is described in full dimensionality by a time-dependent wavepacket evolving according to a two-state 25-dimensional model Hamiltonian. We find that TT-SOKSL converges faster than TT-SOFT with respect to the maximally allowed memory requirement of the tensor-train representation and better preserves the norm of the time-evolving state. When compared to the corresponding simulations based on the TT-KSL method, TT-SOKSL has the advantage of avoiding the need to construct the matrix product state Laplacian by exploiting the linear scaling of multidimensional tensor-train Fourier transforms.

TgEDMD: Approximation of the Kolmogorov Operator in Tensor Train Format

Extracting information about dynamical systems from models learned off simulation data has become an increasingly important research topic in the natural and engineering sciences. Modeling the Koopman operator semigroup has played a central role in this context. As the approximation quality of any such model critically depends on the basis set, recent work has focused on deriving data-efficient representations of the Koopman operator in low-rank tensor formats, enabling the use of powerful model classes while avoiding over-fitting. On the other hand, detailed information about the system at hand can be extracted from models for the infinitesimal generator, also called Kolmogorov backward operator for stochastic differential equations. In this work, we present a data-driven method to efficiently approximate the generator using the tensor train (TT) format. The centerpiece of the method is a TT representation of the tensor of generator evaluations at all data sites. We analyze consistency and complexity of the method, present extensions to practically relevant settings, and demonstrate its applicability to benchmark numerical examples.

Solving the time-independent Schrödinger equation for chains of coupled excitons and phonons using tensor trains.

We demonstrate how to apply the tensor-train format to solve the time-independent Schrödinger equation for quasi-one-dimensional excitonic chain systems with and without periodic boundary conditions. The coupled excitons and phonons are modeled by Fröhlich-Holstein type Hamiltonians with on-site and nearest-neighbor interactions only. We reduce the memory consumption as well as the computational costs significantly by employing efficient decompositions to construct low-rank tensor-train representations, thus mitigating the curse of dimensionality. In order to compute also higher quantum states, we introduce an approach that directly incorporates the Wielandt deflation technique into the alternating linear scheme for the solution of eigenproblems. Besides systems with coupled excitons and phonons, we also investigate uncoupled problems for which (semi-)analytical results exist. There, we find that in the case of homogeneous systems, the tensor-train ranks of state vectors only marginally depend on the chain length, which results in a linear growth of the storage consumption. However, the central processing unit time increases slightly faster with the chain length than the storage consumption because the alternating linear scheme adopted in our work requires more iterations to achieve convergence for longer chains and a given rank. Finally, we demonstrate that the tensor-train approach to the quantum treatment of coupled excitons and phonons makes it possible to directly tackle the phenomenon of mutual self-trapping. We are able to confirm the main results of the Davydov theory, i.e., the dependence of the wave packet width and the corresponding stabilization energy on the exciton-phonon coupling strength, although only for a certain range of that parameter. In future work, our approach will allow calculations also beyond the validity regime of that theory and/or beyond the restrictions of the Fröhlich-Holstein type Hamiltonians.

Generative Modeling Via Tensor Train Sketching

In this paper we introduce a sketching algorithm for constructing a tensor train representation of a probability density from its samples. Our method deviates from the standard recursive SVD-based procedure for constructing a tensor train. Instead we formulate and solve a sequence of small linear systems for the individual tensor train cores. This approach can avoid the curse of dimensionality that threatens both the algorithmic and sample complexities of the recovery problem. Specifically, for Markov models, we prove that the tensor cores can be recovered with a sample complexity that is constant with respect to the dimension. Finally, we illustrate the performance of the method with several numerical experiments.

Expanding the Range of Hierarchical Equations of Motion by Tensor-Train Implementation.

The non-equilibrium thermo-field dynamics formulation of the hierarchical equations of motion combined with the tensor-train representation of the density matrix is discussed, and a new numerical integration scheme is introduced. The numerical methodology is based on an adaptive low-rank Galerkin reduction scheme and can preserve linear invariants (such as the trace of the density matrix). The method is applied to the study of the charge transfer dynamics in model pentacene molecular aggregates. The combined effect of a discrete set of molecular vibrational modes and a thermal bath is investigated, paying special attention to the coherent-incoherent transition of the charge transport. The new computational framework is shown to be a very promising methodology for the study of the quantum dynamics of complex molecular systems in the condensed phase.

Grid methods for Bayes-optimal continuous-discrete filtering and utilizing a functional tensor train representation

Optimal continuous-discrete filtering for a nonlinear system requires evolving the forward Kolmogorov equation, that is a Fokker–Planck equation, in alternation with Bayes' conditional updating. We present two numerical grid-methods that represent density functions on a mesh, or grid. For low-dimensional, smooth systems the finite-volume method is an effective solver that gives estimates that converge to the optimal continuous-time values. We give numerical examples to show that this finite-volume filter is able to handle multi-modal filtering distributions that result from rank-deficient observations, and that Bayes-optimal parameter estimation may be performed within the filtering process. The naïve discretization of density functions used in the finite-volume filter leads to an exponential increase of computational cost and storage with increasing dimension, that makes the finite-volume filter unfeasible for higher-dimensional problems. We circumvent this ‘curse of dimensionality’ by using a tensor train representation (or approximation) of density functions and employ an efficient implicit PDE solver that operates on the tensor train representation. We present numerical examples of tracking n weakly coupled pendulums in continuous time to demonstrate filtering with complex density functions in up to 80 dimensions.

Guaranteed a posteriori error bounds for low-rank tensor approximate solutions

Abstract We propose a guaranteed and fully computable upper bound on the energy norm of the error in low-rank tensor train (TT) approximate solutions of (possibly) high-dimensional reaction–diffusion problems. The error bound is obtained from Euler–Lagrange equations for a complementary flux reconstruction problem, which are solved in the low-rank TT representation using the block alternating linear scheme. This bound is guaranteed to be above the energy norm of the total error, including the discretization error, the tensor approximation error and the error in the solver of linear algebraic equations, although quadrature errors, in general, can pollute its evaluation. Numerical examples with the Poisson equation and the Schrödinger equation with the Henon–Heiles potential in up to 40 dimensions are presented to illustrate the efficiency of this approach.

Tensor train representation of MIMO channels using the JIRAFE method

MIMO technology has been subject of increasing interest in both academia and industry for future wireless standards. However, its performance benefits strongly depend on the accuracy of the channel at the base station. In a recent work, a fourth-order channel tensor model was proposed for MIMO systems. In this paper, we extend this model by exploiting additional spatial diversity at the receiver, which induces a fifth order tensor model for the channel. For such high orders, there is a crucial need to break the initial high-dimensional optimization problem into a collection of smaller coupled optimization sub-problems. This paper exploits new results on the equivalence between the canonical polyadic decomposition (CPD) and the tensor train (TT) decomposition for the multi-path scenario. Specifically, we propose a Joint dImensionality Reduction And Factor rEtrieval (JIRAFE) method to find the transmit and receive spatial signatures as well as the complex path gains (which also capture the polarization effects). Monte Carlo simulations show that our proposed TT-based representation of the channel is more robust to noise and computationally more attractive than available competing tensor-based methods, for physical parameters estimation.

Fast and Accurate Tensor Completion With Total Variation Regularized Tensor Trains

We propose a new tensor completion method based on tensor trains. The to-be-completed tensor is modeled as a low-rank tensor train, where we use the known tensor entries and their coordinates to update the tensor train. A novel tensor train initialization procedure is proposed specifically for image and video completion, which is demonstrated to ensure fast convergence of the completion algorithm. The tensor train framework is also shown to easily accommodate Total Variation and Tikhonov regularization due to their low-rank tensor train representations. Image and video inpainting experiments verify the superiority of the proposed scheme in terms of both speed and scalability, where a speedup of up to 155× is observed compared to state-of-the-art tensor completion methods at a similar accuracy. Moreover, we demonstrate the proposed scheme is especially advantageous over existing algorithms when only tiny portions (say, 1%) of the to-be-completed images/videos are known.

Density matrix dynamics in twin-formulation: An efficient methodology based on tensor-train representation of reduced equations of motion

The twin-formulation of quantum statistical mechanics is employed to describe a new methodology for the solution of the equations of motion of the reduced density matrix in their hierarchical formulation. It is shown that the introduction of tilde operators and of their algebra in the dual space greatly simplifies the application of numerical techniques for the propagation of the density matrix. The application of tensor-train representation of a vector to solve complex quantum dynamical problems within the framework of the twin-formulation is discussed. Next, applications of the hierarchical equations of motion to a dissipative polaron model are presented showing the validity and accuracy of the new approach.

Tensor Algorithms for Advanced Sensitivity Metrics

Following up on the success of the analysis of variance (ANOVA) decomposition and the Sobol indices (SI) for global sensitivity analysis, various related quantities of interest have been defined in the literature, including the effective and mean dimensions, the dimension distribution, and the Shapley values. Such metrics combine up to exponential numbers of SI in different ways and can be of great aid in uncertainty quantification and model interpretation tasks, but are computationally challenging. We focus on surrogate-based sensitivity analysis for independently distributed variables, namely, via the tensor train (TT) decomposition. This format permits flexible and scalable surrogate modeling and can efficiently extract all SI at once in a compressed TT representation of their own. Based on this, we contribute a range of novel algorithms that compute more advanced sensitivity metrics by selecting and aggregating certain subsets of SI in the tensor compressed domain. Drawing on an interpretation of the TT model in terms of deterministic finite automata, we are able to construct explicit auxiliary TT tensors that encode exactly all necessary index selection masks. Having both the SI and the masks in the TT format allows efficient computation of all aforementioned metrics, as we demonstrate in a number of example models.