Quantized Tensor Train Research Articles

QTT-finite-element approximation for multiscale problems I: model problems in one dimension

Tensor-compressed numerical solution of elliptic multiscale-diffusion and high frequency scattering problems is considered. For either problem class, solutions exhibit multiple length scales governed by the corresponding scale parameter: the scale of oscillations of the diffusion coefficient or smallest wavelength, respectively. As is well-known, this imposes a scale-resolution requirement on the number of degrees of freedom required to accurately represent the solutions in standard finite-element (FE) discretizations. Low-order FE methods are by now generally perceived unsuitable for high-frequency coefficients in diffusion problems and high wavenumbers in scattering problems. Accordingly, special techniques have been proposed instead (such as numerical homogenization, heterogeneous multiscale method, oversampling, etc.) which require, in some form, a-priori information on the microstructure of the solution. We analyze the approximation properties of tensor-formatted, conforming first-order FE methods for scale resolution in multiscale problems without a-priori information. The FE methods are based on the dynamic extraction of principal components from stiffness matrices, load and solution vectors by the quantized tensor train (QTT) decomposition. For prototypical model problems, we prove that this approach, by means of the QTT reparametrization of the FE space, allows to identify effective degrees of freedom to replace the degrees of freedom of a uniform "virtual" (i.e. never directly accessed) mesh, whose number may be prohibitively large to realize computationally. Precisely, solutions of model elliptic homogenization and high-frequency acoustic scattering problems are proved to admit QTT-structured approximations whose number of effective degrees of freedom required to reach a prescribed approximation error scales polylogarithmically with respect to the reciprocal of the target Sobolev-norm accuracy ź with only a mild dependence on the scale parameter. No a-priori information on the nature of the problems and intrinsic length scales of the solution is required in the numerical realization of the presently proposed QTT-structured approach. Although only univariate model multiscale problems are analyzed in the present paper, QTT structured algorithms are applicable also in several variables. Detailed numerical experiments confirm the theoretical bounds. As a corollary of our analysis, we prove that for the mentioned model problems, the Kolmogorov n-widths of solution sets are exponentially small for analytic data, independently of the problems' scale parameters. That implies, in particular, the exponential convergence of reduced basis techniques which is scale-robust, i.e., independent of the scale parameter in the problem.

Efficient Computation of Highly Oscillatory Integrals by Using QTT Tensor Approximation

Abstract We propose a new method for the efficient approximation of a class of highly oscillatory weighted integrals where the oscillatory function depends on the frequency parameter ω ≥ 0 ${\omega \ge 0}$ , typically varying in a large interval. Our approach is based, for a fixed but arbitrary oscillator, on the pre-computation and low-parametric approximation of certain ω-dependent prototype functions whose evaluation leads in a straightforward way to recover the target integral. The difficulty that arises is that these prototype functions consist of oscillatory integrals which makes them difficult to evaluate. Furthermore, they have to be approximated typically in large intervals. Here we use the quantized-tensor train (QTT) approximation method for functional M-vectors of logarithmic complexity in M in combination with a cross-approximation scheme for TT tensors. This allows the accurate approximation and efficient storage of these functions in the wide range of grid and frequency parameters. Numerical examples illustrate the efficiency of the QTT-based numerical integration scheme on various examples in one and several spatial dimensions.

Approximation of Singularities by Quantized‐Tensor FEM

AbstractIn d dimensions, first‐order tensor‐product finite‐element (FE) approximations of the solutions of second‐order elliptic problems are well known to converge algebraically, with rate at most 1/d in the energy norm and with respect to the number of degrees of freedom. On the other hand, FE methods of higher regularity may achieve exponential convergence, e.g. global spectral methods for analytic solutions and hp methods for solutions from certain countably normed spaces, which may exhibit singularities.In this note, we revisit, in one dimension, the tensor‐structured approach to the h‐FE approximation of singular functions. We outline a proof of the exponential convergence of such approximations represented in the quantized‐tensor‐train (QTT) format. Compared to special approximation techniques, such as hp, that approach is fully adaptive in the sense that it finds suitable approximation spaces algorithmically. The convergence is measured with respect to the number of parameters used to represent the solution, which is not the dimension of the first‐order FE space, but depends only polylogarithmically on that. We demonstrate the convergence numerically for a simple model problem and find the rate to be approximately the same as for hp approximations. (© 2015 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Tensor Approximation of Stationary Distributions of Chemical Reaction Networks

We prove that the stationary distribution of a system of reacting species with a weakly reversible reaction network of zero deficiency in the sense of Feinberg and separable propensity functions admits the tensor-structured approximation in the quantized tensor train (QTT) format. The complexity of the approximation scales linearly with respect to the number of species and logarithmically in the maximum copy numbers as well as in the desired accuracy. Our result covers the classical mass-action and Michaelis--Menten kinetics which correspond to two widely used classes of propensity functions and also to arbitrary combinations of those. New rank bounds for tensor-structured approximations of the probability density function (PDF) of a truncated one-dimensional Poisson distribution are an auxiliary result of the present paper which might be of independent interest. The present work complements recent results obtained by the authors jointly with Khammash and Nip on the tensor-structured numerical simulation of the evolution of system states distributions, driven by the Kolmogorov forward equation of the system, known also as the chemical master equation (CME). For the two kinetics mentioned above and a general reaction network, we also quantify the error of the low-rank tensor-structured approximation of the CME operator in the QTT format.

Direct Solution of the Chemical Master Equation Using Quantized Tensor Trains

The Chemical Master Equation (CME) is a cornerstone of stochastic analysis and simulation of models of biochemical reaction networks. Yet direct solutions of the CME have remained elusive. Although several approaches overcome the infinite dimensional nature of the CME through projections or other means, a common feature of proposed approaches is their susceptibility to the curse of dimensionality, i.e. the exponential growth in memory and computational requirements in the number of problem dimensions. We present a novel approach that has the potential to “lift” this curse of dimensionality. The approach is based on the use of the recently proposed Quantized Tensor Train (QTT) formatted numerical linear algebra for the low parametric, numerical representation of tensors. The QTT decomposition admits both, algorithms for basic tensor arithmetics with complexity scaling linearly in the dimension (number of species) and sub-linearly in the mode size (maximum copy number), and a numerical tensor rounding procedure which is stable and quasi-optimal. We show how the CME can be represented in QTT format, then use the exponentially-converging -discontinuous Galerkin discretization in time to reduce the CME evolution problem to a set of QTT-structured linear equations to be solved at each time step using an algorithm based on Density Matrix Renormalization Group (DMRG) methods from quantum chemistry. Our method automatically adapts the “basis” of the solution at every time step guaranteeing that it is large enough to capture the dynamics of interest but no larger than necessary, as this would increase the computational complexity. Our approach is demonstrated by applying it to three different examples from systems biology: independent birth-death process, an example of enzymatic futile cycle, and a stochastic switch model. The numerical results on these examples demonstrate that the proposed QTT method achieves dramatic speedups and several orders of magnitude storage savings over direct approaches.

Low-rank tensor structure of linear diffusion operators in the TT and QTT formats

We consider a class of multilevel matrices arising, for example, from the discretization of linear diffusion operators in a d-dimensional hypercube. We derive explicit representations of such matrices in the Tensor Train (TT) format, introduced recently for the non-linear low-parametric approximation of multi-dimensional vectors with the aim to handle the “curse of dimensionality”. We obtain sharp upper bounds on the TT ranks, which are linear or, when the diffusion tensor is semiseparable or quasi-separable, even sublinear in d (cf. the straightforward quadratic estimate). The use of the Quantized Tensor Train (QTT) decomposition allows to further reduce the number of parameters.

Two-Level QTT-Tucker Format for Optimized Tensor Calculus

We propose a combined tensor format, which encapsulates the benefits of Tucker, tensor train (TT), and quantized TT (QTT) formats. The structure is composed of subtensors in TT representations, so the approximation problem is proven to be stable. We describe all important algebraic and optimization operations, which are recast to the TT routines. Several examples on explicit function and operator representations are provided. The asymptotic storage complexity is at most cubic in the rank parameter, that is larger than for the global QTT approximation, but the numerical examples manifest that the ranks in the two-level format usually increase more slowly with the approximation accuracy than the QTT ones. In particular, we observe that high rank peaks, which usually occur in the TT/QTT representations, are significantly relaxed. Thus the reduced costs can be achieved.

Multilevel Toeplitz Matrices Generated by Tensor-Structured Vectors and Convolution with Logarithmic Complexity

We study the tensor structure of two operations: the transformation of a given multidimensional vector into a multilevel Toeplitz matrix and the convolution of two given multidimensional vectors. We show that the low-rank tensor structure of the input is preserved in the output and propose efficient algorithms for these operations in the newly introduced quantized tensor train (QTT) format. Consider a $d$-dimensional $2n \times\cdots\times 2n$-vector $\boldsymbol{x}$. If it is represented elementwise, the number of parameters is $(2n)^{d}$. However, if we assume that $\boldsymbol{x}$ is given in a QTT representation with ranks bounded by $p$, the number of parameters is reduced to $\mathcal{O}\left(dp^{2} \log n\right)$. Under this assumption we show how the multilevel Toeplitz matrix generated by $\boldsymbol{x}$ can be obtained in the QTT format with ranks bounded by $2p$ in $\mathcal{O}\left(dp^{2} \log n\right)$ operations. We also describe how the convolution $\boldsymbol{x}\star\boldsymbol{y}$ of $\...

Low-Rank Explicit QTT Representation of the Laplace Operator and Its Inverse

We focus on the construction of explicit low-rank representations of matrices in the tensor train (TT) and quantized tensor train (QTT) formats which have been proposed recently for the low-parametric structured representation of large-scale tensors. The matrices under consideration are discretizations of the Laplace operator in a hypercube (in one and many dimensions) and their inverses (in one dimension). For these matrices we derive explicit and exact QTT representations of low QTT ranks independent of the numbers $d$ and $n^{2}$ of dimensions and entries in each dimension. This implies that for the matrices considered the storage cost is $\mathcal{O}\left(d\,\log n\right)$, i.e., logarithmic with respect to the total number $n^{2d}$ of entries. The same applies to the computational complexity of the QTT-structured operations with these matrices, which now depends on the QTT ranks of the other operands. The general result of the paper is the notation and technique we introduce in order to examine the QTT structure of matrices analytically. They prove to be an efficient tool of studying other tensors related to particular computational problems, which are not considered in this paper.

Fast Solution of Parabolic Problems in the Tensor Train/Quantized Tensor Train Format with Initial Application to the Fokker--Planck Equation

In this paper we propose two schemes of using the so-called quantized tensor train (QTT)-approximation for the solution of multidimensional parabolic problems. First, we present a simple one-step implicit time integration scheme using a solver in the QTT-format of the alternating linear scheme (ALS) type. As the second approach, we use the global space-time formulation, resulting in a large block linear system, encapsulating all time steps, and solve it at once in the QTT-format. We prove the QTT-rank estimate for certain classes of multivariate potentials and respective solutions in $(x,t)$ variables. The log-linear complexity of storage and the solution time is observed in both spatial and time grid sizes. The method is applied to the Fokker--Planck equation arising from the beads-springs models of polymeric liquids.