Low-rank Expansion Research Articles

A reduced-order model for nonlinear radiative transfer problems based on moment equations and POD-Petrov-Galerkin projection of the normalized Boltzmann transport equation

A data-driven projection-based reduced-order model (ROM) for nonlinear thermal radiative transfer (TRT) problems is presented. The TRT ROM is formulated by (i) a hierarchy of low-order quasidiffusion (aka variable Eddington factor) equations for moments of the radiation intensity and (ii) the normalized Boltzmann transport equation (BTE). The multilevel system of moment equations is derived by projection of the BTE onto a sequence of subspaces which represent elements of the phase space of the problem. Exact closure for the moment equations is provided by the Eddington tensor. A Petrov-Galerkin (PG) projection of the normalized BTE is formulated using a proper orthogonal decomposition (POD) basis representing the normalized radiation intensity over the whole phase space and time. The Eddington tensor linearly depends on the solution of the normalized BTE. By linear superposition of the POD basis functions, a low-rank expansion of the Eddington tensor is constructed with coefficients defined by the PG projected normalized BTE. The material energy balance (MEB) equation is coupled with the effective gray low-order equations which exist on the same dimensional scale as the MEB equation. The resulting TRT ROM is structure and asymptotic preserving. A detailed analysis of the ROM is performed on the classical Fleck-Cummings (F-C) TRT multigroup test problem in 2D geometry. Numerical results are presented to demonstrate the ROM's effectiveness in the simulation of radiation wave phenomena. The ROM is shown to produce solutions with sufficiently high accuracy while using low-rank approximation of the normalized BTE solution. Essential physical characteristics of supersonic radiation wave are preserved in the ROM solutions.

Machine learning assisted construction of a shallow depth dynamic ansatz for noisy quantum hardware.

The development of various dynamic ansatz-constructing techniques has ushered in a new era, making the practical exploitation of Noisy Intermediate-Scale Quantum (NISQ) hardware for molecular simulations increasingly viable. However, such ansatz construction protocols incur substantial measurement costs during their execution. This work involves the development of a novel protocol that capitalizes on regenerative machine learning methodologies and many-body perturbation theoretical measures to construct a highly expressive and shallow ansatz within the variational quantum eigensolver (VQE) framework with limited measurement costs. The regenerative machine learning model used in our work is trained with the basis vectors of a low-rank expansion of the N-electron Hilbert space to identify the dominant high-rank excited determinants without requiring a large number of quantum measurements. These selected excited determinants are iteratively incorporated within the ansatz through their low-rank decomposition. The reduction in the number of quantum measurements and ansatz depth manifests in the robustness of our method towards hardware noise, as demonstrated through numerical applications. Furthermore, the proposed method is highly compatible with state-of-the-art neural error mitigation techniques. This resource-efficient approach is quintessential for determining spectroscopic and other molecular properties, thereby facilitating the study of emerging chemical phenomena in the near-term quantum computing framework.

Low-rank matrix decompositions for ab initio nuclear structure

The extension of ab initio quantum many-body theory to higher accuracy and larger systems is intrinsically limited by the handling of large data objects in form of wave-function expansions and/or many-body operators. In this work we present matrix factorization techniques as a systematically improvable and robust tool to significantly reduce the computational cost in many-body applications at the price of introducing a moderate decomposition error. We demonstrate the power of this approach for the nuclear two-body systems, for many-body perturbation theory calculations of symmetric nuclear matter, and for non-perturbative in-medium similarity renormalization group simulations of finite nuclei. Establishing low-rank expansions of chiral nuclear interactions offers possibilities to reformulate many-body methods in ways that take advantage of tensor factorization strategies.

Calibrated Multivariate Regression Networks

In this paper, we propose a new multi-layer learning architecture, the calibrated multivariate regression network (CMRN). Compared to previous multivariate models, the CMRN is able to simultaneously handle major challenges in multivariate regression including highly nonlinear input-output relationships, underlying inter-output correlations and calibration of multiple outputs within one single framework. The CMRN is comprised of a nonlinear module with cosine activations and a linear module with the low-rank expansion, which establishes a compact multivariate regression network. By seamlessly working with the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> loss, the CMRN automatically calibrates multiple outputs with distinct noise levels to achieve improved performance. Being succinctly formulated but theoretically well-founded, the CMRN offers a compact multi-layer learning architecture that can be efficiently trained to scale up with massive datasets. We conduct extensive experimental evaluation on two representative large multivariate regression tasks for both machine learning and computer vision. The proposed CMRN can produce high performance on all tasks, which is better or competitive to state-of-the-art models. Extensive ablation studies offer deep insights into the effectiveness of the proposed CMRN.

Efficient Bayesian modeling of large lattice data using spectral properties of Laplacian matrix

Spatial data observed on a group of areal units is common in scientific applications. The usual hierarchical approach for modeling this kind of dataset is to introduce a spatial random effect with an autoregressive prior. However, the usual Markov chain Monte Carlo scheme for this hierarchical framework requires the spatial effects to be sampled from their full conditional posteriors one-by-one resulting in poor mixing. More importantly, it makes the model computationally inefficient for datasets with large number of units. In this article, we propose a Bayesian approach that uses the spectral structure of the adjacency to construct a low-rank expansion for modeling spatial dependence. We propose a pair of computationally efficient estimation schemes that select the functions most important to capture the variation in response. Through simulation studies, we validate the computational efficiency as well as predictive accuracy of our method. Finally, we present an important real-world application of the proposed methodology on a massive plant abundance dataset from Cape Floristic Region in South Africa.

Parametric PDEs: sparse or low-rank approximations?

We consider adaptive approximations of the parameter-to-solution map for elliptic operator equations depending on a large or infinite number of parameters, comparing approximation strategies of different degrees of nonlinearity: sparse polynomial expansions, general low-rank approximations separating spatial and parametric variables, and hierarchical tensor decompositions separating all variables. We describe corresponding adaptive algorithms based on a common generic template and show their near-optimality with respect to natural approximability assumptions for each type of approximation. A central ingredient in the resulting bounds for the total computational complexity is a new operator compression result in the case of infinitely many parameters. We conclude with a comparison of the complexity estimates based on the actual approximability properties of classes of parametric model problems, which shows that the computational costs of optimized low-rank expansions can be significantly lower or higher than those of sparse polynomial expansions, depending on the particular type of parametric problem.

Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition

Like other problems in computer vision, offline handwritten Chinese character recognition (HCCR) has achieved impressive results using convolutional neural network (CNN)-based methods. However, larger and deeper networks are needed to deliver state-of-the-art results in this domain. Such networks intuitively appear to incur high computational cost, and require the storage of a large number of parameters, which render them unfeasible for deployment in portable devices. To solve this problem, we propose a Global Supervised Low-rank Expansion (GSLRE) method and an Adaptive Drop-weight (ADW) technique to solve the problems of speed and storage capacity. We design a nine-layer CNN for HCCR consisting of 3755 classes, and devise an algorithm that can reduce the network’s computational cost by nine times and compress the network to 1/18 of the original size of the baseline model, with only a 0.21% drop in accuracy. In tests, the proposed algorithm can still surpass the best single-network performance reported thus far in the literature while requiring only 2.3MB for storage. Furthermore, when integrated with our effective forward implementation, the recognition of an offline character image takes only 9.7 ms on a CPU. Compared with the state-of-the-art CNN model for HCCR, our approach is approximately 30 times faster, yet 10 times more cost efficient.

Low-Rank Correction Methods for Algebraic Domain Decomposition Preconditioners

This paper presents a parallel preconditioning method for distributed sparse linear systems, based on an approximate inverse of the original matrix, that adopts a general framework of distributed sparse matrices and exploits the domain decomposition method and low-rank corrections. The domain decomposition approach decouples the matrix and once inverted, a low-rank approximation is applied by exploiting the Sherman-Morrison-Woodbury formula, which yields two variants of the preconditioning methods. The low-rank expansion is computed by the Lanczos procedure with reorthogonalizations. Numerical experiments indicate that, when combined with Krylov subspace accelerators, this preconditioner can be efficient and robust for solving symmetric sparse linear systems. Comparisons with other distributed-memory preconditioning methods are presented.

An efficient quasi-optimal space-time PGD application to frictional contact mechanics

The proper generalized decomposition (PGD) aims at finding the solution of a generic problems into a low rank approximation. On the contrary to the singular value decomposition (SVD), such a low rank approximation is generally not the optimal one leading to memory issues and loss of computational efficiency. Nonetheless, the computational cost of the SVD is generally prohibitive to be performed. In this paper, authors suggest an algorithm to address this issue. First, the algorithm is described and studied in details. It consists in a cheap iterative method compressing a low rank expansion. It will be shown that given a low rank approximation, the SVD of a provided low rank approximation can be reached at convergence. Behavior of the method is exhibited on a numerical application. Second, the algorithm is embedded into a general space-time PGD solver to compress the iterated separated form for the solution. An application to a quasi-static frictional contact problem is illustrated. Then, efficiency of such a compressing method will be demonstrated.

Divide and Conquer Low-Rank Preconditioners for Symmetric Matrices

This paper presents a preconditioning method based on an approximate inverse of the original matrix, computed recursively from a multilevel low-rank (MLR) expansion approach. The basic idea is to recursively divide the problem in two and apply a low-rank approximation to a matrix obtained from the Sherman--Morrison formula. The low-rank expansion is computed by a few steps of the Lanczos bidiagonalization procedure. The MLR preconditioner has been motivated by its potential for exploiting different levels of parallelism on modern high-performance platforms, though this feature is not yet tested in this paper. Numerical experiments indicate that, when combined with Krylov subspace accelerators, this preconditioner can be efficient and robust for solving symmetric sparse linear systems.

Multiscale Discrete Approximation of Fourier Integral Operators

We develop a discretization and computational procedures for approximation of the action of Fourier integral operators the canonical relations of which are graphs. Such operators appear, for instance, in the formulation of imaging and inverse scattering of seismic reflection data. Our discretization and algorithms are based on a multiscale low-rank expansion of the action of Fourier integral operators using the dyadic parabolic decomposition of phase space and on explicit constructions of low-rank separated representations using prolate spheroidal wave functions, which directly reflect the geometry of such operators. The discretization and computational procedures connect to the discrete almost symmetric wave packet transform. Numerical wave propagation and imaging examples illustrate our computational procedures.