Low-rank Solution Research Articles

Krylov-based adaptive-rank implicit time integrators for stiff problems with application to nonlinear Fokker-Planck kinetic models

We propose a high order adaptive-rank implicit integrators for stiff time-dependent PDEs, leveraging extended Krylov subspaces to efficiently and adaptively populate low-rank solution bases. This allows for the accurate representation of solutions with significantly reduced computational costs. We further introduce an efficient mechanism for residual evaluation and an adaptive rank-seeking strategy that optimizes low-rank settings based on a comparison between the residual size and the local truncation errors of the time-stepping discretization. We demonstrate our approach with the challenging Lenard-Bernstein Fokker-Planck (LBFP) nonlinear equation, which describes collisional processes in a fully ionized plasma. The preservation of the equilibrium state is achieved through the Chang-Cooper discretization, and strict conservation of mass, momentum and energy via a Locally Macroscopic Conservative (LoMaC) procedure. The development of implicit adaptive-rank integrators, demonstrated here up to third-order temporal accuracy via diagonally implicit Runge-Kutta schemes, showcases superior performance in terms of accuracy, computational efficiency, equilibrium preservation, and conservation of macroscopic moments. This study offers a starting point for developing scalable, efficient, and accurate methods for high-dimensional time-dependent problems.

Low-rank solutions to the stochastic Helmholtz equation

In this paper, we consider low-rank approximations for the solutions to the stochastic Helmholtz equation with random coefficients. A Stochastic Galerkin finite element method is used for the discretization of the Helmholtz problem. Existence theory for the low-rank approximation is established when the system matrix is indefinite. The low-rank algorithm does not require the construction of a large system matrix which results in an advantage in terms of CPU time and storage. Numerical results show that, when the operations in a low-rank method are performed efficiently, it is possible to obtain an advantage in terms of storage and CPU time compared to computations in full rank. We also propose a general approach to implement a preconditioner using the low-rank format efficiently.

An adaptive low-rank splitting approach for the extended Fisher–Kolmogorov equation

The extended Fisher–Kolmogorov (EFK) equation has been used to describe some phenomena in physical, material and biological systems. In this paper, we propose a full-rank splitting scheme and a rank-adaptive splitting approach for this equation. We first use a finite difference method to approximate the space derivatives. Then, the resulting semi-discrete system is split into two stiff linear parts and a nonstiff nonlinear part. This leads to our full-rank splitting scheme. The convergence of the proposed scheme is proved rigorously. Based on the frame of the full-rank splitting scheme, we design a rank-adaptive splitting approach for obtaining a low-rank solution of the EFK equation. Numerical examples show that our methods are robust and accurate. They can also preserve the energy dissipation.

PDRLRR: A novel low-rank representation with projection distance regularization via manifold optimization for clustering

The low-rank representation (LRR) method has attracted widespread attention due to its excellent performance in pattern recognition and machine learning. LRR-based variants have been proposed to solve the three existing problems in LRR: (1) the projection matrix is permanently fixed when dimensionality reduction techniques are adopted; (2) LRR fails to capture the local geometric structure; and (3) the solution deviates from the real low-rank solution. To address these problems, this paper proposes a low-rank representation with projection distance regularization (PDRLRR) via manifold optimization for clustering. In detail, we first introduce a low-dimensional projection matrix and a projection distance regularization term to fit the projected data automatically and capture the local structure of the data, respectively. Consequently, the projection matrix and representation matrix are obtained jointly. Then, we obtain a more accurate low-rank solution by minimizing the Schatten-p norm instead of the nuclear norm. Next, the projection matrix is optimized through a generalized Stiefel manifold. Extensive experiments demonstrate that our proposed method outperforms the state-of-the-art methods.

Loraine – an interior-point solver for low-rank semidefinite programming

The aim of this paper is to introduce a new code for the solution of large-and-sparse linear semidefinite programs (SDPs) with low-rank solutions or solutions with few outlying eigenvalues, and/or problems with low-rank data. We propose to use a preconditioned conjugate gradient method within an interior-point SDP algorithm and an efficient preconditioner fully utilizing the low-rank information. The efficiency is demonstrated by numerical experiments using the truss topology optimization problems, Lasserre relaxations of the MAXCUT problems and the sensor network localization problems.

Full-rank and low-rank splitting methods for the Swift–Hohenberg equation

The Swift–Hohenberg (SH) equation is an important model in the study of pattern formation. In this paper, we propose two full-rank splitting schemes and a low-rank approximation for the SH equation. We first employ a second-order finite difference method to approximate the space derivatives. Based on the resulting semi-discrete system, two full-rank splitting schemes are derived. The convergences of these schemes are studied. For finding a low-rank solution of the SH equation, we derive a low-rank splitting approximation. Numerical results indicate that our methods are robust, accurate and energy dissipation-preserving.

Approximately low-rank recovery from noisy and local measurements by convex program

Abstract Low-rank matrix models have been universally useful for numerous applications, from classical system identification to more modern matrix completion in signal processing and statistics. The nuclear norm has been employed as a convex surrogate of the low-rankness since it induces a low-rank solution to inverse problems. While the nuclear norm for low rankness has an excellent analogy with the $\ell _1$ norm for sparsity through the singular value decomposition, other matrix norms also induce low-rankness. Particularly as one interprets a matrix as a linear operator between Banach spaces, various tensor product norms generalize the role of the nuclear norm. We provide a tensor-norm-constrained estimator for the recovery of approximately low-rank matrices from local measurements corrupted with noise. A tensor-norm regularizer is designed to adapt to the local structure. We derive statistical analysis of the estimator over matrix completion and decentralized sketching by applying Maurey’s empirical method to tensor products of Banach spaces. The estimator provides a near-optimal error bound in a minimax sense and admits a polynomial-time algorithm for these applications.

A robust collision source method for rank adaptive dynamical low-rank approximation in radiation therapy

Deterministic models for radiation transport describe the density of radiation particles moving through a background material. In radiation therapy applications, the phase space of this density is composed of energy, spatial position and direction of flight. The resulting six-dimensional phase space prohibits fine numerical discretizations, which are essential for the construction of accurate and reliable treatment plans. In this work, we tackle the high dimensional phase space through a dynamical low-rank approximation of the particle density. Dynamical low-rank approximation (DLRA) evolves the solution on a low-rank manifold in time. Interpreting the energy variable as a pseudo-time lets us employ the DLRA framework to represent the solution of the radiation transport equation on a low-rank manifold for every energy. Stiff scattering terms are treated through an efficient implicit energy discretization and a rank adaptive integrator is chosen to dynamically adapt the rank in energy. To facilitate the use of boundary conditions and reduce the overall rank, the radiation transport equation is split into collided and uncollided particles through a collision source method. Uncollided particles are described by a directed quadrature set guaranteeing low computational costs, whereas collided particles are represented by a low-rank solution. It can be shown that the presented method is L2-stable under a time step restriction which does not depend on stiff scattering terms. Moreover, the implicit treatment of scattering does not require numerical inversions of matrices. Numerical results for radiation therapy configurations as well as the line source benchmark underline the efficiency of the proposed method.

Editorial: Tensor numerical methods and their application in scientific computing and data science

Editorial: Tensor numerical methods and their application in scientific computing and data science

An Unbiased Approach to Low Rank Recovery

Low rank recovery problems have been a subject of intense study in recent years. While the rank function is useful for regularization it is difficult to optimize due to its nonconvexity and discontinuity. The standard remedy for this is to exchange the rank function for the convex nuclear norm, which is known to favor low rank solutions under certain conditions. On the downside the nuclear norm exhibits a shrinking bias that can severely distort the solution in the presence of noise, which motivates the use of stronger nonconvex alternatives. In this paper we study two such formulations. We characterize the critical points and give sufficient conditions for a low rank stationary point to be unique. Moreover, we derive conditions that ensure global optimality of the low rank stationary point and show that these hold under moderate noise levels.

A reweighted matrix completion algorithm for sparse inverse synthetic aperture radar imaging

Sparse inverse synthetic aperture radar (ISAR) imaging can be generally achieved by compressed sensing (CS) methods because sparse sampling disables the approach of conventional range Doppler. However, the CS-based methods have to convert the matrix into a vector when the random sampling is adopted, which results in huge memory usage and high computational complexity. Note that matrix completion (MC) is suitable for matrix operations, which can recover random sparse data directly based on the low-rank property of the echo matrix. In order to improve the performance of the conventional MC approaches, a novel reweighted matrix completion method for sparse ISAR imaging is proposed in this paper. To avoid using the same singular value thresholding of the traditional MC method to achieve a low-rank solution, the reweighted scheme for singular values is applied to restrain the rank. Furthermore, the weights of the current iteration are updated with their singular values obtained in the former iteration, rather than using a fixed value. Then, the alternating direction method of multipliers is utilised to alternatively optimise the proposed model, which has an improved computational efficiency. Once the full data is recovered, the ISAR image can be achieved via the conventional 2D inverse fast Fourier transform. Experimental results based on measured data validate the proposed approach that can result in improving imaging performance within several seconds, which is tens of times faster than some reported low-rank-based methods.

A sweep-based low-rank method for the discrete ordinate transport equation

The dynamical low-rank (DLR) approximation is an efficient technique to approximate the solution to matrix differential equations. Recently, the DLR method was applied to radiation transport calculations to reduce memory requirements and computational costs. This work extends the low-rank scheme for the time-dependent radiation transport equation in 2-D and 3-D Cartesian geometries with discrete ordinates discretization in angle (SN method). The reduced system that evolves on a low-rank manifold is constructed via the “unconventional” basis update & Galerkin integrator to avoid a substep that is backward in time, which could be unstable for dissipative problems. The resulting system preserves the information on angular direction by applying separate low-rank decompositions in each octant where angular intensity has the same sign as the direction cosines. Then, transport sweeps and source iteration can efficiently solve this low-rank-SN system. The numerical results in 2-D and 3-D Cartesian geometries demonstrate that the low-rank solution requires less memory and computational time than solving the full rank equations using transport sweeps without losing accuracy.

Learning coordinate systems for fast and accurate acoustic modeling

Many popular numerical methods, such as finite difference and spectral methods, rely on simple domain geometry and environmental smoothness. Unfortunately, these features are rarely found in real-world simulations. We propose learning coordinate transformations with deep neural networks to facilitate acoustic modeling in complex media. Using automatic differentiation, we obtain new coordinate systems by solving various optimization problems that map the computational domain to a rectangular grid. Different choices of the objective function are utilized to attain different goals, including (i) mapping complicated boundaries such as the seafloor to straight lines and (ii) reducing the rank of two-dimensional slices of the refractive index. The first choice allows for domain decomposition to accurately model sharp discontinuities in density and sound speed. The second drastically accelerates a non-intrusive reduced-order modeling technique referred to as the dynamical low-rank approximation. Using realistic ocean test cases, we compare the performance of our learned coordinate systems with domain decomposition to the classic approach of smoothing sharp discontinuities, and we compare full-rank, low-rank, and coordinate-system-accelerated low-rank solutions of the three-dimensional parabolic wave equation.

A low-rank power iteration scheme for neutron transport criticality problems

Computing effective eigenvalues for neutron transport often requires a fine numerical resolution. The main challenge of such computations is the high memory effort of classical solvers, which limits the accuracy of chosen discretizations. In this work, we derive a method for the computation of effective eigenvalues when the underlying solution has a low-rank structure. This is accomplished by utilizing dynamical low-rank approximation (DLRA), which is an efficient strategy to derive time evolution equations for low-rank solution representations. The main idea is to interpret the iterates of the classical inverse power iteration as pseudo-time steps and apply the DLRA concepts in this framework. In our numerical experiment, we demonstrate that our method significantly reduces memory requirements while achieving the desired accuracy. Analytic investigations show that the proposed iteration scheme inherits the convergence speed of the inverse power iteration, at least for a simplified setting.

A low-rank solution method for Riccati equations with indefinite quadratic terms

Algebraic Riccati equations with indefinite quadratic terms play an important role in applications related to robust controller design. While there are many established approaches to solve these in case of small-scale dense coefficients, there is no approach available to compute solutions in the large-scale sparse setting. In this paper, we develop an iterative method to compute low-rank approximations of stabilizing solutions of large-scale sparse continuous-time algebraic Riccati equations with indefinite quadratic terms. We test the developed approach for dense examples in comparison to other established matrix equation solvers, and investigate the applicability and performance in large-scale sparse examples.

Column $\ell_{2,0}$-Norm Regularized Factorization Model of Low-Rank Matrix Recovery and Its Computation

This paper is concerned with the column $\ell_{2,0}$-regularized factorization model of low-rank matrix recovery problems and its computation. The column $\ell_{2,0}$-norm of factor matrices is introduced to promote column sparsity of factors and low-rank solutions. For this nonconvex discontinuous optimization problem, we develop an alternating majorization-minimization (AMM) method with extrapolation, and a hybrid AMM in which a majorized alternating proximal method is proposed to seek an initial factor pair with less nonzero columns and the AMM with extrapolation is then employed to minimize of a smooth nonconvex loss. We provide the global convergence analysis for the proposed AMM methods and apply them to the matrix completion problem with non-uniform sampling schemes. Numerical experiments are conducted with synthetic and real data examples, and comparison results with the nuclear-norm regularized factorization model and the max-norm regularized convex model show that the column $\ell_{2,0}$-regularized factorization model has an advantage in offering solutions of lower error and rank within less time.

An intelligent fault diagnosis method for roller bearings using an adaptive interactive deviation matrix machine

Support matrix machines (SMMs) take a matrix as the modeled element and can fully mine the structural information of matrix samples. However, relying solely on a pair of parallel hyperplanes limits the performance of SMMs in classifying complex data. Therefore, this paper proposes an adaptive interactive deviation matrix machine (AIDMM). In the AIDMM, a sensitive margin parameter is introduced to construct two deviation hyperplanes, so that the parameter margin between the two deviation hyperplanes becomes flexible. Compared to the original fixed maximum-margin method, the parameter-margin AIDMM can better adjust the boundary of the deviation hyperplane according to the data, which contributes to improving insensitivity to noise and enhancing robustness. In addition, a multi-rank projection matrix is introduced to obtain a low-rank solution, which gives AIDMM a better fitting ability and avoids the problem of large training errors. Two roller bearing fault datasets are applied for experimental verification, and the experimental results show that AIDMM has excellent classification performance in roller bearing fault diagnosis.

A low rank tensor representation of linear transport and nonlinear Vlasov solutions and their associated flow maps

We propose a low-rank tensor approach to approximate linear transport and nonlinear Vlasov solutions and their associated flow maps. The approach takes advantage of the fact that the differential operators in the Vlasov equation are tensor friendly, based on which we propose a novel way to dynamically and adaptively build up low-rank solution basis by adding new basis functions from discretization of the PDE, and removing basis from an SVD-type truncation procedure. For the discretization, we adopt a high order finite difference spatial discretization and a second order strong stability preserving multi-step time discretization. We apply the same procedure to evolve the dynamics of the flow map in a low-rank fashion, which proves to be advantageous when the flow map enjoys the low rank structure, while the solution suffers from high rank or displays filamentation structures. Hierarchical Tucker decomposition is adopted for high dimensional problems. An extensive set of linear and nonlinear Vlasov test examples are performed to show the high order spatial and temporal convergence of the algorithm with mesh refinement up to SVD-type truncation, the significant computational savings of the proposed low-rank approach especially for high dimensional problems, the improved performance of the flow map approach for solutions with filamentations.

A Sweep-Based Low-Rank Method for the Discrete Ordinate Transport Equation

The dynamical low-rank (DLR) approximation is an efficient technique to approximate the solution to matrix differential equations. Recently, the DLR method was applied to radiation transport calculations to reduce memory requirements and computational costs. This work extends the low-rank scheme for the time-dependent radiation transport equation in 2-D and 3-D Cartesian geometries with discrete ordinates discretization in angle (SN method). The reduced system that evolves on a low-rank manifold is constructed via an unconventional basis update and Galerkin integrator to avoid a substep that is backward in time, which could be unstable for dissipative problems. The resulting system preserves the information on angular direction by applying separate low-rank decompositions in each octant where angular intensity has the same sign as the direction cosines. Then, transport sweeps and source iteration can efficiently solve this low-rank-SN system. The numerical results in 2-D and 3-D Cartesian geometries demonstrate that the low-rank solution requires less memory and computational time than solving the full rank equations using transport sweeps without losing accuracy.

Single-Snapshot Nested Virtual Array Completion: Necessary and Sufficient Conditions

We study the problem of completing the virtual array of a nested array with a single snapshot. This involves synthesizing a virtual uniform linear array (ULA) with the same aperture as the nested array by estimating (or interpolating) the missing measurements. A popular approach for virtual array synthesis involves completing a certain Hankel/Toeplitz matrix from partial observations, by seeking low-rank solutions. However, existing theoretical guarantees for such structured rank minimization (which mostly provide sufficient conditions) do not readily extend to nested arrays. We provide the first necessary and sufficient conditions under which it is possible to exactly complete the virtual array of a nested array by minimizing the rank of a certain Toeplitz matrix constructed using a single temporal snapshot. Our results exploit the geometry of nested arrays and do not depend on the source configuration or on the separation between sources.