Solution Of Large Linear Systems Research Articles

A spectrally preconditioned and initially deflated variant of the restarted block GMRES method for solving multiple right-hand sides linear systems

The solution of large linear systems with multiple right-hand sides given simultaneously is required in many large-scale scientific and engineering applications modelled by either partial differential or boundary integral equations. Block Krylov subspace methods are attractive to use for this problem class as they can overcome the memory bottleneck of direct methods and they perform block matrix-vector products, achieving high computational efficiency on modern cache-based computer architectures. In this paper we introduce variants of the block GMRES method that combine initial deflation and eigenvalue recycling strategies to remedy some typical convergence problems of block Krylov solvers. The new class of block iterative solvers has the ability to handle the approximate linear dependence of the block of right-hand sides and exploits approximate invariant subspaces recycled over the iterations to mitigate the bad effects that small eigenvalues can have on the convergence, by adapting an existing preconditioner. We illustrate the numerical behavior of the spectrally updated and initially deflated block GMRES method on a set of linear systems arising from the discretization of the Dirac equation and of boundary integral equations in electromagnetics scattering.

Indexed fast network proximity querying

Node proximity queries are among the most common operations on network databases. A common measure of node proximity is random walk based proximity, which has been shown to be less susceptible to noise and missing data. Real-time processing of random-walk based proximity queries poses significant computational challenges for larger graphs with over billions of nodes and edges, since it involves solution of large linear systems of equations. Due to the importance of this operation, significant effort has been devoted to developing efficient methods for random-walk based node proximity computations. These methods either aim to speed up iterative computations by exploiting numerical properties of random walks, or rely on computation and storage of matrix inverses to avoid computation during query processing. Although both approaches have been well studied, the speedup achieved by iterative approaches does not translate to real-time query processing, and the storage requirements of inversion-based approaches prohibit their use on very large graph databases. We present a novel approach to significantly reducing the computational cost of random walk based node proximity queries with scalable indexing. Our approach combines domain graph-partitioning based indexing with fast iterative computations during query processing using Chebyshev polynomials over the complex elliptic plane. This approach combines the query processing benefits of inversion techniques with the memory and storage benefits of iterative approache. Using real-world networks with billions of nodes and edges, and top- k proximity queries as the benchmark problem, we show that our algorithm, I-C hopper , significantly outperforms existing methods. Specifically, it drastically reduces convergence time of the iterative procedure, while also reducing storage requirements for indexing.

A RBF partition of unity collocation method based on finite difference for initial–boundary value problems

Meshfree radial basis function (RBF) methods are popular tools used to numerically solve partial differential equations (PDEs). They take advantage of being flexible with respect to geometry, easy to implement in higher dimensions, and can also provide high order convergence. Since one of the main disadvantages of global RBF-based methods is generally the computational cost associated with the solution of large linear systems, in this paper we focus on a localizing RBF partition of unity method (RBF-PUM) based on a finite difference (FD) scheme. Specifically, we propose a new RBF-PUM-FD collocation method, which can successfully be applied to solve time-dependent PDEs. This approach allows to significantly decrease ill-conditioning of traditional RBF-based methods. Moreover, the RBF-PUM-FD scheme results in a sparse matrix system, reducing the computational effort but maintaining at the same time a high level of accuracy. Numerical experiments show performances of our collocation scheme on two benchmark problems, involving unsteady convection–diffusion and pseudo-parabolic equations.

A comparison of two preconditioner algorithms within the representer-based four-dimensional variational data assimilation system for the Navy coastal ocean model

ABSTRACTFour-dimensional variational (4D-Var) data assimilation for operational systems requires the solution of large linear systems that are poorly conditioned in general. In addition to efficient iterative solvers for linear systems, using a good preconditioner is required to guarantee an acceptable solution with a small number of iterations. We consider the assimilation of ocean observations using a weak constraints 4D-Var for the Navy Coastal Ocean Model (NCOM), based on the representer method. Two methods of preconditioning the linear system are implemented, namely the scaling of the linear system by the square root of the observations error variances, and the approximating stabilised representer matrix based on the computation of some representer functions. We evaluate their convergence using criteria such as the norm of the residuals and of the gradient of the cost function, the analysis error evaluated at the observation locations, and finally the convergence of the sequence of analyses. Results from all criteria show that the fastest convergence is achieved with the rescaling preconditioner when the conjugate gradient used to solve the linear system is equipped with a suitable inner product instead of the Euclidean.

Global Golub–Kahan bidiagonalization applied to large discrete ill-posed problems

We consider the solution of large linear systems of equations that arise from the discretization of ill-posed problems. The matrix has a Kronecker product structure and the right-hand side is contaminated by measurement error. Problems of this kind arise, for instance, from the discretization of Fredholm integral equations of the first kind in two space-dimensions with a separable kernel and in image restoration problems. Regularization methods, such as Tikhonov regularization, have to be employed to reduce the propagation of the error in the right-hand side into the computed solution. We investigate the use of the global Golub–Kahan bidiagonalization method to reduce the given large problem to a small one. The small problem is solved by employing Tikhonov regularization. A regularization parameter determines the amount of regularization. The connection between global Golub–Kahan bidiagonalization and Gauss-type quadrature rules is exploited to inexpensively compute bounds that are useful for determining the regularization parameter by the discrepancy principle.

A global Lanczos method for image restoration

Image restoration often requires the solution of large linear systems of equations with a very ill-conditioned, possibly singular, matrix and an error-contaminated right-hand side. The latter represents the available blur and noise-contaminated image, while the matrix models the blurring. Computation of a meaningful restoration of the available image requires the use of a regularization method. We consider the situation when the blurring matrix has a Kronecker product structure and an estimate of the norm of the desired image is available, and illustrate that efficient restoration of the available image can be achieved by Tikhonov regularization based on the global Lanczos method, and by using the connection of the latter to Gauss-type quadrature rules.

Accelerated multigrid for graph Laplacian operators

We consider multigrid type techniques for the numerical solution of large linear systems, whose coefficient matrices show the structure of (weighted) graph Laplacian operators. We combine ad hoc coarser-grid operators with iterative techniques used as smoothers. Empirical tests suggest that the most effective smoothers have to be of Krylov type with subgraph preconditioners, while the projectors, which define the coarser-grid operators, have to be designed for maintaining as much as possible the graph structure of the projected matrix at the inner levels. The main theoretical contribution of the paper is the characterization of necessary and sufficient conditions for preserving the graph structure. In this framework it is possible to explain why the classical projectors inherited from differential equations are good in the differential context and why they may behave unsatisfactorily for unstructured graphs. Furthermore, we report and discuss several numerical experiments, showing that our approach is effective even in very difficult cases where the known approaches are rather slow. As a conclusion, the main advantage of the proposed approach is the robustness, since our multigrid type technique behaves uniformly well in all cases, without requiring either the setting or the knowledge of critical parameters, as it happens when using the best known preconditioned Krylov methods.

Some properties of range restricted GMRES methods

The GMRES method is one of the most popular iterative schemes for the solution of large linear systems of equations with a square nonsingular matrix. GMRES-type methods also have been applied to the solution of linear discrete ill-posed problems. Computational experience indicates that for the latter problems variants of the standard GMRES method, that require the solution to live in the range of a positive power of the matrix of the linear system of equations to be solved, generally yield more accurate approximations of the desired solution than standard GMRES. This paper investigates properties of these variants of GMRES.

Compression approaches for the regularized solutions of linear systems from large-scale inverse problems

We introduce and compare new compression approaches to obtain regularized solutions of large linear systems which are commonly encountered in large scale inverse problems. We first describe how to approximate matrix vector operations with a large matrix through a sparser matrix with fewer nonzero elements, by borrowing from ideas used in wavelet image compression. Next, we describe and compare approaches based on the use of the low rank singular value decomposition (SVD), which can result in further size reductions. We describe how to obtain the approximate low rank SVD of the original matrix using the sparser wavelet compressed matrix. Some analytical results concerning the various methods are presented and the results of the proposed techniques are illustrated using both synthetic data and a very large linear system from a seismic tomography application, where we obtain significant compression gains with our methods, while still resolving the main features of the solutions.

Tikhonov regularization via flexible Arnoldi reduction

Flexible GMRES, introduced by Saad, is a generalization of the standard GMRES method for the solution of large linear systems of equations. It is based on the flexible Arnoldi process for reducing a large square matrix to a small matrix. We describe how the flexible Arnoldi process can be applied to implement one-parameter and multi-parameter Tikhonov regularization of linear discrete ill-posed problems. The method proposed is well suited for large-scale problems. Moreover, computed examples show that our method can give approximate solutions of higher accuracy than available direct methods for small-scale problems.

Large Scale Finite Element Analysis Via Assembly-Free Deflated Conjugate Gradient

Large-scale finite element analysis (FEA) with millions of degrees of freedom (DOF) is becoming commonplace in solid mechanics. The primary computational bottleneck in such problems is the solution of large linear systems of equations. In this paper, we propose an assembly-free version of the deflated conjugate gradient (DCG) for solving such equations, where neither the stiffness matrix nor the deflation matrix is assembled. While assembly-free FEA is a well-known concept, the novelty pursued in this paper is the use of assembly-free deflation. The resulting implementation is particularly well suited for large-scale problems and can be easily ported to multicore central processing unit (CPU) and graphics-programmable unit (GPU) architectures. For demonstration, we show that one can solve a 50 × 106 degree of freedom system on a single GPU card, equipped with 3 GB of memory. The second contribution is an extension of the “rigid-body agglomeration” concept used in DCG to a “curvature-sensitive agglomeration.” The latter exploits classic plate and beam theories for efficient deflation of highly ill-conditioned problems arising from thin structures.

Block GMRES Method with Inexact Breakdowns and Deflated Restarting

We consider the solution of large linear systems with multiple right-hand sides using a block GMRES approach. We introduce a new algorithm that effectively handles the situation of almost rank deficient block generated by the block Arnoldi procedure and that enables the recycling of spectral information at restart. The first feature is inherited from an algorithm introduced by Robbé and Sadkane [Linear Algebra Appl., 419 (2006), pp. 265--285], while the second one is obtained by extending the deflated restarting strategy proposed by Morgan [Appl. Numer. Math., 54 (2005), pp. 222--236]. Through numerical experiments, we show that the new algorithm combines efficiently the attractive numerical features of its two parents and outperforms them.

Speedup of water distribution simulation by domain decomposition

The Schur complement domain decomposition method is used for solution of large linear systems. The algorithm is based on the subdivision of the domain into smaller ones and the solution of those sub-domains independently. Regarding water distribution systems modeling, the hydraulic simulation could be formulated as a sequence of systems of linear equations. Therefore, this paper utilizes the domain decomposition method to accelerate the simulation process further. The method is evaluated using a large scale real-world system with 63,616 junctions and 64,200 pipes as case study. The case study shows that the methodology could improve the performance of hydraulic simulation app. by a factor of 8 without losing accuracy at a suitable level of domain decomposition. Although the optimal level of decomposition is case specific, considerable speedup might still be achievable by decomposing a large system into only a few subsystems.

FGMRES for linear discrete ill-posed problems

GMRES is one of the most popular iterative methods for the solution of large linear systems of equations. However, GMRES does not always perform well when applied to the solution of linear systems of equations that arise from the discretization of linear ill-posed problems with error-contaminated data represented by the right-hand side. Such linear systems are commonly referred to as linear discrete ill-posed problems. The FGMRES method, proposed by Saad, is a generalization of GMRES that allows larger flexibility in the choice of solution subspace than GMRES. This paper explores application of FGMRES to the solution of linear discrete ill-posed problems. Numerical examples illustrate that FGMRES with a suitably chosen solution subspace may determine approximate solutions of higher quality than commonly applied iterative methods.

New Preconditioners for Nonsymmetric Saddle Point Systems with Singular (1,1) Block

We investigate the solution of large linear systems of saddle point type with singular (1,1) block by preconditioned iterative methods and consider two parameterized block triangular preconditioners used with Krylov subspace methods which have the attractive property of improved eigenvalue clustering with increased ill-conditioning of the (1,1) block of the saddle point matrix, including the choice of the parameter. Meanwhile, we analyze the spectral characteristics of two preconditioners and give the optimal parameter in practice. Numerical experiments that validate the analysis are presented.

Flexible Variants of Block Restarted GMRES Methods with Application to Geophysics

In a wide number of applications in computational science and engineering the solution of large linear systems of equations with several right-hand sides given at once is required. Direct methods based on Gaussian elimination are known to be especially appealing in that setting. Nevertheless, when the dimension of the problem is very large, preconditioned block Krylov space solvers are often considered as the method of choice. The purpose of this paper is thus to present iterative methods based on block restarted GMRES that allow variable preconditioning for the solution of linear systems with multiple right-hand sides. The use of flexible methods is especially of interest when approximate possibly iterative solvers are considered in the preconditioning phase. First we introduce a new variant of block flexible restarted GMRES that includes a strategy for detecting when a linear combination of the systems has approximately converged. This explicit block size reduction is often called deflation. We analyze the main properties of this flexible method based on deflation and notably prove that the Frobenius norm of the block residual is always nonincreasing. We also present a flexible variant based on both deflation and truncation to especially be used in case of limited memory. Finally we illustrate the numerical behavior of these flexible block methods for large industrial simulations arising in geophysics, where indefinite linear systems of size up to 1 billion unknowns with multiple right-hand sides have been successfully solved in a parallel distributed memory environment.

Decomposition and regularization of nonlinear anisotropic curl‐curl DAEs

PurposeThe space discretization of eddy‐current problems in the magnetic vector potential formulation leads to a system of differential‐algebraic equations. They are typically time discretized by an implicit method. This requires the solution of large linear systems in the Newton iterations. The authors seek to speed up this procedure. In most relevant applications, several materials are non‐conducting and behave linearly, e.g. air and insulation materials. The corresponding matrix system parts remain constant but are repeatedly solved during Newton iterations and time‐stepping routines. The paper aims to exploit invariant matrix parts to accelerate the system solution.Design/methodology/approachFollowing the principle “reduce, reuse, recycle”, the paper proposes a Schur complement method to precompute a factorization of the linear parts. In 3D models this decomposition requires a regularization in non‐conductive regions. Therefore, the grad‐div regularization is revisited and tailored such that it takes anisotropies into account.FindingsThe reduced problem exhibits a decreased effective condition number. Thus, fewer preconditioned conjugate gradient iterations are necessary. Numerical examples show a decrease of the overall simulation time, if the step size is small enough. 3D simulations with large time step sizes might not benefit from this approach, because the better condition does not compensate for the computational costs of the direct solvers used for the Schur complement. The combination of the Schur approach with other more sophisticated preconditioners or multigrid solvers is subject to current research.Originality/valueThe Schur complement method is adapted for the eddy‐current problem. Therefore, a new partitioning approach into linear/non‐linear and static/dynamic domains is proposed. Furthermore, a new variant of the grad‐div gauging is introduced that allows for anisotropies and enables the Schur complement method in 3D.

Deformable 3D Shape Registration Based on Local Similarity Transforms

AbstractIn this paper, a new method for deformable 3D shape registration is proposed. The algorithm computes shape transitions based on local similarity transforms which allows to model not only as‐rigid‐as‐possible deformations but also local and global scale. We formulate an ordinary differential equation (ODE) which describes the transition of a source shape towards a target shape. We assume that both shapes are roughly pre‐aligned (e.g., frames of a motion sequence). The ODE consists of two terms. The first one causes the deformation by pulling the source shape points towards corresponding points on the target shape. Initial correspondences are estimated by closest‐point search and then refined by an efficient smoothing scheme. The second term regularizes the deformation by drawing the points towards locally defined rest positions. These are given by the optimal similarity transform which matches the initial (undeformed) neighborhood of a source point to its current (deformed) neighborhood. The proposed ODE allows for a very efficient explicit numerical integration. This avoids the repeated solution of large linear systems usually done when solving the registration problem within general‐purpose non‐linear optimization frameworks. We experimentally validate the proposed method on a variety of real data and perform a comparison with several state‐of‐the‐art approaches.

On the generation of Krylov subspace bases

Many problems in scientific computing involving a large sparse square matrix A are solved by Krylov subspace methods. This includes methods for the solution of large linear systems of equations with A, for the computation of a few eigenvalues and associated eigenvectors of A, and for the approximation of nonlinear matrix functions of A. When the matrix A is non-Hermitian, the Arnoldi process commonly is used to compute an orthonormal basis for a Krylov subspace associated with A. The Arnoldi process often is implemented with the aid of the modified Gram–Schmidt method. It is well known that the latter constitutes a bottleneck in parallel computing environments, and to some extent also on sequential computers. Several approaches to circumvent orthogonalization by the modified Gram–Schmidt method have been described in the literature, including the generation of Krylov subspace bases with the aid of suitably chosen Chebyshev or Newton polynomials. We review these schemes and describe new ones. Numerical examples are presented.

Reactive Molecular Dynamics: Numerical Methods and Algorithmic Techniques

Modeling atomic and molecular systems requires computation-intensive quantum mechanical methods such as, but not limited to, density functional theory [R. A. Friesner, Proc. Natl. Acad. Sci. USA, 102 (2005), pp. 6648–6653]. These methods have been successful in predicting various properties of chemical systems at atomistic scales. Due to the inherent nonlocality of quantum mechanics, the scalability of these methods ranges from O($N^3$) to O($N^7$) depending on the method used and approximations involved. This significantly limits the size of simulated systems to a few thousand atoms, even on large scale parallel platforms. On the other hand, classical approximations of quantum systems, although computationally (relatively) easy to implement, yield simpler models that lack essential chemical properties such as reactivity and charge transfer. The recent work of van Duin et al. [J. Phys. Chem. A, 105 (2001), pp. 9396–9409] overcomes the limitations of nonreactive classical molecular dynamics (MD) approximations by carefully incorporating limited nonlocality (to mimic quantum behavior) through an empirical bond order potential. This reactive classical MD method, called ReaxFF, achieves essential quantum properties, while retaining the computational simplicity of classical MD, to a large extent. Implementation of reactive force fields presents significant algorithmic challenges. Since these methods model bond breaking and formation, efficient implementations must rely on complex dynamic data structures. Charge transfer in these methods is accomplished by minimizing electrostatic energy through charge equilibration. This requires the solution of large linear systems ($10^8$ degrees of freedom and beyond) with shielded electrostatic kernels at each time-step. Individual time-steps are themselves typically in the range of tenths of femtoseconds, requiring optimizations within and across time-steps to scale simulations to nanoseconds and beyond, where interesting phenomena may be observed. In this paper, we present implementation details of sPuReMD (serial Purdue reactive molecular dynamics program), a unique reactive classical MD code. We describe various data structures, and the charge equilibration solver at the core of the simulation engine. This Krylov subspace solver relies on a preconditioner based on incomplete LU factorization with thresholds (ILUT), specially targeted to our application. We comprehensively validate the performance and accuracy of sPuReMD on a variety of hydrocarbon systems. In particular, we show excellent per-time-step time, linear time scaling in system size, and a low memory footprint. sPuReMD is a freely distributed software with GPL and is currently being used to model diverse systems ranging from oxidative stress in biomembranes to strain relaxation in Si-Ge nanorods.