Block Incomplete LU Research Articles

A fine-grained block ILU scheme on regular structures for GPGPUs

Iterative methods based on block incomplete LU (BILU) factorization are considered highly effective for solving large-scale block-sparse linear systems resulting from coupled PDE systems with n equations. However, efforts on porting implicit PDE solvers to massively parallel shared-memory heterogeneous architectures, such as general-purpose graphics processing units (GPGPUs), have largely avoided BILU, leaving their enormous performance potential unfulfilled in many applications where the use of implicit schemes and BILU-type preconditioners/solvers is highly preferred. Indeed, strong inherent data dependency and high memory bandwidth demanded by block matrix operations render naive adoptions of existing sequential BILU algorithms extremely inefficient on GPGPUs. In this study, we present a fine-grained BILU (FGBILU) scheme which is particularly effective on GPGPUs. A straightforward one-sweep wavefront ordering is employed to resolve data dependency. Granularity is substantially refined as block matrix operations are carried out in a true element-wise approach. Particularly, the inversion of diagonal blocks, a well-known bottleneck, is accomplished by a parallel in-place Gauss–Jordan elimination. As a result, FGBILU is able to offer low-overhead concurrent computation at O(n2N2) scale on a 3D PDE domain with a linear scale of N. FGBILU has been implemented with both OpenACC and CUDA and tested as a block-sparse linear solver on a structured 3D grid. While FGBILU remains mathematically identical to sequential global BILU, numerical experiments confirm its exceptional performance on an Nvidia GPGPU.

An Inverse-Based Multifrontal Block Incomplete LU Preconditioner for the 3-D Finite-Element Eigenvalue Analysis of Lossy Slow-Wave Structures

In this paper, a novel inverse-based multifrontal block incomplete LU preconditioner is proposed, which is derived from the complete multifrontal method and the inverse-based dropping strategy. The new preconditioner is used to solve the large-scale complex unsymmetrical linear systems arising from the 3-D finite-element (FE) eigenvalue analysis of lossy slow-wave structures (SWSs) of traveling-wave tubes (TWTs). An update-matrix-free multifrontal method and an adaptive super-block incomplete factorization framework are proposed and utilized to improve the computational performance and decrease the memory requirements of this preconditioner. Moreover, the inverse-based dropping strategy is introduced to the incomplete multifrontal method for the first time making this preconditioner more effective. By utilizing the proposed preconditioner, the cold characteristic parameters of SWSs can be calculated quickly and accurately. In the simulations of many SWSs, the 3-D FE eigenvalue analysis of lossy SWSs based on this preconditioner has shown the high-efficiency computational performance and the low memory requirements. It is shown that this novel preconditioner is very useful to design lossy SWSs for high-efficiency TWTs.

An advancement in iterative solution schemes for three-dimensional, two-fluid modeling of two-phase flow in PWR fuel bundles

This paper outlines a fully three-dimensional two-fluid one-pressure model with a semi-implicit finite difference scheme coupled with heat conduction which can be applicable to thermal non-equilibrium two-phase flow field in subchannel geometry of Pressurized Water Reactors (PWR). The system of equations was linearized using the Newton–Raphson method and was collapsed into the pressure equations forming a system of the Poisson type. Then, two-phase flow modeling was combined with Krylov methods as advanced computing techniques to investigate the feasibility of implementing preconditioned Krylov subspace solvers as the numerical scheme to solve pressure equations. Six popular Krylov subspace solvers were considered: GMRES, FGMRES, DQGMRES, CGNR, BCG, and TFQMR combined with the block incomplete LU factorization with a dual truncation strategy (BILUT) preconditioner. These proposed iterative solvers were applied to the constructed linear pressure equations in the inner iteration in combination with the outer-Raphson iteration loop. Evaluation was performed in two stages. First, two-fluid numerical scheme capability was evaluated against OECD/NRC NUPEC PWR Bundle tests (PSBT Benchmark). The results for steady-state (PSBT) bundle show that an overall agreement can be found. At the second stage, convergency, stability, and accuracy of the proposed schemes were studied based on PSBT steady-state data through a comparison of utilized Krylov solvers and the direct inversion method as the pressure solution modeling options. It was found that the Krylov solvers do not introduce deviations to the two-phase flow results. Moreover, the convergence history of candidate Krylov solvers demonstrated that GMRES, FGMRES, and DQGMRES have a more efficient and stable convergence performance.

Block compressed storage and computation in the large-scale reservoir simulation

Abstract The block compressed storage and solving methods in reservoir simulation are explored to resolve large sparse linear equations emerging in three-phase black oil model. In this paper, the active nodes are compressed firstly, and then the block main diagonal elements, nonzero block elements in lower triangular and upper triangular are separately stored into three real arrays. Owing to the coefficient matrix arising in the reservoir simulation being symmetrical, the addresses of block elements in lower triangular are deposited in three integer arrays. This block compressed storage method can save lots of memory and reduce the search frequencies of non-zero elements. On the basis of such compression storage, the block incomplete LU preconditioned generalized minimal residual method (GMRES) is adopted to solve the equations, showing that it is an effective method possessing fast convergence and good stability. Testing of black oil model example reveals that the block compressed storage and solving methods are effective in solving the large-scale reservoir simulation.

A scalable Helmholtz solver in GRAPES over large‐scale multicore cluster

SUMMARYThis paper discusses performance optimization on the dynamical core of global numerical weather prediction model in Global/Regional Assimilation and Prediction System (GRAPES). GRAPES is a new generation of numerical weather prediction system developed and currently used by Chinese Meteorology Administration. The computational performance of the dynamical core in GRAPES relies on the efficient solution of three‐dimensional Helmholtz equations, which lead to large‐scale and sparse linear systems formulated by the discretization in space and time. We choose generalized conjugate residual (GCR) algorithm to solve the corresponding linear systems and further propose algorithm optimizations for large‐scale parallelism in two aspects: (i) reduction of iteration number for solution and (ii) performance enhancement of each GCR iteration. The reduction of iteration number is achieved by advanced preconditioning techniques, combining block incomplete LU factorization‐k preconditioner over 7‐diagonals of the coefficient matrix with the restricted additive Schwarz method effectively . The improvement for GCR iteration is to reduce the global communication operations by refactoring the GCR algorithm, which decreases the communication overhead over large number of cores. Performance evaluation on the Tianhe‐1A system shows that the new preconditioning techniques reduce almost one‐third iterations for solving the linear systems, the proposed methods can obtain 25% performance improvement on average compared with the original version of Helmholtz solver in GRAPES, and the speedup with our algorithms can reach 10 using 2048 cores compared with 256 cores. Copyright © 2013 John Wiley & Sons, Ltd.

A supercoarsening multigrid method for poroelasticity in 3D coupled flow and geomechanics modeling

The Galerkin finite-element discretization of the force balance equation typically leads to large linear systems for geomechanical problems with realistic dimensions. In iteratively coupled flow and geomechanics modeling, a large linear system is solved at every timestep often multiple times during coupling iterations. The iterative solution of the linear system stemming from the poroelasticity equations constitutes the most time-consuming and memory-intensive component of coupled modeling. Block Jacobi, LSOR, and Incomplete LU factorization are popular preconditioning techniques used for accelerating the iterative solution of the poroelasticity linear systems. However, the need for more effective, efficient, and robust iterative solution techniques still remains especially for large coupled modeling problems requiring the solution of the poroelasticity system for a large number of timesteps. We developed a supercoarsening multigrid method (SCMG) which can be multiplicatively combined with commonly used preconditioning techniques. SCMG has been tested on a variety of coupled flow and geomechanics problems involving single-phase depletion and multiphase displacement of in-situ hydrocarbons, CO2 injection, and extreme material property contrasts. Our analysis indicates that the SCMG consistently improves the convergence properties of the linear systems arising from the poroelasticity equations, and thus, accelerates the coupled simulations for all cases subject to investigation. The joint utilization of the two-level SCMG with the ILU1 preconditioner emerges as the most optimal preconditioning/iterative solution strategy in a great majority of the problems evaluated in this work. The BiCGSTAB iterative solver converges more rapidly compared to PCG in a number of test cases, in which various SCMG-accelerated preconditioning strategies are applied to both iterators.

Newton-GMRES Preconditioning for Discontinuous Galerkin Discretizations of the Navier–Stokes Equations

We study preconditioners for the iterative solution of the linear systems arising in the implicit time integration of the compressible Navier-Stokes equations. The spatial discretization is carried out using a discontinuous Galerkin method with fourth order polynomial interpolations on triangular elements. The time integration is based on backward difference formulas resulting in a nonlinear system of equations which is solved at each timestep. This is accomplished using Newton's method. The resulting linear systems are solved using a preconditioned GMRES iterative algorithm. We consider several existing preconditioners such as block Jacobi and Gauss-Seidel combined with multilevel schemes which have been developed and tested for specific applications. While our results are consistent with the claims reported, we find that these preconditioners lack robustness when used in more challenging situations involving low Mach numbers, stretched grids, or high Reynolds number turbulent flows. We propose a preconditioner based on a coarse scale correction with postsmoothing based on a block incomplete LU factorization with zero fill-in (ILU0) of the Jacobian matrix. The performance of the ILU0 smoother is found to depend critically on the element numbering. We propose a numbering strategy based on minimizing the discarded fill-in in a greedy fashion. The coarse scale correction scheme is found to be important for diffusion dominated problems, whereas the ILU0 preconditioner with the proposed ordering is effective at handling the convection dominated case. While little can be said in the way of theoretical results, the proposed preconditioner is shown to perform remarkably well for a broad range of representative test problems. These include compressible flows ranging from very low Reynolds numbers to fully turbulent flows using the Reynolds averaged Navier-Stokes equations discretized on highly stretched grids. For low Mach number flows, the proposed preconditioner is more than one order of magnitude more efficient than the other preconditioners considered.

Distributed block independent set algorithms and parallel multilevel ILU preconditioners

We present a class of parallel preconditioning strategies utilizing multilevel block incomplete LU (ILU) factorization techniques to solve large sparse linear systems. The preconditioners are constructed by exploiting the concept of block independent sets (BISs). Two algorithms for constructing BISs of a sparse matrix in a distributed environment are proposed. We compare a few implementations of the parallel multilevel ILU preconditioners with different BIS construction strategies and different Schur complement preconditioning strategies. We also use some diagonal thresholding and perturbation strategies for the BIS construction and for the last level Schur complement ILU factorization. Numerical experiments indicate that our domain-based parallel multilevel block ILU preconditioners are robust and efficient.

Parallel Multigrid Preconditioner for the Cardiac Bidomain Model

The bidomain equations are widely used for the simulation of electrical activity in cardiac tissue but are computationally expensive, limiting the size of the problem which can be modeled. The purpose of this study is to determine more efficient ways to solve the elliptic portion of the bidomain equations, the most computationally expensive part of the computation. Specifically, we assessed the performance of a parallel multigrid (MG) preconditioner for a conjugate gradient solver. We employed an operator splitting technique, dividing the computation in a parabolic equation, an elliptical equation, and a nonlinear system of ordinary differential equations at each time step. The elliptic equation was solved by the preconditioned conjugate gradient method, and the traditional block incomplete LU parallel preconditioner (ILU) was compared to MG. Execution time was minimized for each preconditioner by adjusting the fill-in factor for ILU, and by choosing the optimal number of levels for MG. The parallel implementation was based on the PETSc library and we report results for up to 16 nodes on a distributed cluster, for two and three dimensional simulations. A direct solver was also available to compare results for single processor runs. MG was found to solve the system in one third of the time required by ILU but required about 40% more memory. Thus, MG offered an attractive tradeoff between memory usage and speed, since its performance lay between those of the classic iterative methods (slow and low memory consumption) and direct methods (fast and high memory consumption). Results suggest the MG preconditioner is well suited for quickly and accurately solving the bidomain equations.

Accurate Numerical Modeling of the TARA Reflector System

The radiation pattern of the large parabolic reflectors of the Transportable Atmospheric RAdar system (TARA), developed at Delft University of Technology, has been accurately simulated. The electric field integral equation (EFIE) formulation has been applied to a model of the reflectors including the feed housing and supporting struts, discretised using the method of moments. Because the problem is electrically large (the reflector has a diameter of 33/spl lambda/) and nonsymmetrical, this lead to a badly conditioned linear system of approximately half a million unknowns. In order to solve this system, an iterative solver (generalized minimum residual method) was used, in combination with the multilevel fast multipole method. Because of the bad conditioning, the system could only be solved by using a huge preconditioner. A new block-incomplete LU preconditioner (ILU) algorithm has been employed to allow for efficient out-of-computer core memory preconditioning.

Parallel performance of block ILU preconditioners for a block-tridiagonal matrix

The parallelizable block ILU (incomplete LU) factorization preconditioners for a block-tridiagonal matrix have been recently proposed by the author. In this paper, we describe a parallelization of Krylov subspace methods with the block ILU factorization preconditioners on distributed-memory computers such as the Cray T3E, and then parallel performance results of a preconditioned Krylov subspace method are provided to evaluate the effectiveness and efficiency of the block ILU preconditioners on the Cray T3E.

A class of multilevel recursive incomplete LU preconditioning techniques

We introduce a class of multilevel recursive incomplete LU preconditioning techniques (RILUM) for solving general sparse matrices. This technique is based on a recursive two by two block incomplete LU factorization on the coefficient matrix. The coarse level system is constructed as an (approximate) Schur complement. A dynamic preconditioner is obtained by solving the Schur complement matrix approximately. The novelty of the proposed techniques is to solve the Schur complement matrix by a preconditioned Krylov subspace method. Such a reduction process is repeated to yield a multilevel recursive preconditioner.

Enhanced multi-level block ILU preconditioning strategies for general sparse linear systems

This paper introduces several strategies to deal with pivot blocks in multi-level block incomplete LU factorization (BILUM) preconditioning techniques. These techniques are aimed at increasing the robustness and controlling the amount of fill-ins of BILUM for solving large sparse linear systems when large-size blocks are used to form block-independent set. Techniques proposed in this paper include double-dropping strategies, approximate singular-value decomposition, variable size blocks and use of an arrowhead block submatrix. We point out the advantages and disadvantages of these strategies and discuss their efficient implementations. Numerical experiments are conducted to show the usefulness of the new techniques in dealing with hard-to-solve problems arising from computational fluid dynamics. In addition, we discuss the relation between multi-level ILU preconditioning methods and algebraic multi-level methods.

Hermite collocation solution of partial differential equations via preconditioned Krylov methods

We are concerned with the numerical solution of partial differential equations (PDEs) in two spatial dimensions discretized via Hermite collocation. To efficiently solve the resulting systems of linear algebraic equations, we choose a Krylov subspace method. We implement two such methods: Bi-CGSTAB [1] and GMRES [2]. In addition, we utilize two different preconditioners: one based on the Gauss–Seidel method with a block red-black ordering (RBGS); the other based upon a block incomplete LU factorization (ILU). Our results suggest that, at least in the context of Hermite collocation, the RBGS preconditioner is superior to the ILU preconditioner and that the Bi-CGSTAB method is superior to GMRES. © 2001 John Wiley & Sons, Inc. Numer Methods Partial Differential Eq 17:120–136, 2001

A Multilevel Dual Reordering Strategy for Robust Incomplete LU Factorization of Indefinite Matrices

A dual reordering strategy based on both threshold and graph reorderings is introduced to construct robust incomplete LU (ILU) factorization of indefinite matrices. The ILU matrix is constructed as a preconditioner for the original matrix to be used in a preconditioned iterative scheme. The matrix is first divided into two parts according to a threshold parameter to control diagonal dominance. The first part with large diagonal dominance is reordered using a graph-based strategy, followed by an ILU factorization. A partial ILU factorization is applied to the second part to yield an approximate Schur complement matrix. The whole process is repeated on the Schur complement matrix and continues for a few times to yield a multilevel ILU factorization. Analyses are conducted to show how the Schur complement approach removes small diagonal elements of indefinite matrices and how the stability of the LU factor affects the quality of the preconditioner. Numerical results are used to compare the new preconditioning strategy with two popular ILU preconditioning techniques and a multilevel block ILU threshold preconditioner.

Hermite collocation solution of partial differential equations via preconditioned Krylov methods

We are concerned with the numerical solution of partial differential equations (PDEs) in two spatial dimensions discretized via Hermite collocation. To efficiently solve the resulting systems of linear algebraic equations, we choose a Krylov subspace method. We implement two such methods: Bi-CGSTAB [1] and GMRES [2]. In addition, we utilize two different preconditioners: one based on the Gauss–Seidel method with a block red-black ordering (RBGS); the other based upon a block incomplete LU factorization (ILU). Our results suggest that, at least in the context of Hermite collocation, the RBGS preconditioner is superior to the ILU preconditioner and that the Bi-CGSTAB method is superior to GMRES. © 2001 John Wiley & Sons, Inc. Numer Methods Partial Differential Eq 17:120–136, 2001

Block ILU factorization preconditioners for a block-tridiagonal H-matrix

We propose new parallelizable block ILU (incomplete LU) factorization preconditioners for a nonsymmetric block-tridiagonal H-matrix. Theoretical properties of these block ILU preconditioners are compared with those of block ILU preconditioners for the corresponding comparison matrix. Numerical results of the BiCGSTAB using these block preconditioners are compared with those of the BiCGSTAB using a standard ILU factorization preconditioner to see the effectiveness of the block ILU preconditioners. Lastly, parallel computations of the block ILU preconditioners are carried out and evaluated on the Cray C90.

Block ILU Preconditioners for a Nonsymmetric Block-Tridiagonal M -Matrix

We propose block ILU (incomplete LU) factorization preconditioners for a nonsymmetric block-tridiagonal M-matrix whose computation can be done in parallel based on matrix blocks. Some theoretical properties for these block ILU factorization preconditioners are studied and then we describe how to construct them effectively for a special type of matrix. We also discuss a parallelization of the preconditioner solver step used in nonstationary iterative methods with the block ILU preconditioners. Numerical results of the right preconditioned BiCGSTAB method using the block ILU preconditioners are compared with those of the right preconditioned BiCGSTAB using a standard ILU factorization preconditioner to see how effective the block ILU preconditioners are.

Multilevel algorithms for 3D simulation of nonlinear elasticity problems

This study is devoted to the numerical solution of 3D elasticity problems in multilayer media. The problem is described by a coupled system of second-order nonlinear elliptic partial differential equations with strongly varying coefficients. The boundary value problem is discretized by trilinear finite elements. The goal of the paper is to analyze the performance of three hierarchical algorithms for the arisen discrete problems. The secant method is applied as a general outer nonlinear iterative procedure. The two-level block-size reduction block-incomplete LU (BSR BILU) algorithm, and the algebraic multilevel iteration (AMLI) algorithm are implemented as preconditioners to the linearized problems into the framework of the conjugate gradient method. The BSR BILU is a special case two-level algorithm while the AMLI preconditioner is based on a regular multilevel mesh refinement. A simple patched local refinement in combination with the BSR BILU algorithm is considered at the end, as an efficient approach for problems with localized zones of active interactions. The developed FEM codes are applied for 3D simulation of pile systems in weak multilayer soil media. The benchmark problem is taken from the real-life bridge engineering practice. The presented test data illustrate the abilities of the proposed methods and algorithms as well as the robustness of the codes.

A robust and efficient numerical model for analyses of density-dependent flow in porous media

A new computer code (MITSU3D) has been developed to simulate variable density-dependent flow in porous media. The techniques involve a control-volume discretization, Newton–Raphson linearization, and a flux limiter scheme. The assembled Jacobian matrices are solved by using preconditioned conjugate gradient-type methods employing a block-incomplete LU factorization and reduced system preconditioning. Numerical simulations involving contaminant migration in a two-dimensional cross-section show MITSU3D to be much faster and more robust than SUTRA. MITSU3D successfully avoids an “error-driven” contaminant plume instability in a coarse grid discretization and shows increased performance by a factor of five. The enhanced speed and robustness will enable us to examine much larger coupled problems than heretofore had been considered computationally practical.