Block Jacobi Preconditioner Research Articles

Fast finite element electrostatic analysis with domain decomposition method

Accurate simulation of electrostatic field in electronic devices is very important to improve their performance. When the traditional finite element method is used to analyze the electrostatic field of large and complex structures, the calculation speed is slow and the computing resources are greatly consumed. Therefore, in this paper, we investigate electrostatic analysis with a non-overlapping domain decomposition method based on the scalar finite element method under a novel transmission condition. In order to improve the efficiency of matrix solving and reduce the memory consumption, the block Jacobi preconditioner is introduced. In addition, the multifrontal block incomplete Cholesky preconditioner is utilized to further improve the computational performance based on the block Jacobi preconditioner. Several numerical examples are simulated and compared with the analytical solutions and the commercial software Maxwell 3D. The results demonstrate the accuracy and efficiency of the proposed domain decomposition method.

Deflation techniques applied on mixed model equations

In this paper, we consider a block Jacobi preconditioner and various deflation techniques applied in the Deflated Preconditioned Conjugate Gradient (DPCG) method for solving a sparse system of linear equations derived from a statistical linear mixed model that analyses simultaneously phenotypic and pedigree information of genotyped and ungenotyped animals with Single Polymorphism Nucleotide genotypes of genotyped animals. In livestock production systems, evaluating the genetic merit of the animals through such a model is a key process to ensure an improvement of animals for some characteristics of interest at each generation. First, we propose to define the deflation vectors using a subdomain deflation approach that considers some biological properties of the genotypes. Using simulated data, this approach reduces the number of iterations by up to 87% in comparison to a Preconditioned Conjugate Gradient method with a Jacobi preconditioner. Furthermore, compared to a DPCG method with same number of subdomains but defined randomly, this approach reduces the number of iterations by up to 20% for the same computational costs of one DPCG iteration. The properties of the resulting systems show that this approach annihilates the largest eigenvalues of the preconditioned coefficient matrix. Second, we propose the use of solution vectors of 12 systems of equations that include between 0.25% and 3% less data, as deflation vectors. For reducing the computational costs, we also consider a Proper Orthogonal Decomposition-reduced set of these 12 vectors. The properties of the resulting systems show that this recycling information approach annihilates the smallest eigenvalues of the preconditioned coefficient matrix, and results in a reduction of up to 39% in comparison to the PCG method. Finally, based on our experiment, the combination of the subdomain deflation approach relying on biological properties and of the POD-based approach to recycle previous solution vectors, for defining the deflation vectors, results in annihilating both the smallest and largest eigenvalues, and in a reduction of up to 88 % of the number of iterations in comparison to the PCG method.

Comparison of CBFM-Enhanced Iterative Methods for MoM-Based Finite Antenna Array Analysis

In this article, we compare different iterative techniques enhanced by the CBFM, which are used to analyze finite arrays of disjoint antenna elements. These are based on the stationary-type methods (Jacobi, Gauss–Seidel, and macroblock Jacobi), the nonstationary GMRES, and the hybrid alternating GMRES-Jacobi (AGJ) method that combines these two types. In each iteration, the reduced CBFM system is constructed based on the previous iterates, the solution of which is used to update the solution vector in the next iteration with improved accuracy. In this way, the convergence of the classical iterative techniques can be greatly improved. The convergence rates and computational costs of the CBFM-enhanced iterative methods are analyzed by considering several MoM-based problems. The GMRES-based method, which employs the block-Jacobi preconditioner, outperforms the other methods when the MoM matrix is ill-conditioned. For well-conditioned MoM matrices with reduced diagonal dominance due to the increased presence of the interelement coupling effects, the AGJ method or the methods based on the stationary-type iterations may require smaller computational effort to converge to the desired solution accuracy in comparison to the GMRES-based approach.

An adapted deflated conjugate gradient solver for robust extended/generalised finite element solutions of large scale, 3D crack propagation problems

An adapted deflation preconditioner is employed to accelerate the solution of linear systems resulting from the discretisation of fracture mechanics problems with well-conditioned extended/generalised finite elements. The deflation space typically used for linear elasticity problems is enriched with additional vectors, accounting for the enrichment functions used, thus effectively removing low frequency components of the error. To further improve performance, deflation is combined, in a multiplicative way, with a block-Jacobi preconditioner, which removes high frequency components of the error as well as near-linear dependencies introduced by enrichment. The resulting scheme is tested on a series of non-planar crack propagation problems and compared to alternative linear solvers in terms of performance.

A Comparative Study of Block Incomplete Sparse Approximate Inverses Preconditioning on Tesla K20 and V100 GPUs

Incomplete Sparse Approximate Inverses (ISAI) has shown some advantages over sparse triangular solves on GPUs when it is used for the incomplete LU based preconditioner. In this paper, we extend the single GPU method for Block–ISAI to multiple GPUs algorithm by coupling Block–Jacobi preconditioner, and introduce the detailed implementation in the open source numerical package PETSc. In the experiments, two representative cases are performed and a comparative study of Block–ISAI on up to four GPUs are conducted on two major generations of NVIDIA’s GPUs (Tesla K20 and Tesla V100). Block–Jacobi preconditioning with Block–ISAI (BJPB-ISAI) shows an advantage over the level-scheduling based triangular solves from the cuSPARSE library for the cases, and the overhead of setting up Block–ISAI and the total wall clock times of GMRES is greatly reduced using Tesla V100 GPUs compared to Tesla K20 GPUs.

Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software

The use of mixed precision in numerical algorithms is a promising strategy for accelerating scientific applications. In particular, the adoption of specialized hardware and data formats for low-precision arithmetic in high-end GPUs (graphics processing units) has motivated numerous efforts aiming at carefully reducing the working precision in order to speed up the computations. For algorithms whose performance is bound by the memory bandwidth, the idea of compressing its data before (and after) memory accesses has received considerable attention. One idea is to store an approximate operator–like a preconditioner–in lower than working precision hopefully without impacting the algorithm output. We realize the first high-performance implementation of an adaptive precision block-Jacobi preconditioner which selects the precision format used to store the preconditioner data on-the-fly, taking into account the numerical properties of the individual preconditioner blocks. We implement the adaptive block-Jacobi preconditioner as production-ready functionality in the Ginkgo linear algebra library, considering not only the precision formats that are part of the IEEE standard, but also customized formats which optimize the length of the exponent and significand to the characteristics of the preconditioner blocks. Experiments run on a state-of-the-art GPU accelerator show that our implementation offers attractive runtime savings.

Replicated Computational Results (RCR) Report for “Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software”

The article by Flegar et al. titled “Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software” presents a novel, practical implementation of an adaptive precision block-Jacobi preconditioner. Performance results using state-of-the-art GPU architectures for the block-Jacobi preconditioner generation and application demonstrate the practical usability of the method, compared to a traditional full-precision block-Jacobi preconditioner. A production-ready implementation is provided in the Ginkgo numerical linear algebra library. In this report, the Ginkgo library is reinstalled and performance results are generated to perform a comparison to the original results when using Ginkgo’s Conjugate Gradient solver with either the full or the adaptive precision block-Jacobi preconditioner for a suite of test problems on an NVIDIA GPU accelerator. After completing this process, the published results are deemed reproducible.

Developing a Multi-GPU-Enabled Preconditioned GMRES with Inexact Triangular Solves for Block Sparse Matrices

Solving triangular systems is the building block for preconditioned GMRES algorithm. Inexact preconditioning becomes attractive because of the feature of high parallelism on accelerators. In this paper, we propose and implement an iterative, inexact block triangular solve on multi-GPUs based on PETSc’s framework. In addition, by developing a distributed block sparse matrix-vector multiplication procedure and investigating the optimized vector operations, we form the multi-GPU-enabled preconditioned GMRES with the block Jacobi preconditioner. In the implementation, the GPU-Direct technique is employed to avoid host-device memory copies. The preconditioning step used by PETSc’s structure and the cuSPARSE library are also investigated for performance comparisons. The experiments show that the developed GMRES with inexact preconditioning on 8 GPUs can achieve up to 4.4x speedup over the CPU-only implementation with exact preconditioning using 8 MPI processes.

Advantages of static condensation in implicit compressible Navier–Stokes DGSEM solvers

We consider implicit time-marching schemes for the compressible Navier-Stokes equations, discretised using the Discontinuous Galerkin Spectral Element Method with Gauss-Lobatto nodal points (GL-DGSEM). We compare classic implicit strategies for the full Jacobian system to our recently developed static condensation technique for GL-DGSEM Rueda-Ramírez et al.(2019), A Statically Condensed Discontinuous Galerkin Spectral Element Method on Gauss-Lobatto Nodes for the Compressible Navier-Stokes Equations [1]. The Navier-Stokes system is linearised using a Newton-Raphson method and solved using an iterative preconditioned-GMRES solver. Both the full and statically condensed systems benefit from a Block-Jacobi preconditioner.We include theoretical estimates for the various costs involved (i.e. calculation of full and condensed Jacobians, factorising and inverting the preconditioners, GMRES steps and overall costs) to clarify the advantages of using static condensation in GL-DGSEM, for varying polynomial orders. These estimates are then examined for a steady three-dimensional manufactured solution problem and for an two-dimensional unsteady laminar flow over a NACA0012 airfoil. In all cases, we test the schemes for high polynomial orders, which range from 2 to 8 for a manufactured solution case and from 2 to 5 for the NACA0012 airfoil. The statically condensed system shows computational savings, which relate to the smaller system size and cheaper Block-Jacobi preconditioner with smaller blocks and better polynomial scaling, when compared to the preconditioned full Jacobian system (not condensed). The advantage of using static condensation is more noticeable for higher polynomial orders.

MPI+OpenMP реализация метода сопряженных градиентов с предобусловливателем блочного Якоби IC1

MPI+OpenMP реализация метода сопряженных градиентов с предобусловливателем блочного Якоби IC1

Recursively preconditioned hierarchical interpolative factorization for elliptic partial differential equations

The hierarchical interpolative factorization for elliptic partial differential equations is a fast algorithm for approximate sparse matrix inversion in linear or quasilinear time. Its accuracy can degrade, however, when applied to strongly ill-conditioned problems. Here, we propose a simple modification that can significantly improve the accuracy at no additional asymptotic cost: applying a block Jacobi preconditioner before each level of skeletonization. This dramatically limits the impact of the underlying system conditioning and enables the construction of robust and highly efficient preconditioners even at quite modest compression tolerances. Numerical examples demonstrate the performance of the new approach.

Iteration-fusing conjugate gradient for sparse linear systems with MPI + OmpSs

In this paper, we target the parallel solution of sparse linear systems via iterative Krylov subspace-based method enhanced with a block-Jacobi preconditioner on a cluster of multicore processors. In order to tackle large-scale problems, we develop task-parallel implementations of the preconditioned conjugate gradient method that improve the interoperability between the message-passing interface and OmpSs programming models. Specifically, we progressively integrate several communication-reduction and iteration-fusing strategies into the initial code, obtaining more efficient versions of the method. For all these implementations, we analyze the communication patterns and perform a comparative analysis of their performance and scalability on a cluster consisting of 32 nodes with 24 cores each. The experimental analysis shows that the techniques described in the paper outperform the classical method by a margin that varies between 6 and 48%, depending on the evaluation.

Parallelization of the solve phase in a task-based Cholesky solver using a sequential task flow model

We describe the parallelization of the solve phase in the sparse Cholesky solver SpLLT when using a sequential task flow model. In the context of direct methods, the solution of a sparse linear system is achieved through three main phases: the analyse, the factorization and the solve phases. In the last two phases, which involve numerical computation, the factorization corresponds to the most computationally costly phase, and it is therefore crucial to parallelize this phase in order to reduce the time-to-solution on modern architectures. As a consequence, the solve phase is often not as optimized as the factorization in state-of-the-art solvers, and opportunities for parallelism are often not exploited in this phase. However, in some applications, the time spent in the solve phase is comparable to or even greater than the time for the factorization, and the user could dramatically benefit from a faster solve routine. This is the case, for example, for a conjugate gradient (CG) solver using a block Jacobi preconditioner. The diagonal blocks are factorized once only, but their factors are used to solve subsystems at each CG iteration. In this study, we design and implement a parallel version of a task-based solve routine for an OpenMP version of the SpLLT solver. We show that we can obtain good scalability on a multicore architecture enabling a dramatic reduction of the overall time-to-solution in some applications.

Accurate and fast three-dimensional free vibration analysis of large complex structures using the finite element method

Free vibration analysis of large complex structures with improved Lanczos iteration method is investigated through the finite element method. An important step in the traditional Lanczos iteration method for large generalized eigenvalue problems arising from free vibration analysis is that large-scale sparse linear system must be repeatedly solved within each step of the Lanczos iteration, which generally accounts for 70% to 90% of the calculation time of the eigenvalue problems. In the improved Lanczos iteration method given here, three-level preconditioner is proposed and utilized to improve the computational performance of Lanczos iteration. In this way, the multifrontal block incomplete Cholesky (MFBIC) preconditioner and p-type multigrid preconditioner based on higher-order hierarchical basis functions are proposed and utilized to improve the computational performance and decrease the memory requirements. To further reduce the memory consumption, the block Jacobi preconditioner is introduced in the Lanczos iteration, and a new three-level preconditioner that combines MFBIC, p-type multigrid and block Jacobi is proposed. The accuracy and performance of the improved method has been compared with exact solution, basic method and ANSYS. It is shown that this free vibration analysis method can be extremely useful to design structure dynamic characteristics of large complex structures.

Toward a modular precision ecosystem for high-performance computing

With the memory bandwidth of current computer architectures being significantly slower than the (floating point) arithmetic performance, many scientific computations only leverage a fraction of the computational power in today’s high-performance architectures. At the same time, memory operations are the primary energy consumer of modern architectures, heavily impacting the resource cost of large-scale applications and the battery life of mobile devices. This article tackles this mismatch between floating point arithmetic throughput and memory bandwidth by advocating a disruptive paradigm change with respect to how data are stored and processed in scientific applications. Concretely, the goal is to radically decouple the data storage format from the processing format and, ultimately, design a “modular precision ecosystem” that allows for more flexibility in terms of customized data access. For memory-bounded scientific applications, dynamically adapting the memory precision to the numerical requirements allows for attractive resource savings. In this article, we demonstrate the potential of employing a modular precision ecosystem for the block-Jacobi preconditioner and the PageRank algorithm—two applications that are popular in the communities and at the same characteristic representatives for the field of numerical linear algebra and data analytics, respectively.

Scalable tensor-product preconditioners for high-order finite-element methods: Scalar equations

We present a tensor-product-based preconditioner for high-order discontinuous-Galerkin (DG) discretizations. The preconditioner is based on approximating the block diagonal of the Jacobian matrix corresponding to element-wise coupling with the sum of tensor products of small one-dimensional matrices. The preconditioner is obtained through an algebraic procedure which minimizes the error between the tensor-product approximation and the exact elemental block Jacobian.Inverting the full elemental block Jacobian requires O(Nd) memory storage and O(N2d) operations, while applying its inverse requires O(Nd) operations per degree of freedom, where N is the solution order and d is the dimension of the problem. Thus, traditional block preconditioners become impractical with increasing dimension, d, and order, N. The cost of forming, storing, or applying the current tensor-product-based preconditioner scales linearly with solution order (O(N)) per degree of freedom for arbitrary number of dimensions. Furthermore, the tensor-product-based preconditioner recovers the exact block-Jacobi preconditioner in the case of constant-coefficient scalar problems on right parallelepiped elements. Numerical results demonstrate the effectiveness of the preconditioner for solving 4D (3D-space+time) DG discretizations of scalar advection-diffusion problems up to 32nd order.

“Traveling-wave tube mechanics simulator suite” a CAD/CAE integrated rapid redesign system of vibration analysis for traveling-wave tubes

Traveling-wave tubes (TWTs) are popularly used as the microwave power amplifier in civil and military applications and have a high mechanical sensitivity. It is necessary to understand their vibration characteristics under all operating conditions. To address this issue, a CAD/CAE integrated rapid redesign system of vibration analysis with unified data architecture (UDA) has been developed and presented in this paper. The UDA is presented to make CAD and CAE be an organic entity. This system is composed of the following four layers: (i) CAD/CAE integrated design environment, (ii) parametric modeling and fast modeling platform with custom-built tools, (iii) accurate and fast finite element free vibration analysis solver based on rapid solving techniques of equation, and (iv) fast finite element random vibration analysis solver. The CAD/CAE integrated system concept is typically applied in loop design-analysis-redesign procedures. In this CAD/CAE integrated design system, the designers can freely and rapidly complete component design and performance analysis simultaneously instead of using two different software or two interface environment. To improve the accuracy and efficiency of large-scale finite element vibration analysis, especially for free vibration analysis, an improved Lanczos iteration method with a new three-level preconditioner is developed and pseudo excitation method is introduced to perform random vibration analysis. The new three-level preconditioner is composed of a multifrontal block incomplete cholesky preconditioner, a p-type multigrid preconditioner, and a block Jacobi preconditioner. The developed system has been applied to the design of TWTs and been found to shorten the design cycle significantly compared with ANSYS.

Variable-size batched Gauss–Jordan elimination for block-Jacobi preconditioning on graphics processors

Abstract In this work, we address the efficient realization of block-Jacobi preconditioning on graphics processing units (GPUs). This task requires the solution of a collection of small and independent linear systems. To fully realize this implementation, we develop a variable-size batched matrix inversion kernel that uses Gauss-Jordan elimination (GJE) along with a variable-size batched matrix–vector multiplication kernel that transforms the linear systems’ right-hand sides into the solution vectors. Our kernels make heavy use of the increased register count and the warp-local communication associated with newer GPU architectures. Moreover, in the matrix inversion, we employ an implicit pivoting strategy that migrates the workload (i.e., operations) to the place where the data resides instead of moving the data to the executing cores. We complement the matrix inversion with extraction and insertion strategies that allow the block-Jacobi preconditioner to be set up rapidly. The experiments on NVIDIA’s K40 and P100 architectures reveal that our variable-size batched matrix inversion routine outperforms the CUDA basic linear algebra subroutine (cuBLAS) library functions that provide the same (or even less) functionality. We also show that the preconditioner setup and preconditioner application cost can be somewhat offset by the faster convergence of the iterative solver.

Multi Space Reduced Basis Preconditioners for Large-Scale Parametrized PDEs

In this work we introduce a new two-level preconditioner for the efficient solution of large-scale linear systems arising from the discretization of parametrized PDEs. The proposed preconditioner combines in a multiplicative way a reduced basis solver, which plays the role of coarse component, and a “traditional” fine-grid preconditioner, such as one-level additive Schwarz, block Gauss--Seidel, or block Jacobi preconditioners. The coarse component is built upon a new multi space reduced basis (MSRB) method that we introduce for the first time in this paper, where a reduced basis space is built through the proper orthogonal decomposition algorithm at each step of the iterative method at hand, like the flexible GMRES method. MSRB strategy consists in building reduced basis spaces that are well suited to perform a single iteration, by addressing the error components which have not been treated yet. The Krylov iterations employed to solve the resulting preconditioned system target small tolerances with a very...

Approximate tensor-product preconditioners for very high order discontinuous Galerkin methods

In this paper, we develop a new tensor-product based preconditioner for discontinuous Galerkin methods with polynomial degrees higher than those typically employed. This preconditioner uses an automatic, purely algebraic method to approximate the exact block Jacobi preconditioner by Kronecker products of several small, one-dimensional matrices. Traditional matrix-based preconditioners require O(p2d) storage and O(p3d) computational work, where p is the degree of basis polynomials used, and d is the spatial dimension. Our SVD-based tensor-product preconditioner requires O(pd+1) storage, O(pd+1) work in two spatial dimensions, and O(pd+2) work in three spatial dimensions. Combined with a matrix-free Newton–Krylov solver, these preconditioners allow for the solution of DG systems in linear time in p per degree of freedom in 2D, and reduce the computational complexity from O(p9) to O(p5) in 3D. Numerical results are shown in 2D and 3D for the advection, Euler, and Navier–Stokes equations, using polynomials of degree up to p=30. For many test cases, the preconditioner results in similar iteration counts when compared with the exact block Jacobi preconditioner, and performance is significantly improved for high polynomial degrees p.