Parallel Algebraic Multigrid Research Articles

A mass conserving arbitrary Lagrangian–Eulerian formulation for three‐dimensional multiphase fluid flows

AbstractThe arbitrary Lagrangian–Eulerian (ALE) framework presented in [Sahin and Guventurk, An arbitrary Lagrangian–Eulerian framework with exact mass conservation for the numerical simulation of 2D rising bubble problem.Int J Numer Methods Eng. 2017;112:2110–2134] has been extended to simulate three‐dimensional immiscible incompressible multiphase fluid flows. The div‐stable side centered unstructured finite volume formulation is used for the discretization of the incompressible Navier–Stokes equations. A novel kinematic boundary condition that satisfies the local discrete geometric conservation law (DGCL) is implemented to ensure the mass conservation of each species at the machine precision. The pressure field is treated to be discontinuous across the interface with the discontinuous treatment of density and viscosity. Surface tension force is treated as a tangent force and discretized in a semi‐implicit form. Two different approaches for the computation of unit normal vector have been implemented: the least squares biquadratic surface fitting (LSBSF) and the mean weighted by sine and edge length reciprocals (MWSELR). The resulting large system of algebraic equations is solved in a fully coupled manner in order to improve the time step restrictions. As a preconditioner, an approximate matrix factorization similar to that of the projection method is employed and the parallel algebraic multigrid solver BoomerAMG provided by the HYPRE library, which is accessed through the PETSc library, has been utilized for the scaled discrete Laplacian of pressure and the diagonal blocks of mesh deformation equations. The accuracy of the proposed method is initially validated on the static bubble problem, since the surface tension force is highly sensitive to the accurate evaluation of the unit normal vector and the inaccuracies significantly contribute to unphysical velocities, called parasitic currents. The calculations indicate that the parasitic currents can be reduced to machine precision for the MWSELR method. Then, the proposed approach is applied to the single bubble rising in a viscous quiescent liquid for both low and high density ratios. The calculations produce accurate predictions of the bubble shape, center of mass, rise velocity, and so forth. Furthermore, the mass of each species is conserved at machine precision and discontinuous pressure field is obtained in order to avoid errors due to the incompressibility restriction in the vicinity of liquid‐liquid interfaces at large density and viscosity ratios.

A monolithic fluid-structure interaction framework applied to red blood cells.

A parallel fully coupled (monolithic) fluid-structure interaction (FSI) algorithm has been applied to the deformation of red blood cells (RBCs) in capillaries, where cell deformability has significant effects on blood rheology. In the present FSI algorithm, fluid domain is discretized using the side-centered unstructured finite volume method based on the Arbitrary Lagrangian-Eulerian (ALE) formulation; meanwhile, solid domain is discretized with the classical Galerkin finite element formulation for the Saint Venant-Kirchhoff material in a Lagrangian frame. In addition, the compatible kinematic boundary condition is enforced at the fluid-solid interface in order to conserve the mass of cytoplasmic fluid within the red cell at machine precision. In order to solve the resulting large-scale algebraic linear systems in a fully coupled manner, a new matrix factorization is introduced similar to that of the projection method, and the parallel algebraic multigrid solver BoomerAMG is used for the scaled discrete Laplacian provided by the HYPRE library, which we access through the PETSc library. Three important physical parameters for the blood flow are simulated and analyzed: (1) the effect of capillary diameter, (2) the effect of red cell membrane thickness, and (3) the effect of red cell spacing (hematocrit). The numerical calculations initially indicate a shape deformation in which biconcave discoid shape changes to a parachute-like shape. Furthermore, the parachute-like cell shape in small capillaries undergoes a cupcake-shaped buckling instability, which has not been observed in the literature. The instability forms thin riblike features, and the red cell deformation is not axisymmetric but three-dimensional.

Accelerating algebraic multigrid solvers on NVIDIA GPUs

This paper presents the development of parallel algebraic multigrid solvers on NVIDIA GPUs. A classical algebraic multigrid solver and a smoothed aggregation algebraic multigrid solver are implemented. The W-cycle, F-cycle and V-cycle are investigated. Various coarsening strategies and interpolation operators are implemented. A group of smoothers are also provided. Numerical experiments show that the solution phase of our GPU-based algebraic solvers is up to 13 times faster than that of the sequential CPU-based algebraic multigrid solvers on our workstation.

Non-Galerkin Multigrid Based on Sparsified Smoothed Aggregation

Algebraic multigrid (AMG) methods are known to be efficient in solving linear systems arising from the discretization of partial differential equations and other related problems. These methods employ a hierarchy of representations of the problem on successively coarser meshes. The coarse-grid operators are usually defined by (Petrov--)Galerkin coarsening, which is a projection of the original operator using the restriction and prolongation transfer operators. Therefore, these transfer operators determine the sparsity pattern and operator complexity of the multigrid hierarchy. In many scenarios the multigrid operators tend to become much denser as the coarsening progresses. Such behavior is especially problematic in parallel AMG computations, where it imposes an expensive communication overhead. In this work we present a new algebraic technique for controlling the sparsity pattern of the operators in the AMG hierarchy, independently of the choice of the restriction and prolongation. Our method is based on the aggregation multigrid framework, and it “sparsifies” smoothed aggregation operators while preserving their right and left near null-spaces. Numerical experiments for problems of convection-diffusion and diffusion with discontinuous coefficients demonstrate the efficiency and potential of this approach.

An efficient parallel algebraic multigrid method for 3D injection moulding simulation based on finite volume method

Elapsed time is always one of the most important performance measures for polymer injection moulding simulation. Solving pressure correction equations is the most time-consuming part in the mould filling simulation using finite volume method with SIMPLE-like algorithms. Algebraic multigrid (AMG) is one of the most promising methods for this type of elliptic equations. It, thus, has better performance by contrast with some common one-level iterative methods, especially for large problems. And it is also suitable for parallel computing. However, AMG is not easy to be applied due to its complex theory and poor generality for the large range of computational fluid dynamics applications. This paper gives a robust and efficient parallel AMG solver, A1-pAMG, for 3D mould filling simulation of injection moulding. Numerical experiments demonstrate that, A1-pAMG has better parallel performance than the classical AMG, and also has algorithmic scalability in the context of 3D unstructured problems.

Algebraic multigrid techniques for discontinuous Galerkin methods with varying polynomial order

We present a parallel algebraic multigrid (AMG) algorithm for the implicit solution of the Darcy problem discretized by the discontinuous Galerkin (DG) method that scales optimally for regular and irregular meshes. The main idea centers on recasting the preconditioning problem so that existing AMG solvers for nodal lower order finite elements can be leveraged. This is accomplished by a transformation operator which maps the solution from a Lagrange basis representation to a Legendre basis representation. While this mapping function must be user supplied, we demonstrate how easily it can be constructed for somepopular finite element representations includingquadrilateral/hexahedral and triangular/tetrahedral DG formulations. Furthermore, we show that the mapping does not depend on the Jacobian transformation between reference and physical space and so it can be constructed with very limited mesh information. Parallel performance studies demonstrate the versatility of this approach.

Non-Galerkin Coarse Grids for Algebraic Multigrid

Algebraic multigrid (AMG) is a popular and effective solver for systems of linear equations that arise from discretized partial differential equations. While AMG has been effectively implemented on large scale parallel machines, challenges remain, especially when moving to exascale. In particular, stencil sizes (the number of nonzeros in a row) tend to increase further down in the coarse grid hierarchy, and this growth leads to more communication. Thus, as problem size increases and the number of levels in the hierarchy grows, the overall efficiency of the parallel AMG method decreases, sometimes dramatically. This growth in stencil size is due to the standard Galerkin coarse grid operator, $P^T A P$, where $P$ is the prolongation (i.e., interpolation) operator. For example, the coarse grid stencil size for a simple three-dimensional (3D) seven-point finite differencing approximation to diffusion can increase into the thousands on present day machines, causing an associated increase in communication costs...

An arbitrary Lagrangian–Eulerian formulation for solving moving boundary problems with large displacements and rotations

An Arbitrary Lagrangian–Eulerian (ALE) formulation based on the unstructured finite volume method is proposed for solving moving boundary problems with large displacements and rotations. The numerical method is based on the side-centered arrangement of the primitive variables that does not require any ad-hoc modifications in order to enhance pressure coupling. The continuity equation is satisfied within each element at machine precision and the summation of the continuity equations can be exactly reduced to the domain boundary, which is important for the global mass conservation. A special attention is given to construct an ALE algorithm obeying the discrete geometric conservation law (DGCL). The mesh deformation algorithm is based on the indirect Radial Basis Function (RBF) algorithm at each time level while avoiding remeshing in order to enhance numerical robustness. For the parallel solution of resulting large-scale algebraic equations in a fully coupled form, a matrix factorization is introduced similar to that of the projection method for the whole system and the parallel algebraic multigrid solver BoomerAMG is used for the scaled discrete Laplacian provided by the HYPRE library which we access through the PETSc library. The present numerical algorithm is initially validated for the decaying Taylor–Green vortex flow, the flow past an oscillating circular cylinder in a channel and the flow induced by an oscillating sphere in a cubic cavity. Then the numerical algorithm is applied to the numerical simulation of flow field around a pair of flapping Drosophila wings in hover flight. The time variation of the Eulerian coherent structures in the near wake is shown along with the aerodynamic loads.

Application of Parallel Algebraic Multigrid Algorithms in Geophysics

With the further development of the electromagnetic exploration technologies, the forward and inversion modeling of geophysical in three-dimensional numerical simulation fields is confronted with huge challenges. During the process of solving the partial differential equations, the methods of finite difference, finite element and volume element methods are usually adopted. For the complex topographic condition and geological structure, the conditions of the matrix formed finally will be very poor, seriously affecting the iterative and convergence rate in equation solution. In this paper, the algebraic multigrid preconditioned methods and conjugate gradient solution process are adopted to conduct parallel processing in combination of graphics processing unit (GPU), and the efficiency of DCs threedimensional forward modeling will be effectively improved.

Efficient Parallel AMG Methods for Approximate Solutions of Linear Systems in CFD Applications

In engineering practice quite simple algebraic multigrid (AMG) methods with inexpensive coarse-grid selection and constant interpolation are frequently found to be the method of choice to solve systems of linear equations where accurate solutions of these systems are not needed. In a case study we analyze the poor parallel performance of such algorithms on common multicore cluster architectures and suggest using a class of algorithms with hybrid coarse-grid selection featuring a superior parallel performance.

Parallel Unstructured-Solver Methods for Simulation of Complex Giant Reservoirs

Summary The major issues for parallel solvers in a modern reservoir simulator are robustness, scalability, efficiency, and flexibility. There is significant interest in running fast field-scale simulations for complex giant Middle Eastern reservoirs, which will require tens of millions to hundreds of millions of grid cells to give reasonable resolution. At the same time, significant geologic complexity will require the treatment of dual-permeability regions, faulting and fractures, and high variations of reservoir and fluid properties. Of course, the methods should also work well for extracted-sector simulation with local grid refinements in both the structured and unstructured discretization. The preconditioning methods considered in this work include both the single-stage and multistage frameworks. In the single-stage framework, a novel method is considered in addition to the well-known variants of incomplete lower-upper (ILU) factorizations [ILU0, ILU(k), and ILUT]. The new method is a highly parallel method, which, in this paper, will be referred to as the unstructured line-solve power-series (LSPS) method. The method will be discussed and contrasted in light of key issues for parallel linear solvers. The unstructured LSPS has certain interesting properties in the parallel construct, which make it a highly effective component. The multistage method researched in this work is of the constraint pressure residual (CPR) framework. The method uses approximate pressure solve as the first-stage preconditioning to the full-system preconditioning. A number of original adaptations based on this concept were researched. Here, the use of the parallel algebraic multigrid (PAMG) method and other single-level methods mentioned previously in combinations within the multistage CPR framework were explored. Certain methods constructed in this way are found to be highly efficient, scalable, and robust. The methods developed are discussed, and several test problems are included, in this paper. The largest simulation model tested to date using these solver methods is a 172-million-cell full-field model of a supergiant carbonate complex with more than 3,000 wells and 60 years of history simulation.

Parallelization of an Additive Multigrid Solver

This article deals with the implementation and performance analysis of a parallel algebraic multigrid solver (pAMG) for a finite-volume, unstructured computational fluid dynamics (CFD) code. The parallelization of the solver is based on the domain decomposition approach using the single program, multiple data paradigm. The Message Passing Interface library (MPI) is used for communication of data. An ILU(0) iterative solver is used for smoothing the errors arising within each partition at the different grid levels, and a multi-level synchronization across the computational domain partitions is enforced in order to improve the performance of the parallelized multigrid solver. Two synchronization strategies are evaluated. In the first the synchronization is applied across the multigrid levels during the restriction step in addition to the base level, while in the second the synchronization is enforced during the restriction and prolongation steps. The effect of gathering the coefficients across partitions for the coarsest level is also investigated. Tests on grids up to 800,000 elements are conducted for a number of diffusion and advection problems on up to 20 processors. Results show that synchronization across partitions for multigrid levels plays an essential role in ensuring good scalability. Furthermore, for a large number of partitions, gathering coefficients across partitions is important to ensure a convergence history that is consistent with the sequential solver, thus yielding the same number of iterations for parallel and sequential runs, which is crucial for retaining high scalability. The shadow-to-core elements ratio is also shown to be a good indicator for scalability.

Effects of variable permeability on aqueous humor outflow

All previous efforts to quantify the aqueous humor (AH) outflow resistance in the juxtacanalicular tissue (JCT) of the human eye have attempted to treat the tissue as a homogeneous porous material. In this study, we treat the JCT as a heterogeneous tissue with variable permeability. The variable permeability of each sub-region of the JCT is estimated from Quick-Freeze/Deep-Etch images using Karmen–Kozeny theory. Finite element simulations of the fluid flow using Darcy’s law are computed at a micron resolution using a parallel algebraic multigrid (AMG) algorithm. Analyzing strictly the JCT, the overall resistance of this tissue is increased by 35–68% when assuming variable permeability instead of a homogenous tissue. Further, the resistance increase associated with the inclusion of the IW is 2–5 fold greater with a heterogeneous tissue compared to a homogeneous tissue. Because cellular and extracellular structures become denser proximal to the inner wall (IW) endothelium, resistance to AH outflow in this region of the JCT and funneling effects created by pores of the IW become amplified. Overall, the resistance of the JCT/IW domain is under-predicted when treating the JCT as a homogeneous tissue.

Relaxed RS0 or CLJP coarsening strategy for parallel AMG

Algebraic multigrid (AMG) algorithm is well known for it efficiencies in solving of larger scale sparse linear systems arising from the computationally challenging applications especially on unstructured or deformed structured grid. Though most of its components can be parallelized in a straightforward way, the classical coarsening process such as the Ruge–Stüben (RS) strategy is highly sequential and requires new parallel approaches. In recent years, many parallel coarsening strategies are presented towards running efficiently on hundreds or thousands of processors. This paper presents two new parallel coarsening strategies towards more efficiently distributing C-points for smaller operator complexity and more robust convergence of iterations. The main idea of these strategies is to smartly synchronize processors for the well-known RS0 or CLJP strategies during the process of coarsening. Qualitative analyses and numerical experiments show that our new strategies always perform better performance not only for faster convergence but also for smaller operator complexity if we compare them with the currently well-known parallel coarsening strategies such as RS3, Falgout or CLJP using hundreds of processors.

Coarse grid classification: a parallel coarsening scheme for algebraic multigrid methods

AbstractIn this paper, we present a new approach to the parallelization of algebraic multigrid (AMG), i.e. to the parallel coarse‐grid selection in AMG. Our approach involves (almost) no special treatment of processor subdomain boundaries and hence avoids a number of drawbacks of other AMG parallelization techniques. The key idea is to select an appropriate (local) coarse grid on each processor from a set of admissible grids such that the composed coarse grid forms a suitable coarse grid for the whole domain, i.e. there is no need for a special boundary treatment. To this end, we first construct multiple equivalent coarse grids on each processor subdomain. In a second step we then select exactly one grid per processor by a graph clustering technique. The results of our numerical experiments clearly indicate that this approach results in coarse grids of high quality which are very close to those obtained with sequential AMG. Furthermore, the operator and grid complexities of our parallel AMG are mostly smaller than those obtained by other parallel AMG methods, whereas the scale‐up behaviour of the proposed algorithm is similar to that of other parallel AMG techniques. However a significant improvement with respect to the speed‐up performance is achieved. Copyright © 2006 John Wiley & Sons, Ltd.

Parallel Algebraic Multigrids for Structural Mechanics

This paper presents the results of a comparison of three parallel algebraic multigrid (AMG) preconditioners for structural mechanics applications. In particular, we are interested in investigating both the scalability and robustness of the preconditioners. Numerical results are given for a range of structural mechanics problems with various degrees of difficulty, and they show that algebraic multigrid methods can be applied successfully even to difficult problems in structural mechanics.

Mechanical indentation as a tool for functional characterisation of gene therapy constructs

All previous efforts to quantify the aqueous humor (AH) outflow resistance in the juxtacanalicular tissue (JCT) of the human eye have attempted to treat the tissue as a homogeneous porous material. In this study, we treat the JCT as a heterogeneous tissue with variable permeability. The variable permeability of each sub-region of the JCT is estimated from Quick-Freeze/Deep-Etch images using Karmen–Kozeny theory. Finite element simulations of the fluid flow using Darcy’s law are computed at a micron resolution using a parallel algebraic multigrid (AMG) algorithm. Analyzing strictly the JCT, the overall resistance of this tissue is increased by 35–68% when assuming variable permeability instead of a homogenous tissue. Further, the resistance increase associated with the inclusion of the IW is 2–5 fold greater with a heterogeneous tissue compared to a homogeneous tissue. Because cellular and extracellular structures become denser proximal to the inner wall (IW) endothelium, resistance to AH outflow in this region of the JCT and funneling effects created by pores of the IW become amplified. Overall, the resistance of the JCT/IW domain is under-predicted when treating the JCT as a homogeneous tissue.

Reducing Complexity in Parallel Algebraic Multigrid Preconditioners

Algebraic multigrid (AMG) is a very efficient iterative solver and preconditioner for large unstructured sparse linear systems. Traditional coarsening schemes for AMG can, however, lead to computational complexity growth as problem size increases, resulting in increased memory use and execution time, and diminished scalability. Two new parallel AMG coarsening schemes are proposed that are based solely on enforcing a maximum independent set property, resulting in sparser coarse grids. The new coarsening techniques remedy memory and execution time complexity growth for various large three-dimensional (3D) problems. If used within AMG as a preconditioner for Krylov subspace methods, the resulting iterative methods tend to converge fast. This paper discusses complexity issues that can arise in AMG, describes the new coarsening schemes, and examines the performance of the new preconditioners for various large 3D problems.

A parallel algebraic multigrid solver for fast magnetic edge-element analyses

This paper presents a parallel algebraic multigrid (AMG) solver for linear systems of equations arising in magnetic finite edge-element analyses. To parallelize the smoothing process, which consumes most of the computational costs of the AMG algorithm, we apply multicolor (MC) ordering to the symmetric Gauss-Seidel (SGS) method. Advantages of MC ordering are: 1) that the number of processors employed does not affect the convergence of the approximate solution and 2) that only the information of the coefficient matrix is utilized to parallelize the smoother. The numerical results show that the developed solver achieves sufficient scalability in magnetic finite edge-element analyses.

A parallel algebraic multigrid solver for finite element method based source localization in the human brain

Time plays an important role in medical and neuropsychological diagnosis and research. In the field of Electro- and MagnetoEncephaloGraphy (EEG/MEG) source localization, a current distribution in the human brain is reconstructed noninvasively by means of measured fields outside the head. High resolution finite element modeling for the field computation leads to a sparse, large scale, linear equation system with many different right hand sides to be solved. The presented solution process is based on a parallel algebraic multigrid method. It is shown that very short computation times can be achieved through the combination of the multigrid technique and the parallelization on distributed memory computers. A solver time comparison to a classical parallel Jacobi preconditioned conjugate gradient method is given.