Restricted Additive Schwarz Research Articles

Accurate implementation of two-level asynchronous domain decomposition solvers

Recently, asynchronous coarse-grid correction has been achieved within additive Schwarz-type and primal Schur domain decomposition frameworks. Both additive and multiplicative coarse-grid corrections were discussed, however, the implemented asynchronous Schwarz-type solver with additive correction relies on the specific design of the restricted additive Schwarz (RAS) method, and also requires an overlap between the subsets of unknowns. In this paper, we first highlight a gap between the theoretical analysis from the literature and the associated RAS implementation. It turns out that communications delays would actually need to be bounded in order to fit the theory. This has to be stressed since, despite the asynchronous nature of the solver, the coarse-grid correction requires non-blocking global synchronization, which is subject to communications bottleneck. Second, we propose an implementation approach which applies to a wider class of additive Schwarz-type methods while still coping with the bounded delays requirement.

Accelerating multivariate functional approximation computation with domain decomposition techniques

Modeling large datasets through Multivariate Functional Approximations (MFA) provide an elegant way to handle many visualization and scientific analysis workflows. The process necessitates scalable data partitioning methods to compute MFA representations efficiently without compromising the accuracy or continuity of the reconstructed solution. We propose a domain-decomposed method for computing the MFA with B-spline bases, which reduces the total work per task and uses a restricted Additive Schwarz (RAS) method to converge the control point data degrees-of-freedom along subdomain boundaries. We provide an in-depth analysis of the parallel approach with domain decomposition solvers, aiming to minimize local subdomain error residuals and recover high-order continuity at subdomain interfaces with appropriate choices of knot overlaps. The communication cost, determined by the overlap regions in the RAS implementation, is optimized to recover the numerical error profile of the single subdomain case. Our proposed method stands in contrast to previous methods, which typically only recover either C0 or at best C1 continuity for arbitrary B-spline degree expansions, or those that require post-processing to blend discontinuities in the reconstructed data. We demonstrate the effectiveness of our approach using analytical and real-world datasets in 1D, 2D, and 3D through both strong and weak scaling studies. The performance results indicate that the overall cost of computing the approximation is directly proportional to the underlying nearest-neighbor communication implementation, and is only weakly dependent on the overlap region size that determines the size of the messages. This finding underscores the efficiency and scalability of our proposed method, making it a promising solution for handling large datasets in scientific workflows.

An Efficient and Effective Preconditioner for the Discontinuous Galerkin Domain Decomposition Method of Surface Integral Equation

This paper proposes a new preconditioning technique based on a restricted additive Schwarz (RAS) approach to improve the efficiency of the discontinuous Galerkin surface integral equation method. The RAS preconditioning is implemented efficiently using an oct-tree structure derived from the multilevel fast multipole algorithm (MLFMA) and is constructed using near-field matrices associated with boxes at the finest level. Compared to existing domain decomposition preconditioning methods based on subdomain block matrices, the proposed RAS preconditioning significantly reduces computation time and memory requirements, while providing scalable convergence for iterative solutions. Numerical experiments are presented to demonstrate the performance of the proposed cost-effective preconditioning technique.

Asynchronous multisplitting-based primal Schur method

This paper presents a new scalable asynchronous domain decomposition method which significantly outperforms the recent two-level asynchronous restricted additive Schwarz (RAS) method. The novel approach consists of considering the classical primal Schur complement framework and solving the interface problem using a multisplitting relaxation based on a weighted additive Schwarz preconditioner. The computational model does not require any significant implementation effort compared to an asynchronous RAS solver. Experiments were conducted on a linear elasticity test case, and the results clearly show the practical superiority of the new asynchronous method, which is even competitive with its conjugate gradient counterpart.

Multilevel field-split preconditioners with domain decomposition for steady and unsteady flow problems

With the incremental applications of Newton–Krylov methods for solving large sparse nonlinear systems of equations, the design of robust and scalable linear preconditioners plays an essential role for the whole solver. In this paper, we investigate the family of field-split (FS) preconditioners with different combinations of physics-based and domain decomposition methods, applied to the two typical fluid problems, i.e., the unsteady flow through fractured porous media and the steady buoyancy driven flow. In the implementation, several new versions of FS preconditioners are considered under the framework of the domain decomposition technique: additive FS, multiplicative FS, Schur-complement FS, and the constrained pressure residual (CPR) method, where the inverse of corresponding matrices is approximated by using the restricted additive Schwarz (RAS) algorithm. Rigorous eigenvalue analysis for various FS preconditioners is also provided for facilitating the design of algorithms. In particular, our approach further enhances the numerical performance by presenting a family of multilevel field-split methods for efficiently preconditioning. Numerical experiments are presented to demonstrate the robustness and parallel scalability of the proposed preconditioning strategies for both standard benchmarks as well as realistic flow problems on a supercomputer.

Linear and nonlinear substructured Restricted Additive Schwarz iterations and preconditioning

Iterative substructuring Domain Decomposition (DD) methods have been extensively studied, and they are usually associated with nonoverlapping decompositions. It is less known that classical overlapping DD methods can also be formulated in substructured form, i.e., as iterative methods acting on variables defined exclusively on the interfaces of the overlapping domain decomposition. We call such formulations substructured domain decomposition methods. We introduce here a substructured version of Restricted Additive Schwarz (RAS) which we call SRAS. We show that RAS and SRAS are equivalent when used as iterative solvers, as they produce the same iterates, while they are substantially different when used as preconditioners for GMRES. We link the volume and substructured Krylov spaces and show that the iterates are different by deriving the least squares problems solved at each GMRES iteration. When used as iterative solvers, SRAS presents computational advantages over RAS, as it avoids computations with matrices and vectors at the volume level. When used as preconditioners, SRAS has the further advantage of allowing GMRES to store smaller vectors and perform orthogonalization in a lower dimensional space. We then consider nonlinear problems, and we introduce SRASPEN (Substructured Restricted Additive Schwarz Preconditioned Exact Newton), where SRAS is used as a preconditioner for Newton’s method. In contrast to the linear case, we prove that Newton’s method applied to the preconditioned volume and substructured formulation produces the same iterates in the nonlinear case. Next, we introduce two-level versions of nonlinear SRAS and SRASPEN. Finally, we validate our theoretical results with numerical experiments.

Parallel multilevel restricted Schwarz preconditioners for implicit simulation of subsurface flows with Peng-Robinson equation of state

Parallel algorithms and simulators with good scalabilities are particularly important for large-scale reservoir simulations on modern supercomputers with a large number of processors. In this paper, we introduce and study a family of highly scalable multilevel restricted additive Schwarz (RAS) methods for the fully implicit solution of subsurface flows with Peng-Robinson equation of state in two and three dimensions. With the use of a second-order fully implicit scheme, the proposed simulator is unconditionally stable with the relaxation of the time step size by the stability condition. The investigation then focuses on the development of several types of multilevel overlapping additive Schwarz methods for the preconditioning of the resultant linear system arising from the inexact Newton iteration, and some fast solver technologies are presented for the assurance of the multilevel approach efficiency and scalability. We numerically show that the proposed fully implicit framework is highly efficient for solving both standard benchmarks as well as realistic problems with several hundreds of millions of unknowns and scalable to 8192 processors on the Tianhe-2 supercomputer.

Parallel finite-element codes for the simulation of two-dimensional and three-dimensional solid–liquid phase-change systems with natural convection

We present and distribute a FreeFem++ Toolbox for the parallel computing of two- or three-dimensional liquid–solid phase-change systems involving natural convection. FreeFem++ (www.freefem.org) is a free finite-element software available for all existing operating systems. We use the recent library ffddm that makes available in FreeFem++ state-of-the-art scalable Schwarz domain decomposition methods (DDM). The single domain approach used in our previous contribution (Rakotondrandisa et al., 2020) is adapted for the use of the DDM method. As a result, the computational time is considerably reduced for 2D configurations and furthermore 3D problems become affordable. The numerical method is based on an enthalpy-porosity model. The same set of equations is solved in both liquid and solid phases: the incompressible Navier–Stokes equations with Boussinesq approximation for thermal effects. A Carman–Kozeny-type penalty term is added to the momentum equations to bring progressively the velocity to zero into the solid. Model equations are discretized using Galerkin triangular or tetrahedral finite elements. The coupled system of equations is integrated in time using a second-order Gear implicit scheme. The resulting discrete equations are solved using a Newton algorithm. The DDM approach is based on an overlapping Schwarz method. The mesh is first split in subdomains using Scotch or Metis libraries. The final linear system is then solved in parallel using a GMRES Krylov method, with a Restricted Additive Schwarz (RAS) preconditioner. The mesh is adapted during the computation using metrics control. The 3D-mesh adaptivity uses the mmg (www.mmgtools.org) open source library. Parallel 2D and 3D computations of benchmark cases of increasing difficulty are presented: natural convection of air, natural convection of water, melting or solidification of a phase-change material, and, finally, a water freezing case. For each case, careful validations are provided and the performance of the code is assessed. The robustness of the Toolbox in 3D is also demonstrated by adapting the number of processors to the number of tetrahedra, which can considerably vary after the mesh adaptation. Program summaryProgram Title: PCM_Toolbox_DDM_2D and PCM_Toolbox_DDM_3DCPC Library link to program files:http://dx.doi.org/10.17632/dk49rfrz9y.1Licensing provisions: Apache License, 2.0Programming language: FreeFem++(www.freefem.org), mmg (www.mmgtools.org)Nature of problem: The software is scoped to parallel computations of 2D or 3D configurations of liquid–solid phase-change problems with convection in the liquid phase. Natural convection, melting and solidification processes are illustrated in the paper. The software can be easily modified to take into account different related physical models.Solution method: We use a single domain approach, solving the incompressible Navier–Stokes equations with Boussinesq approximation in both liquid and solid phases. A Carman–Kozeny-type penalty term is added to the momentum equations to bring the velocity to zero into the solid phase. An enthalpy model is used in the energy equation to take into account the phase change. Discontinuous variables (latent heat, material properties) are regularized through an intermediate (mushy) region. Space discretization is based on Galerkin triangular/tetrahedral finite elements. A second order Gear implicit scheme is used for the time integration of the coupled system of equations. The resulting discrete equations are solved using a Newton algorithm. Piecewise quadratic (P2) finite-elements are used for the velocity and piecewise linear (P1) for the pressure. For the temperature both P2 or P1 discretization are possible. The mesh is first split in subdomains using Scotch or Metis libraries, which are interfaced with FreeFem++. Then, a Schwarz domain decomposition method is used through the FreeFem++library ffddm. The final linear system is solved in parallel using a GMRES Krylov method, with a Restricted Additive Schwarz (RAS) preconditioner. Mesh adaptivity using metrics control makes possible the optimization of the distribution of mesh elements. For 3D case, the mmg open source library is used to adapt the mesh.

Schwarz Solvers and Preconditioners for the Closest Point Method

The discretization of surface intrinsic elliptic partial differential equations (PDEs) poses interesting challenges not seen in flat spaces. The discretization of these PDEs typically proceeds by either parametrizing the surface, triangulating the surface, or embedding the surface in a higher dimensional flat space. The closest point method (CPM) is an embedding method that represents surfaces using a function that maps points in the embedding space to their closest points on the surface. In the CPM, this mapping also serves as an extension operator that brings surface intrinsic data onto the embedding space, allowing PDEs to be numerically approximated by standard methods in a narrow tubular neighborhood of the surface. We focus on numerically approximating the positive Helmholtz equation, $\left(c-\Delta_{\mathcal{S}}\right)u=f,~c\in\mathbb{R}^+$, by the CPM paired with finite differences. This yields a large, sparse, and nonsymmetric system to be solved. Herein, we develop restricted additive Schwarz (RAS) and optimized restricted additive Schwarz (ORAS) solvers and preconditioners for this discrete system. In particular, we develop a general strategy for computing overlapping partitions of the computational domain and defining the corresponding Dirichlet and Robin transmission conditions. We demonstrate that the convergence of the ORAS solvers and preconditioners can be improved by using a modified transmission condition where more than two overlapping subdomains meet. Numerical experiments are provided for a variety of analytical and triangulated surfaces. We find that ORAS solvers and preconditioners outperform their RAS counterparts and that, as expected using domain decomposition (DD) as a preconditioner rather than as a solver gives faster convergence. The methods exhibit good parallel scalability over the range of process counts tested.

New approach for accelerating the nonlinear Schwarz iterations

The vector Epsilon algorithm is an effective extrapolation method used for accelerating the convergence of vector sequences. In this paper, this method is used to accelerate the convergence of Schwarz iterative methods for stationary linear and nonlinear partial differential equations (PDEs). The vector Epsilon algorithm is applied to the vector sequences produced by additive Schwarz (AS) or restricted additive Schwarz (RAS) methods after discretization. Some convergence analysis is presented, and several test-cases of analytical problems are performed in order to illustrate the interest of such algorithm. The obtained results show that the proposed algorithm yields much faster convergence than the classical Schwarz iterations.

Parallel domain decomposition methods for a quantum-corrected drift–diffusion model for MOSFET devices

In this paper, we describe parallel domain decomposition methods based on the restricted additive Schwarz (RAS) method for a quantum-corrected drift–diffusion (QCDD) model for MOSFET devices. We have developed hybrid Message Passing Interface (MPI)/OpenMP parallelization algorithms of the QCDD system. For internode parallelization, two extensions of the RAS method are newly developed for the QCDD model. For intranode parallelization, we combine the conjugate gradient (CG) and BiCGSTAB procedures with a splitting-up operator method to realize parallelization of the incomplete factorization. The parallel numerical results for a three-dimensional Si bulk n-MOSFET on a multi-core NEC SX-ACE parallel computer are demonstrated. The intranode parallel numerical results are further evaluated on a many-core Cray XC40 parallel computer.

The convergence theory for the restricted version of the overlapping Schur complement preconditioner

The restricted version of the overlapping Schur complement (SchurRAS) preconditioner was introduced by Li and Saad (2006) for the solution of linear system Ax=b, and numerical results have shown that the SchurRAS method outperforms the restricted additive Schwarz (RAS) method both in terms of iteration count and CPU time. In this paper, based on meticulous derivation, we give an algebraic representation of the SchurRAS preconditioner, and prove that the SchurRAS method is convergent under the condition that A is an M-matrix and it converges faster than the RAS method.

Efficient CPR-type preconditioner and its adaptive strategies for large-scale parallel reservoir simulations

With the rapid development of high performance parallel computers and the increasing demands of large-scale reservoir simulation, the development of effective and fast solvers and preconditioners for parallel reservoir simulation has become increasingly important. The incomplete LU factorization (ILU) preconditioners are the most commonly used preconditioners in reservoir simulations due to their low computational cost. They have been implemented in commercial simulators as the default preconditioners. However, for reservoirs with highly heterogeneous permeability, the CPR (constrained pressure residual)-type preconditioners are more effective than the ILU preconditioners, especially when parallel computing is employed. The CPR-type preconditioners consist of multi-stages of preconditioning processes, which lead to huge computational costs. Employing the algebraic multigrid (AMG) method to solve a pressure system and the ILU method to solve the whole system has two indispensable stages. For parallel computation, due to the serial nature of the ILU method, the ILU method is usually combined with the restricted additive Schwarz (RAS) method and used as a subdomain solver. Different settings of the AMG and RAS methods, such as a grid coarsening strategy, interpolation and smoothing in the AMG method, and a subdomain solver and an overlap level in the RAS method, as well as different CPR-type preconditioners, yield significantly different performance. This paper first gives an overview of the CPR-type preconditioners and the AMG and RAS methods. Then a detailed comparison between different CPR-type preconditioners with different settings of the AMG and RAS methods is presented. Based on the comparison, the efficient settings of the AMG and RAS methods and the most efficient CPR-type preconditioner are given. Furthermore, an adaptive preconditioning strategy is designed to save the computational time. The adaptive preconditioning strategy introduces an indicator to measure the difficulty level of solving a linear system. According to difficulties, a RAS–ILU preconditioner or a CPR-type preconditioner is automatically and dynamically selected to achieve both efficiency and effectiveness.

Parallel ILU preconditioners in GPU computation

Accelerating large-scale linear solvers is always crucial for scientific research and industrial applications. In this regard, preconditioners play a key role in improving the performance of iterative linear solvers. This paper presents a summary and review of our work about the development of parallel ILU preconditioners on GPUs. The mechanisms of ILU(0), ILU(k), ILUT, enhanced ILUT, and block-wise ILU(k) are reviewed and analyzed, which give a clear guidance in the development of iterative linear solvers. ILU(0) is the most commonly used preconditioner, and the nonzero pattern of its matrix is exactly the same as the original matrix to be solved. ILU(k) uses k levels to control the pattern of its preconditioner matrix. ILUT selects entries for its preconditioner matrix by setting thresholds without considering its original matrix pattern. In addition to point-wise ILU preconditioners, a block-wise ILU(k) preconditioner is designed delicately in support of block-wise matrices. In implementation, the RAS (Restricted Additive Schwarz) method is adopted to optimize the parallel structure of a preconditioner matrix. Coupling with the configuration parameters of ILU preconditioners, a complex situation appears in the parallel solution process, so decoupled algorithms are adopted. These algorithms are implemented and tested on NVIDIA GPUs. The experiment results show that a single-GPU implementation can speed up an ILU preconditioner by a factor of 10, compared to traditional CPU implementation. The results also show that the ILU(0) has better speedup than ILU(k) but slower convergence than ILU(k). Level k of ILU(k) and threshold (p, t) of ILUT are effective adjustment factors for controlling the equilibrium point between acceleration and convergence for ILU(k) and ILUT, respectively. All these ILU preconditioners are characterized and compared in this work, which shows a clear picture and numerical insights for practitioners in the ILU family.

Aitken's acceleration of the Restricted Additive Schwarz preconditioning using coarse approximations on the interface

An enhancement of the restricted Additive Schwarz (RAS) preconditioning, based on the Aitken's acceleration of the convergence of the Schwarz method, is proposed. Its numerical performance is compared with the RAS preconditioning on the two dimensional Helmholtz problem.

Parallel Multilevel Restricted Schwarz Preconditioners with Pollution Removing for PDE‐Constrained Optimization

We develop a class of V‐cycle‐type multilevel restricted additive Schwarz (RAS) methods and study the numerical and parallel performance of the new fully coupled methods for solving large sparse Jacobian systems arising from the discretization of some optimization problems constrained by nonlinear partial differential equations. Straightforward extensions of the one‐level RAS to multilevel do not work due to the pollution effects of the coarse interpolation. We then introduce, in this paper, a pollution removing coarse‐to‐fine interpolation scheme for one of the components of the multicomponent linear system and show numerically that the combination of the new interpolation scheme with the RAS smoothed multigrid method provides an effective family of techniques for solving rather difficult PDE‐constrained optimization problems. Numerical examples involving the boundary control of incompressible Navier–Stokes flows are presented in detail.

SchurRAS: A Restricted Version of the Overlapping Schur Complement Preconditioner

This paper presents a preconditioner based on solving approximate Schur complement systems with the overlapping restricted additive Schwarz (RAS) methods. The construction of the preconditioner, called SchurRAS, is as simple as in the standard RAS. The communication requirements for each application of the preconditioning operation are also similar to those of the standard RAS approach. In the particular case when the degree of overlap is one, then SchurRAS and RAS involve exactly the same communication volume per step. In addition, SchurRAS has the same degree of parallelism as RAS. In some numerical experiments with a model problem, the convergence rate of the method was found to be similar to that of the multiplicative Schwarz (MS) method. The Schur-based RAS usually outperforms the standard RAS both in terms of iteration count and CPU time. For a few two-dimensional scaled problems, SchurRAS was about twice as fast as the stardard RAS on 64 processors.

Restricted Additive Schwarz Preconditioners with Harmonic Overlap for Symmetric Positive Definite Linear Systems

A restricted additive Schwarz (RAS) preconditioning technique was introduced recently for solving general nonsymmetric sparse linear systems. In this paper, we provide one-level and two-level extensions of RAS for symmetric positive definite problems using the so-called harmonic overlaps (RASHO). Both RAS and RASHO outperform their counterparts of the classical additive Schwarz variants (AS). The design of RASHO is based on a much deeper understanding of the behavior of Schwarz-type methods in overlapping subregions and in the construction of the overlap. In RASHO, the overlap is obtained by extending the nonoverlapping subdomains only in the directions that do not cut the boundaries of other subdomains, and all functions are made harmonic in the overlapping regions. As a result, the subdomain problems in RASHO are smaller than those of AS, and the communication cost is also smaller when implemented on distributed memory computers, since the right-hand sides of discrete harmonic systems are always zero and therefore do not need to be communicated. We also show numerically that RASHO-preconditioned CG takes fewer iterations than the corresponding AS-preconditioned CG. A nearly optimal theory is included for the convergence of RASHO-preconditioned CG for solving elliptic problems discretized with a finite element method.

Why Restricted Additive Schwarz Converges Faster than Additive Schwarz

Recently a variant of the additive Schwarz (AS) preconditioner, the restricted additive Schwarz (RAS) preconditioner has been introduced, and numerical experiments showed that RAS converges faster and requires less communication than AS. We show in this paper how RAS, which is defined at the matrix level, can be interpreted as an iteration at the continuous level of the underlying problem. This interpretation reveals why RAS converges faster than classical AS.

Convergence Theory of Restricted Multiplicative Schwarz Methods

Convergence results for the restricted multiplicative Schwarz (RMS) method, the multiplicative version of the restricted additive Schwarz (RAS) method for the solution of linear systems of the form Ax = b, are provided. An algebraic approach is used to prove convergence results for nonsymmetric M-matrices. Several comparison theorems are also established. These theorems compare the asymptotic rate of convergence with respect to the amount of overlap, the exactness of the subdomain solver, and the number of domains. Moreover, comparison theorems are given between the RMS and RAS methods as well as between the RMS and the classical multiplicative Schwarz method.