Dynamic Domain Decomposition Research Articles

D 3 PBO: D ynamic D omain D ecomposition-based P arallel B ayesian O ptimization for Large-scale Analog Circuit Sizing

Bayesian optimization (BO) is an efficient global optimization method for expensive black-box functions, but the expansion for high-dimensional problems and large sample budgets still remains a severe challenge. In order to extend BO for large-scale analog circuit synthesis, a novel computationally efficient parallel BO method, D 3 PBO, is proposed for high-dimensional problems in this work. We introduce the dynamic domain decomposition method based on maximum variance between clusters. The search space is decomposed into subdomains progressively to limit the maximal number of observations in each domain. The promising domain is explored by multi-trust region-based batch BO with the local Gaussian process (GP) model. As the domain decomposition progresses, the basin-shaped domain is identified using a GP-assisted quadratic regression method and exploited by the local search method BOBYQA to achieve a faster convergence rate. The time complexity of D 3 PBO is constant for each iteration. Experiments demonstrate that D 3 PBO obtains better results with significantly less runtime consumption compared to state-of-the-art methods. For the circuit optimization experiments, D 3 PBO achieves up to 10× runtime speedup compared to TuRBO with better solutions.

GPU-based transient analysis of modern grids deploying a hybrid DDM algorithm

Solving a significant number of differential-algebraic equations is essential for conducting transient stability simulations on a complex and interconnected electrical power system. Because of the increasing size and intricacy of modern grids, traditional sequential simulation methods are becoming more computationally challenging and time-consuming to use for dynamic simulations. This research aims to examine parallelism techniques from an innovative algorithm viewpoint to the hardware level by utilizing multiple graphics processing units as the main computational tool designed to simulate the dynamic behaviour of power systems on a large scale. Our approach features a novel hybrid algorithm to improve the dynamic simulations of grids with significant renewable energy integration. The algorithm employs a Schwarz-based approach in its initial decomposition phase to isolate synchronous machines and renewable sources from the grid. This separation enhances the algorithm's ability to manage these components effectively. Subsequently, each subsystem undergoes further division into subdomains in the second stage, with separate computations carried out using the Schur-complement technique. The simulations were conducted on various test systems, with a maximum of 25,000 buses, 8,000 synchronous generators, and 256 PV farms all modelled in detail. We introduce a GPU-oriented preprocessing and vectorization parallelization method for the implementation of our two-stage algorithm. This approach provides the flexibility to tailor parallelization techniques, including nested for loops and kernel pragmas, to craft a hardware solution that enhances the performance of our dynamic domain decomposition algorithm. Simulation outcomes have verified that this approach can yield a notable 7.8x acceleration in speed when executed on an NVIDIA GeForce RTX 2070 SUPER GPU.

The muphyII code: Multiphysics plasma simulation on large HPC systems

Collisionless astrophysical and space plasmas cover regions that typically display a separation of scales that exceeds any code's capabilities. To help address this problem, the muphyII code utilizes a hierarchy of models with different inherent scales, unified in an adaptive framework that allows stand-alone use of models as well as a model-based dynamic and adaptive domain decomposition. This requires ensuring excellent conservation properties, careful treatment of inner-domain model boundaries for interface coupling, and robust time-stepping algorithms, especially with the use of electron subcycling. This multi-physics approach is implemented in the muphyII code, tested on different scenarios of space plasma magnetic reconnection, and evaluated against space probe data and higher-fidelity simulation results from literature. Adaptive model refinement is highlighted in particular, and a hybrid model with kinetic ions, pressure-tensor fluid electrons, and Maxwell fields is appraised.

A coupled domain–boundary type meshless method for phase-field modelling of dendritic solidification with the fluid flow

PurposeThis study aims to simulate the dendritic growth in Stokes flow by iteratively coupling a domain and boundary type meshless method.Design/methodology/approachA preconditioned phase-field model for dendritic solidification of a pure supercooled melt is solved by the strong-form space-time adaptive approach based on dynamic quadtree domain decomposition. The domain-type space discretisation relies on monomial augmented polyharmonic splines interpolation. The forward Euler scheme is used for time evolution. The boundary-type meshless method solves the Stokes flow around the dendrite based on the collocation of the moving and fixed flow boundaries with the regularised Stokes flow fundamental solution. Both approaches are iteratively coupled at the moving solid–liquid interface. The solution procedure ensures computationally efficient and accurate calculations. The novel approach is numerically implemented for a 2D case.FindingsThe solution procedure reflects the advantages of both meshless methods. Domain one is not sensitive to the dendrite orientation and boundary one reduces the dimensionality of the flow field solution. The procedure results agree well with the reference results obtained by the classical numerical methods. Directions for selecting the appropriate free parameters which yield the highest accuracy and computational efficiency are presented.Originality/valueA combination of boundary- and domain-type meshless methods is used to simulate dendritic solidification with the influence of fluid flow efficiently.

Dynamic domain decomposition method based on weighted Voronoi diagrams

Dynamic load balancing during the high performance computing is an important part of parallel workflow. An efficient way to achieve the desired distribution of load during computation is to use the Voronoi diagram (VD) for the domain decomposition. The advantage of this approach is that it can be used in conjunction with various numerical methods and balancing strategies. Domain decomposition based on Voronoi diagrams can be used together with Eulerian meshes, Lagrangian meshes, and meshfree methods, such as molecular dynamic or smoothed-particle hydrodynamics (SPH). However, for the basic VD decomposition it may be difficult to obtain load balance when the adaptive mesh refinement (AMR) is used: because of the high heterogeneity of the load on the adapted mesh, the balancing algorithms may be unstable or have poor convergence.In the present study we propose an improvement of the ordinary VD domain decomposition method with using the weighted Voronoi diagrams. We demonstrate the possibilities of such domain decomposition to achieve a better balanced configuration on two- and three-dimensional problems using mesh configurations typical for computational fluid dynamics.

Acceleration of RBF-FD meshless phase-field modelling of dendritic solidification by space-time adaptive approach

• A novel meshless approach for phase-field modelling of dendritic growth. • Reduction of discretisation-induced anisotropy by the scattered nodes. • Increase in computational efficiency by a space-time adaptive approach. A novel adaptive numerical approach is developed for an accurate and computationally efficient phase-field modelling of dendritic solidification. The adaptivity is based on the dynamic quadtree domain decomposition. A quadtree decomposes the computational domain into rectangular sub-domains of different sizes. Each sub-domain is extended to ensure overlap communication between neighbouring sub-domains. In each sub-domain, uniform distribution of computational nodes is generated. The product between the node density and the sub-domain area is fixed to ensure the h-adaptivity. The adaptive approach provides the highest density of computational nodes at the solid-liquid interface and the lowest density in the bulk of the phases. The meshless radial basis function generated finite difference (RBF-FD) method is applied for the spatial discretisation of the partial differential equations which arise from the phase-field model. The RBF-FD method is especially appealing since it allows straightforward spatial discretisation of partial differential equations on scattered node distributions. The use of scattered node distribution reduces the discretisation-induced anisotropy in the phase-field modelling of dendritic growth. The forward Euler scheme is used for temporal discretisation. The adaptive time-stepping is employed to speed up the calculations further. The performance of the novel numerical approach is tested for dendritic solidification of supercooled pure melts and supersaturated dilute binary alloys at arbitrary preferential growth directions. The impact of the numerical parameters on the accuracy and computational efficiency is thoroughly analysed. It is shown that the RBF-FD method, defined on scattered node distribution, together with the space-time adaptive approach, represents an accurate and efficient technique for solving the phase-field models of dendritic solidification.

Parallel framework for dynamic domain decomposition of data assimilation problems: a case study on Kalman Filter algorithm

We focus on Partial Differential Equation (PDE) based Data Assimilatio problems (DA) solved by means of variational approaches and Kalman filter algorithm. Recently, we presented a Domain Decomposition framework (we call it DD-DA, for short) performing a decomposition of the whole physical domain along space and time directions, and joining the idea of Schwarz' methods and parallel in time approaches. For effective parallelization of DD-DA algorithms, the computational load assigned to subdomains must be equally distributed. Usually computational cost is proportional to the amount of data entities assigned to partitions. Good quality partitioning also requires the volume of communication during calculation to be kept at its minimum. In order to deal with DD-DA problems where the observations are nonuniformly distributed and general sparse, in the present work we employ a parallel load balancing algorithm based on adaptive and dynamic defining of boundaries of DD -- which is aimed to balance workload according to data location. We call it DyDD. As the numerical model underlying DA problems arising from the so-called discretize-then-optimize approach is the constrained least square model (CLS), we will use CLS as a reference state estimation problem and we validate DyDD on different scenarios.

Parallel Dynamic Domain Decomposition in Space - Time for Data Assimilation problems

In the present work we employ a load balancing scheme involving an adaptive and dynamic workload redistribution both along Space and Time directions for solving Data Assimilation problems where the observations are non-uniformly distributed, general sparse and its distribution changes during the time. We will consider the Constrained Least Square model (CLS) as prototype of Data Assimilation problems and we will validate the proposed approach on different configurations. Validation is performed using Parallel Computing Toolbox of MATLABR2013a on high performance hybrid computing architecture.

A dynamic coarsening approach to immiscible multiphase flows in heterogeneous porous media

We have developed a sequential approach to accelerate immiscible multiphase flow modelling in heterogeneous porous media using discontinuous Galerkin methods and dynamic mesh coarsening. This approach involves dynamic domain decomposition and different solution strategies in the different flow regions, using a criterion that can be fastly evaluated. We use high-resolution grids and low order methods in the multiphase flow regions near the saturation discontinuity and a discontinuous Galerkin method along with low-resolution grids in single-phase flow regions of the domain. We present a fast technique to estimate the position of the saturation front and identify the flow zones that need high-resolution gridding. We demonstrate the accuracy of our approach through test cases from the second SPE10 model by comparing our results with fine-scale simulations.

Implementation of the moving particle semi-implicit method for free-surface flows on GPU clusters

The moving particle semi-implicit (MPS) method performs well in simulating incompressible flows with free surfaces. Despite its applicability, the MPS method suffers from the fundamental instability problem and high computational cost in its practical application. Substantial research has been conducted on improving the stability and accuracy of the MPS method. Moreover, graphics processing units (GPUs), which are multi-processors that execute many three-dimensional geometric processes at high speed, provide unprecedented capability for scientific computations. However, the usage of a single GPU card is not sufficient for engineering applications that require several million particles that predict the desired physical processes, because the available memory space is insufficient. In this work, the dynamic stability (DS) algorithm and particle shifting (PS) algorithm have been used to overcome the instability and inaccuracies caused by tensile instability and non-uniform particle distribution, respectively. Based on the stable MPS method, a GPU-based MPS code that uses the compute unified device architecture (CUDA) language has been developed. An efficient neighborhood particle search is performed using an indirect method, and the matrix for the pressure Poisson equation (PPE) is assembled in parallel. Based on the single-GPU version, a multi-GPU MPS code has been developed. The approach uses a non-geometric dynamic domain decomposition method that provides homogeneous load balancing whereby different portions (subdomains) of the physical system under study are assigned to different GPUs. Communication between devices is achieved with the use of a message passing interface (MPI). Based on the neighborhood particle search, the techniques for building and updating the “halo” are described in detail. The speed-up of the single-GPU version is analyzed for different numbers of particles, and the scalability of the multi-GPU version is analyzed for different numbers of particles and different numbers of GPUs. Last, an application with more than 107 particles is presented to show the capability of the code in handling large-scale simulations.

Parallel adaptive mesh refinement method based on bubble-type local mesh generation

An efficient parallel adaptive mesh refinement method based on bubble-type local mesh generation (BLMG), employing ParMETIS-based dynamic domain decomposition method, is developed. The BLMG method is applied to generate the mesh with high quality, and the local mesh for each processor can be generated simultaneously without communication. The node-based distributed mesh structure is designed to reduce the communication amount spent in mesh generation and finite element calculation. To gain the load balance, a new load balancing algorithm for the new mesh structure is devised to make sure the whole algorithm is efficient. Several numerical examples are carried out to verify the high efficiency of the algorithm.

Auto-balancing algorithm for parallel SPH simulation of materials in extremes

We developed a highly efficient SPH (Smoothed Particle Hydrodynamics) code using a parallel algorithm based on the dynamic Voronoi domain decomposition of simulated samples. The used self-adaptive algorithm aimed to minimize processor load imbalance is suitable for any particle-based method with a short-range interaction. The strong scalability test performed for a sample in rest demonstrates almost linear speedup up to 1024 cores. Test of sample on extreme conditions leading to material flow with highly inhomogeneous distribution of particles demonstrates up to 3 times speedup for the presented code over a static rectangular decomposition of simulated domain.

Game Theory Based Evolutionary Algorithms: A Review with Nash Applications in Structural Engineering Optimization Problems

A general review of game-theory based evolutionary algorithms (EAs) is presented in this study. Nash equilibrium, Stackelberg game and Pareto optimality are considered, as game-theoretical basis of the evolutionary algorithm design, and also, as problems solved by evolutionary computation. Applications of game-theory based EAs in computational engineering are listed, with special emphasis in structural optimization and, particularly, in skeletal structures. Additionally, a set of three problems are solved: reconstruction inverse problem, fully stressed design problem and minimum constrained weight, for discrete sizing of frame skeletal structures. We compare panmictic EAs, Nash EAs using 4 different static domain decompositions, including also a new dynamic domain decomposition. Two frame structural test cases of 55 member size and 105 member size are evaluated with the linear stiffness matrix method. Numerical experiments show the efficiency of the Nash EAs approach, confirmed with statistical significance analysis of results, and enhanced with the dynamic domain decomposition.

A Multi-GPU Fast Iterative Method for Eikonal Equations Using on-the-fly Adaptive Domain Decomposition

The recent research trend of Eikonal solver focuses on employing state-of-the-art parallel computing technology, such as GPUs. Even though there exists previous work on GPU-based parallel Eikonal solvers, only little research literature exists on the multi-GPU Eikonal solver due to its complication in data and work management. In this paper, we propose a novel on-the-fly, adaptive domain decomposition method for efficient implementation of the Block-based Fast Iterative Method on a multi-GPU system. The proposed method is based on dynamic domain decomposition so that the region to be processed by each GPU is determined on-the-fly when the solver is running. In addition, we propose an efficient domain assignment algorithm that minimizes communication overhead while maximizing load balancing between GPUs. The proposed method scales well, up to 6.17× for eight GPUs, and can handle large computing problems that do not fit to limited GPU memory. We assess the parallel efficiency and runtime performance of the proposed method on various distance computation examples using up to eight GPUs.

A dynamic domain decomposition for the eikonal-diffusion equation

We propose a parallel algorithm for the numerical solution of the eikonal-diffusion equation, by means of a dynamic domain decomposition technique.The new method is an extension of the patchy domain decomposition method presented in [5] for first order Hamilton-Jacobi-Bellman equations.Using the connection with stochastic optimal control theory, the semi-Lagrangian scheme underlying the original method is modified in order to deal with(possibly degenerate) diffusion. We show that under suitable relations between the discretization parameters and the diffusion coefficient, the parallelcomputation on the proposed dynamic decomposition can be faster than that on a static decomposition.Some numerical tests in dimension two are presented, in order to show the features of the proposed method.

A domain decomposition strategy for hybrid parallelization of moving particle semi-implicit (MPS) method for computer cluster

The MPS is a particle method developed to simulate incompressible flows with free surfaces that has several applications in nonlinear hydrodynamics. Much effort has been done to handle the large amount of particles required for simulating practical problems with desired refinement. However, the efficient use of the currently available computational resources, such as computer cluster, remains as a challenge. The present paper proposes a new strategy to parallelize the MPS method for fully distributed computing in cluster, which enables to simulate models with hundreds of millions of particles and keeps the required runtime within reasonable limits, as shown by the analysis of scalability and performance. The proposed strategy uses a non-geometric dynamic domain decomposition method that provides homogeneous load balancing and for very large models the scalability is supra-linear. Also, the domain decomposition (DD) is carried out only in the initial setup. As a result, the DD method is based on renumbering of particles using an original fully distributed sorting algorithm. Moreover, unlike the usual strategies, none of the processors require access to global data of the particles on any time step. Therefore, the limit for the maximum size of the model depends more on the total memory of the allocated nodes than the quantity of the local memory of each node. Thus, by extending the application of MPS method to very large models, this study contributes to consolidating the method as a practical tool to investigate complex engineering problems.

The comparison of two domain repartitioning methods used for parallel discrete element computations of the hopper discharge

The paper presents an application of two domain repartitioning methods to solving hopper discharge problem simulated by the discrete element method. Quantitative comparison of parallel speed-up obtained by using the multilevel k-way graph partitioning method and the recursive coordinate bisection method is presented. The detailed investigation of load balance, interprocessor communication and repartitioning is performed. Speed-up of the parallel computations based on the dynamic domain decomposition is investigated by a series of benchmark tests simulating the granular visco-elastic frictional media in hoppers containing 0.3×106 and 5.1×106 spherical particles. A soft-particle approach is adopted, when the simulation is performed by small time increments and the contact forces between the particles are calculated using the contact law. The parallel efficiency of 0.87 was achieved on 2048 cores, modelling the hopper filled with 5.1×106 particles.

Space decomposition based parallelization solutions for the combined finite–discrete element method in 2D

The combined finite–discrete element method (FDEM) belongs to a family of methods of computational mechanics of discontinua. The method is suitable for problems of discontinua, where particles are deformable and can fracture or fragment. The applications of FDEM have spread over a number of disciplines including rock mechanics, where problems like mining, mineral processing or rock blasting can be solved by employing FDEM. In this work, a novel approach for the parallelization of two-dimensional (2D) FDEM aiming at clusters and desktop computers is developed. Dynamic domain decomposition based parallelization solvers covering all aspects of FDEM have been developed. These have been implemented into the open source Y2D software package and have been tested on a PC cluster. The overall performance and scalability of the parallel code have been studied using numerical examples. The results obtained confirm the suitability of the parallel implementation for solving large scale problems.

Implementation and Evaluation of an AMR Framework for FDM Applications

In order to e xecute various finite-difference method applications on large-scale parallel computers with a reasonable cost of computer resources, a framework using an adaptive mesh refinement (AMR) technique has been developed. AMR can realize high -resolution simulations while saving computer resources by generating and removing hierarchica l grids dynamically. In the AMR frame work, a dynamic domain decomposition (DDD) technique, as a dynamic load balancing method, is also implemented to correct the computational load imbalance between each process associated with parallelization. By performing a 3D AM R test simulation, it is confirmed that dynamic load balancing can be achieved and execution time can be reduced by introducing the DDD technique.

Development of Parallelized AMR-PIC Plasma Simulation Code with Dynamic Domain Decomposition

To maintain load balance among processes in parallel calculation, we introduced a dynamic load balancing technique called Dynamic Domain Decomposition (DDD) into our newly developed multi-scale Particle-In-Cell (PIC) simulation code in which Adaptive Mesh Refinement (AMR) is incorporated. To evaluate the effectiveness of DDD, we performed test simulations with a model in which four particle clusters are non-uniformly distributed with different velocities. We confirmed that load imbalance among processes caused by non-uniform plasma distribution was successfully resolved by DDD and the computational time becomes almost half of that for simulation of the same model without using DDD.