Parallel Mesh Research Articles

Fully Parallel Mesh I/O Using PETSc DMPlex with an Application to Waveform Modeling

Large-scale PDE simulations using high-order finite-element methods on unstructured meshes are an indispensable tool in science and engineering. The widely used open-source PETSc library offers an efficient representation of generic unstructured meshes within its DMPlex module. This paper details our recent implementation of parallel mesh reading and topological interpolation (computation of edges and faces from a cell-vertex mesh) into DMPlex. We apply these developments to seismic wave propagation scenarios on Mars as an example application. The principal motivation is to overcome single-node memory limits and reach mesh sizes which were impossible before. Moreover, we demonstrate that scalability of I/O and topological interpolation goes beyond 12,000 cores, and memory-imposed limits on mesh size vanish.

Parallel Efficient Mesh Deformation Method Based On Support Vector Regression

Mesh deformation method is widely used in unsteady numerical simulations involving moving boundaries. This kind of method redistributes the position of grid points in accordance with the movement of the computational domain without changing their connectivity relations. In this paper, we present a parallel mesh deformation method based on the support vector machine regression (SVR). In each time step, the proposed method first trains three SVRs by the coordinates of the boundary points and their known displacements in each direction, and then predicts the displacements of the internal points using the SVRs. After deforming the mesh, the dual-time step flow solver is used to solve the governing equations. To ensure the consistency of the method running in parallel, the training part of the method is executed with all global boundary points in each decomposed domain. Therefore, each CPU needs to maintain a copy of the entire boundary points via a point-to-point communication. The internal evaluation of the method is predicted separately in each decomposed domain without any data dependency. An oscillatory and transient pitching airfoil case is simulated to demonstrate the applicability of the proposed mesh deformation method, and its parallel efficiency is over 60% with 64 cores.

Numerical results for adaptive (negative norm) constrained first order system least squares formulations

We perform a followup computational study of the recently proposed space–time first order system least squares ( FOSLS ) method subject to constraints referred to as CFOSLS where we now combine it with the new capability we have developed, namely, parallel adaptive mesh refinement (AMR) in 4D. The AMR is needed to alleviate the high memory demand in the combined space time domain and also allows general (4D) meshes that better follow the physics in space–time. With an extensive set of computational experiments, performed in parallel, we demonstrate the feasibility of the combined space–time AMR approach in both two space plus time and three space plus time dimensions.

Tetrahedral adaptive mesh refinement for two‐phase flows using conservative level‐set method

SummaryIn this article, we describe a parallel adaptive mesh refinement strategy for two‐phase flows using tetrahedral meshes. The proposed methodology consists of combining a conservative level‐set method with tetrahedral adaptive meshes within a finite volume framework. Our adaptive algorithm applies a cell‐based refinement technique and adapts the mesh according to physics‐based refinement criteria defined by the two‐phase application. The new adapted tetrahedral mesh is obtained from mesh manipulations of an input mesh: operations of refinement and coarsening until a maximum level of refinement is achieved. For the refinement method of tetrahedral elements, geometrical characteristics are taking into consideration to preserve the shape quality of the subdivided elements. The present method is used for the simulation of two‐phase flows, with surface tension, to show the capability and accuracy of 3D adapted tetrahedral grids to bring new numerical research in this context. Finally, the applicability of this approach is shown in the study of the gravity‐driven motion of a single bubble/droplet in a quiescent viscous liquid on regular and complex domains.

Dynamic reconfiguration of unbalanced distribution network considering DG uncertainties

In order to solve the multi-objective dynamic reconfiguration problem of the unbalanced source distribution network, this paper proposes a multi-objective dynamic reconfiguration strategy for active distribution network based on affine number. The affine number is introduced in this paper to describe the uncertainty of distributed generation output. The active network loss, voltage deviation and load balance are the objective functions, the dynamic reconfiguration of unbalanced distribution network is modeled. The multi-objective particle swarm optimization algorithm based on Pareto entropy and parallel mesh is used to obtain the static reconfiguration solution in hours. Finally, a reconfiguration time division scheme based on Mahalanobis distance and target expectation is proposed, and the time period is divided again until the switch action constraint is satisfied.

Parallel hybrid mesh adaptation by refinement and coarsening

In this paper we propose a robust adaptation method for hybrid mesh including both refinement and coarsening. Elements of different types are refined through isotropic subdivision in a similar way. Their unrefined neighbours are converted to polyhedrons, which benefits to localize mesh refinement. Mesh elements are coarsened by reversing refinement based on a multi-levels data structure. Elements subdivision can be conducted in entirely parallel mode by introducing a non-overlap number range for each processor. Then feature data is devised to recover the uniqueness of new ghost points and elements through parallel communication. A repartition-migration method is adopted to rebalance the computing load. We illustrate our method by an application of flow simulation. With mesh adaptation, the structures of shocks and vortices are resolved better.

Lightweight task offloading exploiting MPI wait times for parallel adaptive mesh refinement

SummaryBalancing the workload of sophisticated simulations is inherently difficult, since we have to balance both computational workload and memory footprint over meshes that can change any time or yield unpredictable cost per mesh entity, while modern supercomputers and their interconnects start to exhibit fluctuating performance. We propose a novel lightweight balancing technique for MPI+X to accompany traditional, prediction‐based load balancing. It is a reactive diffusion approach that uses online measurements of MPI idle time to migrate tasks temporarily from overloaded to underemployed ranks. Tasks are deployed to ranks which otherwise would wait, processed with high priority, and made available to the overloaded ranks again. This migration is nonpersistent. Our approach hijacks idle time to do meaningful work and is totally nonblocking, asynchronous and distributed without a global data view. Tests with a seismic simulation code developed in the ExaHyPE engine uncover the method's potential. We found speed‐ups of up to 2‐3 for ill‐balanced scenarios without logical modifications of the code base and show that the strategy is capable to react quickly to temporarily changing workload or node performance.

Phase-field-lattice Boltzmann simulation of dendrite motion using an immersed boundary method

A phase-field-lattice Boltzmann model (PF-LBM) coupled with immersed boundary method is developed to simulate the growth and motion of dendrites against convection. A conservative phase-field model is employed to maintain mass conservation. A parallel adaptive mesh refinement (Para-AMR) approach is developed to improve the computational efficiency. Multi-dendrite growth and motion against 3-D cavity flow is simulated and simulation results reveal the morphological transition of moving dendrites, together with collision and coalescence. The interaction between dendrite and flow is accurately restored, enabling one to grasp and understand the microstructure evolution in practical solidification condition.

A scalable framework for the partitioned solution of fluid–structure interaction problems

In this work, we present a scalable and efficient parallel solver for the partitioned solution of fluid–structure interaction problems through multi-code coupling. Two instances of an in-house parallel software, TermoFluids, are used to solve the fluid and the structural sub-problems, coupled together on the interface via the preCICE coupling library. For fluid flow, the Arbitrary Lagrangian–Eulerian form of the Navier–Stokes equations is solved on an unstructured conforming grid using a second-order finite-volume discretization. A parallel dynamic mesh method for unstructured meshes is used to track the moving boundary. For the structural problem, the nonlinear elastodynamics equations are solved on an unstructured grid using a second-order finite-volume method. A semi-implicit FSI coupling method is used which segregates the fluid pressure term and couples it strongly to the structure, while the remaining fluid terms and the geometrical nonlinearities are only loosely coupled. A robust and advanced multi-vector quasi-Newton method is used for the coupling iterations between the solvers. Both the fluid and the structural solver use distributed-memory parallelism. The intra-solver communication required for data update in the solution process is carried out using non-blocking point-to-point communicators. The inter-code communication is fully parallel and point-to-point, avoiding any central communication unit. Inside each single-physics solver, the load is balanced by dividing the computational domain into fairly equal blocks for each process. Additionally, a load balancing model is used at the inter-code level to minimize the overall idle time of the processes. Two practical test cases in the context of hemodynamics are studied, demonstrating the accuracy and computational efficiency of the coupled solver. Strong scalability test results show a parallel efficiency of 83% on 10,080 CPU cores.

Mesh generation for thin layered domains and its application to parallel multigrid simulation of groundwater flow

The generation of detailed three dimensional meshes for the simulation of groundwater flow in thin layered domains is crucial to capture important properties of the underlying domains and to reach a satisfying accuracy. At the same time, this level of detail poses high demands both on suitable hardware and numerical solver efficiency. Parallel multigrid methods have been shown to exhibit near optimal weak scalability for massively parallel computations of density driven flow. A fully automated parameterized algorithm for prism based meshing of coarse grids from height data of individual layers is presented. Special structures like pinch outs of individual layers are preserved. The resulting grid is used as a starting point for parallel mesh and hierarchy creation through interweaved projected refinement and redistribution. Efficiency and applicability of the proposed approach are demonstrated for a parallel multigrid based simulation of a realistic sample problem.

Study and Verification of Large-Scale Parallel Mesh Generation Algorithm for Centrifugal Pump

In order to reveal the details of the internal flow in a centrifugal pump, a large-scale mesh is needed. However, the mesh generated by the serial grid algorithm cannot meet the calculation requirements due to the huge amount of time. A large-scale parallel mesh generation algorithm of a centrifugal pump for high-performance computers is presented in this paper. First, a grid point set for the 3D Delaunay triangular mesh on the surface of the centrifugal pump is generated. Then, the S-H (Sutherland–Hodgman) algorithm for cropping and segmenting these grid point sets on the surface is employed. A uniform boundary mesh is generated and is divided into different subregions. In addition, in order to ensure the consistency of the interface mesh and to avoid the boundary mesh intersection overlap error, a parallel constrained Delaunay mesh generation algorithm based on region numbering is proposed, which can improve the quality and efficiency of the generated parallel mesh. Finally, the centrifugal pump is tested for verifying the parallel mesh algorithm in the Tianhe-2 supercomputer. PIV (particle image velocimetry) internal flow experiment is comparatively analyzed with the numerical simulation of large-scale mesh. The results show that the algorithm can generate 108 3D unstructured grid elements in 5 minutes, and the parallel efficiency can achieve 80%. The proposed algorithm not only ensures high grid quality with the serial grid algorithm but also accurately simulates the flow law in the centrifugal pump. The double-vortex structure which is obtained by PIV experiment is captured by the large-scale mesh.

Parallel and automatic isotropic tetrahedral mesh generation of misaligned assemblies

Mesh generation is a challenge for high-performance numerical simulation, one reason is the complex geometry representing solution domain makes pre-processing difficult, especially for those assembly model containing hundreds and thousands of components involving misaligned interfaces between neighboring parts, and no state-of-art meshing tools could provide automatic functions for processing such complex model, another reason is hundreds of millions or even billions meshes should be generated quickly, which also exceeds the capabilities of available tools. In this paper, a novel parallel and automatic mesh generation method is proposed. Firstly, a surface imprinting algorithm based on the hybrid representation of discrete and continuous surfaces is proposed to process misaligned assembly model automatically. Then, the repaired assembly model is used as an input for a carefully designed mesh generation pipeline which connects the procedures of mesh sizing control, and three-level parallel tetrahedral mesh generation in order. This proposed method could produce hundreds of millions consistent mesh qualified for high-performance numerical simulation based on thousands of geometry components. Numerical experiments on a giant dam model and an integrated circuit board model demonstrates the effectiveness of this method.

A consistent parallel isotropic unstructured mesh generation method based on multi-phase SPH

In this paper, we propose a consistent parallel unstructured mesh generator based on a multi-phase SPH method. A set of physics-motivated modeling equations are developed to achieve the targets of domain decomposition, communication volume optimization and high-quality unstructured mesh generation simultaneously. A unified density field is defined as the target function for both partitioning the geometry and distributing the mesh-vertexes. A multi-phase Smoothing Particle Hydrodynamics (SPH) method is employed to solve the governing equations. All the optimization targets are achieved implicitly and consistently by the particle relaxation procedure without constructing triangulation/tetrahedralization explicitly. The target of communication reduction is achieved by introducing a surface tension model between distinct partitioning sub-domains, which are characterized by colored SPH particles. The resulting partitioning diagram features physically localized sub-domains and optimized interface communication. The target of optimizing the mesh quality is achieved by introducing a tailored equation-of-state (EOS) and a smooth isotropic kernel function. The mesh quality near the interface of neighboring sub-domains is improved by gradually removing the surface-tension force once a steady state is achieved. The proposed method is developed basing on a new parallel environment for multi-resolution SPH to exploit both coarse- and fine-grained parallelism. A set of benchmarks are conducted to verify that all the optimization targets are achieved consistently within the current framework.

Investigation of processing factors affecting flexible heating wire by coating polyester yarns with carbon nanotubes

A method for preparing a flexible heating wire coated with carbon nanotubes on a surface of a polyester yarn is proposed in view of the problems of poor flexibility and water washing resistance of the existing heating wire. Based on the flexible heating wire forming system, the orthogonal test method was used to study the effects of process parameters such as solution concentration and polymer content on the formation of flexible heating wire; the prepared heating wire was subjected to resistance measurement, microscopic morphology observation, heating and washing resistance tests. The results show that the optimal parameter parameters for the preparation of flexible hot wire is A3B3C2D2. In this combination, the polymer content is 15 g/L and the heating wire resistance is 180.5 Ω/cm. Moreover, the photomicrograph shows that the surface has a conductive layer composed of polymer and carbon nanotubes, which has no obvious protrusions and breaks; the surface of the parallel mesh structure can obtain a heating temperature of 52.2°C at a voltage of 12 V; the resistance of heating wire does not increase with latter washing after the first washing.

Microstructural evolution mechanism of semi-solid slurry: a study using Phase-Field-Lattice-Boltzmann scheme

Abstract The formation mechanism study of a spherical crystal is the theoretical basis for preparing high-quality semi-solid slurries with homogeneous spherical microstructure. The interaction between thermal-solute diffusion during solidification and solid-liquid interface migration determines the morphology evolution of microstructure according to the representation theory. Comprehensive research about the effects of local variates on the microstructural evolution of semi-solid slurry in a Representative Elementary Volume (REV) by 2D cases was conducted. A modified Phase-Field-Lattice-Boltzmann scheme using techniques of parallel computing and adaptive mesh refinement (Para-AMR) was adopted. Actual processing parameters were coupled with the numerical scheme. Mesoscopic results were upscaled by averaging them over the simulation domain. Five aspects, growth space, initial undercooling degree, cooling rate, natural convection, and forced convection, were investigated. It is found that the formation of spherical grain requires a specific initial undercooling degree, cooling rate, and nuclei density. Study results indicate that melt convection has little effect on the morphological evolution of grain in the case of rapid isotropic growth, which was verified by experiments using the SEED process. Besides, forced convection has a significant effect on the morphology evolution of grain controlled by solute diffusion, which provides a possibility of an extension of the process window about pouring temperature and alloys with narrow solidification range for preparing semi-solid slurries.

A Generic Finite Element Framework on Parallel Tree-Based Adaptive Meshes

In this work we formally derive and prove the correctness of the algorithms and data structures in a parallel, distributed-memory, generic finite element framework that supports h-adaptivity on computational domains represented as forest-of-trees. The framework is grounded on a rich representation of the adaptive mesh suitable for generic finite elements that is built on top of a low-level, light-weight forest-of-trees data structure handled by a specialized, highly parallel adaptive meshing engine, for which we have identified the requirements it must fulfill to be coupled into our framework. Atop this two-layered mesh representation, we build the rest of data structures required for the numerical integration and assembly of the discrete system of linear equations. We consider algorithms that are suitable for both subassembled and fully-assembled distributed data layouts of linear system matrices. The proposed framework has been implemented within the FEMPAR scientific software library, using p4est as a practical forest-of-octrees demonstrator. A strong scaling study of this implementation when applied to Poisson and Maxwell problems reveals remarkable scalability up to 32.2K CPU cores and 482.2M degrees of freedom. Besides, a comparative performance study of FEMPAR and the state-of-the-art deal.ii finite element software shows at least comparative performance, and at most factor 2-3 improvements in the h-adaptive approximation of a Poisson problem with first- and second-order Lagrangian finite elements, respectively.

A Three-Dimensional Cartesian Mesh Generation Algorithm Based on the GPU Parallel Ray Casting Method

Robust and efficient Cartesian mesh generation for large-scale scene is of great significance for fluid dynamics simulation and collision detection. High-quality and large-scale mesh generation task in a personal computer is hard to achieve. In this paper, a parallel Cartesian mesh generation algorithm based on graphics processing unit (GPU) is proposed. The proposed algorithm is optimized based on the traditional ray casting method in computer graphics, and is more efficient and stable for large-scale Cartesian mesh generation. In the process of mesh generation, the geometries represented by triangular facets are transformed into a mesh composed of orthogonal hexahedrons. A parallel ray generation method is proposed to reduce the data exchange between the host memory and device memory. A parallel primitives searching method based on lattice grid is adopted to search the triangular facets for intersection calculation between rays and triangles. The parallel Cartesian mesh generation algorithm has been implemented using CUDA library. The performance of parallel Cartesian mesh generation algorithm has been promoted enormously compared with the traditional the sequential algorithm, which is shown in different numerical experiments. Through some tests, the performance of parallel algorithm is analyzed, and the results show that the parallel computing power of the GPU is fully utilized. Finally, examples of Cartesian mesh generation are presented.

Preparation of superhydrophobic mesh and application in multi-stage parallel oil/water separation system

Oil/water separation has become a worldwide problem due to the industrial oil pollution and oil-spill accident. Here, the superhydrophobic stainless steel mesh was coated with hydrophobic fumed silica (TS720) and vinyltrimethoxysilane (VTMO) using dip-coating method. The water contact angle was measured to be 152 ± 0.8° and oil contact angle was 0°, which indicated that the coated mesh exhibits good superhydrophobic and superoleophilic properties. Therefore, the coated mesh could be applied to separate different kind of oil/water mixture and the separation efficiency of gasoline/water mixture was up to 95%. Furthermore, the coated mesh had a high recyclability that could maintain separation efficiency above 92.5% after 50 separation cycles, meanwhile, the VTMO/TS720 coated mesh also could keep good separation property under acid or alkali salt environment, showing good potential for industrialized oil/water separation. In addition, utilizing superhydrophobic mesh materials in multi-stage parallel oil/water separation device for industrialized oil/water separation was proposed in this work.

An automated approach for parallel adjoint-based error estimation and mesh adaptation

In finite element simulations, not all of the data are of equal importance. In fact, the primary purpose of a numerical study is often to accurately assess only one or two engineering output quantities that can be expressed as functionals. Adjoint-based error estimation provides a means to approximate the discretization error in functional quantities and mesh adaptation provides the ability to control this discretization error by locally modifying the finite element mesh. In the past, adjoint-based error estimation has only been accessible to expert practitioners in the field of solid mechanics. In this work, we present an approach to automate the process of adjoint-based error estimation and mesh adaptation on parallel machines. This process is intended to lower the barrier of entry to adjoint-based error estimation and mesh adaptation for solid mechanics practitioners. We demonstrate that this approach is effective for example problems in Poisson’s equation, nonlinear elasticity, and thermomechanical elastoplasticity.

Adaptive mesh refinement for steady flows in Nek5000

Adaptive mesh refinement is performed in the framework of the spectral element method augmented by approaches to error estimation and control. The h-refinement technique is used for adapting the mesh, where selected grid elements are split by a quadtree (2D) or octree (3D) structure. Continuity between parent–child elements is enforced by high-order interpolation of the solution across the common faces. Parallel mesh partitioning and grid management respectively, are taken care of by the external libraries ParMETIS and p4est. Two methods are considered for estimating and controlling the error of the solution. The first error estimate is local and based on the spectral properties of the solution on each element. This method gives a local measure of the L2-norm of the solution over the entire computational domain. The second error estimate uses the dual-weighted residuals method — it is based on and takes into account both the local properties of the solution and the global dependence of the error in the solution via an adjoint problem. The objective of this second approach is to optimize the computation of a given functional of physical interest. The simulations are performed by using the code Nek5000 and three steady-state test cases are studied: a two-dimensional lid-driven cavity at Re=7,500, a two-dimensional flow past a cylinder at Re=40, and a three-dimensional lid-driven cavity at Re=2,000 with a moving lid tilted by an angle of 30∘. The efficiency of both error estimators is compared in terms of refinement patterns and accuracy on the functional of interest. In the case of the adjoint error estimators, the trend on the error of the functional is shown to be correctly represented up to a multiplicative constant.