Parallel Lattice Research Articles

Numerical predictions of laminar and turbulent forced convection: Lattice Boltzmann simulations using parallel libraries

This paper presents the performance comparison of various parallel lattice Boltzmann codes for simulation of incompressible laminar convection in 2D and 3D channels. Five different parallel libraries namely; matlabpool, pMatlab, GPU-Matlab, OpenMP and OpenMP+OpenMPI were used to parallelize the serial lattice Boltzmann method code. Domain decomposition method was adopted for parallelism for 2D and 3D uniform lattice grids. Bhatnagar-Gross-Krook approximation with lattice types D2Q9, D2Q19 and D2Q5, D2Q6 were considered to solve 2D and 3D fluid flow and heat transfer respectively. Parallel computations were conducted on a workstation and an IBM HPC cluster with 32 nodes. Laminar forced convection in a 2D and turbulent forced convection in a 3D channels was considered as a test case. The performance of parallel LBM codes was compared with serial LBM code. Results show that for a given problem, parallel simulations using matlabpool and pMatlab library perform almost equal. Parallel simulations using C language with OpenMP libraries were 10 times faster than simulations involving Matlab parallel libraries. Parallel simulations with OpenMP+OpenMPI were 0.35 times faster than the reported parallel lattice Boltzmann method code in the literature.

Numerical simulation of fluid flow and heat transfer in rotating channels using parallel lattice Boltzmann method

This paper presents a numerical modeling and simulation of incompressible laminar mixed convection in rotating channels using parallel lattice Boltzmann method. Individual distribution functions with D3Q19 and D3Q6 lattice types were considered to solve fluid flow and heat transfer problems, respectively. The Reynolds number was set to 100, and wall-to-inlet fluid density ratio was set to 0.2. Two rotation modes namely orthogonal and parallel modes were considered with rotation number equal to 0.2. LBM code was written in C language and was parallelized using OpenMP libraries. Domain decomposition method of data parallelism was adopted here, and simulations were conducted on a workstation with dual processors and 64GB RAM. Predicted velocity and temperature fields were found to agree well with velocity and temperature obtained from Fluent.

Analysis of heterogeneity and permeability anisotropy in carbonate rock samples using digital rock physics

The heterogeneity of a carbonate reservoir exists at all scales and always presents a challenge because it cannot be studied visually during laboratory measurements. Permeability is an important parameter and plays a key role in enhanced oil recovery operations. However, permeability anisotropy is observed at different scales of hydrocarbon production, especially in carbonate reservoirs, and complicates reservoir characterization and description. Digital rock physics (DRP) uses digitized images of reservoir rocks to compute their properties, such as porosity and directional permeability, and provide a better view of the internal structure of the rock sample, which can be used to study the heterogeneity and permeability anisotropy in carbonate rocks and reservoirs.In this study, we processed and used images of five carbonate rock samples obtained from three-dimensional (3D) computed tomography (CT) scans to calculate porosity and directional permeability. Subsamples taken at three positions along each original core plug image were processed to segmented images using image processing methods. Porosities were calculated by the voxel statistics method based on a segmented image. Absolute permeabilities of these subsamples in the three main perpendicular directions (one axial, z, and two transverse, x and y) were calculated by applying the parallel lattice Boltzmann method (PLBM) in our high-performance computing cluster (HPCC). The heterogeneity and permeability anisotropy were analyzed by combining the experimental results, the rock images, and simulated results, which included visualization of the velocity field and calculated parameter values. By analyzing these data, we found that these carbonate rock samples had strong heterogeneity and clear permeability anisotropy. In addition, we also found a close relationship between the measured values of the whole core samples and the maximum (minimum) values of the simulations for porosity (permeability). We verified that the characteristics of pore geometry, flow velocity distribution, and transport properties can be captured using DRP.

Lattice Boltzmann parallel simulation of microflow dynamics over structured surfaces

In the present work, a parallel lattice Boltzmann multiphase model was developed to investigate the effects of surface structures on wettabilities and flow dynamics in a microchannel. The theory of wetting transition was firstly discussed. Then three types including triangular, rectangle and hierarchical shaped microstructures were constructed on the surface to examine the effects on wettabilities and drag reduction. It was found that flow behaviour is strongly affected by the surface morphology of the channel. The results indicated that hierarchical structures on the surface could improve the hydrophobicity significantly. For rectangular structures, they can improve the hydrophobicity with the increase of height and distance ratio h/d of the structures, and the improvement will reach its optimal hydrophobicity when the value h/d is over a certain value of 0.6. Moreover, to accelerate computational speed, the Open Multi-Processing (OpenMP) was employed for the parallelization of the model. A maximum speedup of 2.95 times was obtained for 4 threads on a multi-core CPU platform.

Simulation of gas flow in organic-rich mudrocks using digital rock physics

Organic-rich mudrock systems play an important role in the world's volatile petroleum industry. Due to their complex pore structures with a large portion of nano pores, the flow mechanics of gas flow in the pore space may involve nanoscale effects that could not be accounted for by classic Darcy's law. It is also very difficult to experimentally probe the transport properties of extremely low permeability rock samples. Digital rock physics (DRP) obtains a direct view of the internal structure of reservoir rock sample by digitized high resolution images to compute their properties, such as porosity and directional permeability, and hence to enable the studies of multi-scale flow mechanics in mudrocks. In this study, we present a parallel lattice Boltzmann method (PLBM) for studying nanoscale effects on a three dimensional (3D) tube flow, and then to simulate the flow process of the gas flow in real 3D pore geometry of two perpendicularly cored mudrock samples (S62H and S62V), combining with some image processing methods in DRP. The pore geometry is obtained by 3D digital image data based on ultra-high resolution X-ray computed tomography (XCT) imaging. The numerical advantages of lattice Boltzmann method (LBM) make it a powerful tool to simulate gas flow in such nano pores and estimate the absolute permeability. The new parallel implementation has been successfully performed on a powerful high performance computing cluster (HPCC) and validated with the analytical solution of a 3D tube flow model, which is taken as the elementary model for the pore networks. By simulating the nanoscale effects of rarefaction, slippage and roughness on gas flow in a nano tube model, we propose some more realistic parameters for simulating the gas flow in real mudrocks to study the flow mechanics, and to calculate the apparent directional permeability. The results show: 1) the simulated porosities (7.66% for S62H and 2.58% for S62V) are close to measured porosities (5.00% for S62H and 1.76% for S62V); 2) the estimated apparent permeability of S62H is 0.032 mD, which is close to the measured permeability of 0.02 mD; 3) the apparent directional permeability of S62V is not able to be estimated because the identified pores of subsamples are not connected in any direction with the image resolution of 65 nm/voxel; 4) the permeability anisotropy of S62H is clearly observed through the velocity field visualization and calculated permeability values; 5) the characteristics of pore geometry, flow velocity distribution and transport properties can be captured using DRP and the PLBM is an effective way to simulate complex gas flow in kerogen pores and estimate the apparent directional permeability.

Local heteroepitaxy as an adhesion mechanism in aluminium coatings cold gas sprayed on AlN substrates

Cold gas sprayed Al coatings deposited onto wurtzitic AlN substrates show excellent adhesion. As a possible adhesion mechanism, the local heteroepitaxy between Al and AlN was considered and verified experimentally in Al coatings, which were deposited using magnetron sputtering or cold gas spraying on single-crystalline and polycrystalline AlN substrates. Analysis of the local orientation relationships at the Al/AlN interfaces revealed that preferentially such lattice planes of Al align parallel with the upright lattice planes of AlN, which possess similar interplanar distances. The matching lattice planes in the Al coatings grew as continuations of the lattice planes in the AlN substrates. In all samples under study, the parallel alignment of the lattice planes {220}Al and {110}AlN was found. Additional orientation relationships between Al and AlN arose if parallel lattice planes with similar interplanar spacing could be found in both counterparts via rotation of the lattice planes {220}Al around their normal direction. Still, the oriented growth of Al on AlN is only possible if Al atoms in the deposited coatings are mobile enough to rearrange along the AlN surface. Whereas the mobility of Al atoms in a magnetron sputtering process is expected to be sufficiently high, the intrinsic mobility of Al atoms in the cold gas sprayed particles is anticipated to be low. However, the auxiliary microstructure analyses have shown that local recrystallization and partial melting are two phenomena, which can facilitate the rearrangement of Al atoms within the cold gas sprayed coating.

Simulation of micro aggregate breakage in turbulent flows by the cumulant lattice Boltzmann method

The production of nano-particles from larger aggregates is an important industrial process, especially for life-science products. A micro-machined disperser is studied numerically by the cumulant lattice Boltzmann method. The aggregates are modeled as tracer particles with mass and drag coefficient. They record the history of the experienced stresses and the relative velocity of aggregates with respect to the fluid. The tracer particles are implemented in a massively parallel multi-resolution lattice Boltzmann framework. In this paper, the effect of fractal dimension and the number of primary particles on the breakage of aggregates due to the relative velocities of the particles with respect to the surrounding fluid are investigated. It is found that aggregates with realistic geometry (fractal number 1.85) usually have Stokes numbers smaller than one such that the load on these aggregates is dominated by the strain rate in the surrounding fluid. This is in contrast to spherical particles (fractal number 3) that have larger Stokes numbers exceeding one such that the load from their relative velocity with respect to the surrounding fluid is not negligible.

Massively parallel lattice–Boltzmann codes on large GPU clusters

This paper describes a massively parallel code for a state-of-the art thermal lattice- Boltzmann method. Our code has been carefully optimized for performance on one GPU and to have a good scaling behavior extending to a large number of GPUs. Versions of this code have been already used for large-scale studies of convective turbulence. GPUs are becoming increasingly popular in HPC applications, as they are able to deliver higher performance than traditional processors. Writing efficient programs for large clusters is not an easy task as codes must adapt to increasingly parallel architectures, and the overheads of node-to-node communications must be properly handled. We describe the structure of our code, discussing several key design choices that were guided by theoretical models of performance and experimental benchmarks. We present an extensive set of performance measurements and identify the corresponding main bot- tlenecks; finally we compare the results of our GPU code with those measured on other currently available high performance processors. Our results are a production-grade code able to deliver a sustained performance of several tens of Tflops as well as a design and op- timization methodology that can be used for the development of other high performance applications for computational physics.

A Parallel Non-Dimensional Lattice Boltzmann Method for fluid flow and heat transfer with solid–liquid phase change

A new mesoscopic scale Parallel Non-Dimensional Lattice Boltzmann Method (P-NDLBM) is developed to dramatically speed up the computation of transient fluid flow and heat transfer problems. The P-NDLBM code using message passing interface (MPI) was compiled in Fortran. The model is presented in dimensionless form to simplify application to a broad range of problems. The effective domain decomposition, data transfer between CPUs, and various boundary conditions based on the P-NDLBM are presented. The code is validated by comparison to prior experimental and single CPU computational studies of cavity flow and the Rayleigh–Bénard problem. The time costs of simulations with equal mesh size are compared for CPU numbers from 1 to 64 to show the effectiveness of the approach. To illustrate the utility of the code, simulations of the transient temperature and fluid velocity during charging and discharging of an integral collector storage solar system are presented. The modeled system has encapsulated phase change embedded in the water-filled enclosure. The results illustrate the capability of the code to capture the phase transition within the encapsulated phase change material as well as the details of the overall flow structure and temperature field.

Predictive wind turbine simulation with an adaptive lattice Boltzmann method for moving boundaries

Operating horizontal axis wind turbines create large-scale turbulent wake structures that affect the power output of downwind turbines considerably. The computational prediction of this phenomenon is challenging as efficient low dissipation schemes are necessary that represent the vorticity production by the moving structures accurately and that are able to transport wakes without significant artificial decay over distances of several rotor diameters. We have developed a parallel adaptive lattice Boltzmann method for large eddy simulation of turbulent weakly compressible flows with embedded moving structures that considers these requirements rather naturally and enables first principle simulations of wake-turbine interaction phenomena at reasonable computational costs. The paper describes the employed computational techniques and presents validation simulations for the Mexnext benchmark experiments as well as simulations of the wake propagation in the Scaled Wind Farm Technology (SWIFT) array consisting of three Vestas V27 turbines in triangular arrangement.

Investigating cross-wind stability of high-speed trains with large-scale parallel CFD

ABSTRACT The side-wind loading on a simplified train model at scale 1:25 is investigated by parallel large eddy simulation (LES) with incompressible solvers from the OpenFOAM package and a novel dynamically adaptive, parallel LES-type lattice Boltzmann method (LBM) implemented in our own AMROC framework. It is found that the new LBM code provides more accurate time-averaged force predictions, while compute times are reduced.

Performance and portability of accelerated lattice Boltzmann applications with OpenACC

SummaryAn increasingly large number of HPC systems rely on heterogeneous architectures combining traditional multi‐core CPUs with power efficient accelerators. Designing efficient applications for these systems have been troublesome in the past as accelerators could usually be programmed using specific programming languages threatening maintainability, portability, and correctness. Several new programming environments try to tackle this problem. Among them, OpenACC offers a high‐level approach based on compiler directives to mark regions of existing C, C++, or Fortran codes to run on accelerators. This approach directly addresses code portability, leaving to compilers the support of each different accelerator, but one has to carefully assess the relative costs of portable approaches versus computing efficiency. In this paper, we address precisely this issue, using as a test‐bench a massively parallel lattice Boltzmann algorithm. We first describe our multi‐node implementation and optimization of the algorithm, using OpenACC and MPI. We then benchmark the code on a variety of processors, including traditional CPUs and GPUs, and make accurate performance comparisons with other GPU implementations of the same algorithm using CUDA and OpenCL. We also asses the performance impact associated with portable programming, and the actual portability and performance‐portability of OpenACC‐based applications across several state‐of‐the‐art architectures. Copyright © 2016 John Wiley & Sons, Ltd.

Massively Parallel Algorithms for the Lattice Boltzmann Method on NonUniform Grids

The lattice Boltzmann method exhibits excellent scalability on current supercomputing systems and has thus increasingly become an alternative method for large-scale non-stationary flow simulations, reaching up to a trillion grid nodes. Additionally, grid refinement can lead to substantial savings in memory and compute time. These saving, however, come at the cost of much more complex data structures and algorithms. In particular, the interface between subdomains with different grid sizes must receive special treatment. In this article, we present parallel algorithms, distributed data structures, and communication routines that are implemented in the software framework waLBerla in order to support large-scale, massively parallel lattice Boltzmann-based simulations on non-uniform grids. Additionally, we evaluate the performance of our approach on two current petascale supercomputers. On an IBM Blue Gene/Q system, the largest weak scaling benchmarks with refined grids are executed with almost two million threads, demonstrating not only near-perfect scalability but also an absolute performance of close to a trillion lattice Boltzmann cell updates per second. On an Intel-based system, the strong scaling of a simulation with refined grids and a total of more than 8.5 million cells is demonstrated to reach a performance of less than one millisecond per time step. This enables simulations with complex, non-uniform grids and four million time steps per hour compute time.

Parallel‐computing‐based implementation of fast algorithms for discrete Gabor transform

Parallel-computing-based implementation of the two recent fast parallel algorithms for the discrete Gabor transform (DGT) is presented in this paper. First of all, the first existing block time-recursive DGT algorithm with parallel lattice structure is analysed, and then an improved implementation method under a parallel computing environment is presented. Each parallel channel (i.e. process in parallel computing) in the improved method is independent, thereby reducing the interprocess communication by 99.2% on average over the original algorithm. Second, the second existing fast parallel DGT algorithm based on multirate filtering is analysed. Through the use of parallel computing, the communication overhead of the multirate filtering-based parallel DGT algorithm is optimised and its time efficiency is raised from 31.26 times to 54.52 times faster than the serial fast DGT algorithm in processing of long sequences. Finally, the experimental results are compared and analysed, which indicate that the proposed fast DGT implementation methods are attractive for real-time signal processing.

Numerical Modelling and Simulation of Seepage in Tight Sandstone Based on Parallel Lattice Boltzmann Method

Numerical Modelling and Simulation of Seepage in Tight Sandstone Based on Parallel Lattice Boltzmann Method

Parallel lattice Boltzmann simulation of bubble rising and coalescence in viscous flows

A parallel three-dimensional lattice Boltzmann scheme for multicomponent immiscible fluids is proposed to simulate bubble rising and coalescence process in viscous flows. The lattice Boltzmann scheme is based on the free-energy model and is parallelized in the share-memory model by using the OpenMP. Bubble interface is described by a diffusion interface method solving the Cahn–Hilliard equation and both the surface tension force and the buoyancy are introduced in a form of discrete body force. To avoid the numerical instability caused by the interface deformation, the 18 point finite difference scheme is utilized to calculate the first- and second-order space derivative. The correction of the parallel scheme handling three-dimensional interfaces is verified by the Laplace law and the dynamic characteristics of an isolated bubble in stationary flows. Subsequently, effects of the initially relative position, accompanied by the size ratio on bubble–bubble interaction are studied. The results show that the present scheme can effectively describe the bubble interface dynamics, even if rupture and restructure occurs. In addition to the repulsion and coalescence phenomenon due to the relative position, the size ratio also plays an insignificant role in bubble deformation and trajectory.

Lattice–Boltzmann simulation of fluid flow through packed beds of uniform ellipsoids

Fluid flow through packed beds of uniform ellipsoids is studied by a parallel lattice–Boltzmann (LB) model. The packed beds are generated by means of discrete element method and have a wide range of particle sphericity. First, the LB model is used to validate the drag force coefficient of a single ellipsoid. The validated model is then extended to study the effects of sphericity and porosity on the fluid flow and quantify the drag force on particles in packed beds. Some unique flow structures are identified in relation to tortuous pore structures. Due to non-uniform structure and flow, the fluid-particle interaction forces vary giving a distribution. The results suggest that the mean drag forces on oblate particles are more sensitive to aspect ratio and hence sphericity than prolate ones. Based on the simulated data, a new equation is proposed to calculate the mean drag force on particles. The results show that the new equation is more accurate than the correlations in the literature.

Orthogonal and parallel lattice plasmon resonance in core-shell SiO(2)/Au nanocylinder arrays.

Height induced coupling behavior between the plasmonic modes and diffraction orders were studied in the core-shell SiO(2)/Au nanocylinder arrays (NCAs) using finite difference time domain (FDTD) simulations. New lattice plasmon modes (LPMs) are observed in the structures with high aspect ratio. Specifically, parallel coupling between the plasmonic modes and diffraction orders is obtained here, which shows different coupling behavior from orthogonal LPMs. Electromagnetic (EM) field distributions indicate that horizontal propagation of the magnetic or electric field component is responsible for the generation of these orthogonal and parallel LPMs, respectively. Radiative loss could be effectively suppressed when the height increases. This is important for the applications of fluorescence enhancement and nano laser. Further studies confirm that the LPMs associated with the superstrate diffraction orders could be well maintained even when the Au coating is imperfect. The interference from the substrate associated LPMs could be eliminated by cutting off the corresponding diffraction waves by inducing a Si(3)N(4) substrate. This study of coupling behavior in the core-shell NCAs enables a novel route to design and optimize the LPMs for applications of bio-sensing and nano laser.

Efficient Parallel Computing-Based Implementation Methods of DCT-Kernel-Based Real-Valued Discrete Gabor Transform and Expansion

The existing researches of fast parallel algorithms for DCT-based real-valued discrete Gabor transform and expansion are limited to theoretical analysis. In this paper, parallel computing based implementations of parallel lattice structure algorithm are presented. The communication issues are not considered in original algorithms which enable the efficiency lower than serial fast algorithm in implementation. In view of this, improved implementation methods are proposed both for transform and expansion. The recursive part of the original algorithms is expanded into an iterative form. This makes the interprocess communication converted into serial calculation. Thus, each parallel channel (i.e., process in parallel computing) in the improved method is independent, thereby reducing the interprocess communication greatly. Finally, algorithms are tested on a parallel computer. The experimental results are compared and analyzed, which indicate that the proposed implementation methods are attractive for real-time signal processing as compared to the existing parallel lattice structure algorithm and the fastest serial algorithm.

Parallel Lattice Boltzmann Computing and Applications in Core Sample Feature Evaluation

Micro-CT scans with QEMSCAN mapping provide visualization of core samples to quantify heterogeneous physical properties important for subsurface flow including, as examples, pore size distribution and connectivity, mineral compositions, porosity and permeability, amongst many others. 3D high-resolution micro-CT scans can deliver a very high level of microstructures detail, which also implies enormous numerical data sets and associated computational processing load. It is, therefore, important to understand (1) the voxel resolution of micro-CT scans required to retain physical structure fidelity (e.g., mineral compositions, pore size and throats, and porosity and tortuosity), (2) the sensitivity of individual mineral property and voxel resolution on the directional permeability, and (3) the smallest sample size that provides reliable and representative transport calculations (e.g., directional permeability and connectivity). The lattice Boltzmann method is capable of simulating flow in both open pore spaces and porous media and is used here to allow for flow in multiple matrices (quartz aggregate and low-permeable clay matrix). As an application example, a permeability study of the Precipice Sandstone from the Chinchilla 4 well in the Surat Basin has been conducted. Regarding the Chinchilla sample, we established that (1) the composition ratio is relatively sensitive to voxel resolution: higher resolution imaging is required to retain narrow pore throats and excessively coarsened voxel resolutions result in severe loss of internal microstructures information; (2) both voxel resolutions and individual mineral properties affect flow dynamics, and the clay permeability slightly affects the whole permeability at \(\sim \)nD scale; (3) it is not feasible to define the accurate ratio of lattice nodes versus pore apertures for meeting the grid independence due to the complex sample tortuosity; and (4) the directional permeability approaches a constant value as the sample size increases to its representative elementary volume scale size (10 mm in this case). Significantly smaller sample sizes cannot retain the representative physical structure fidelity even using higher resolution imaging.