Block-structured Grids Research Articles

Bricks: A high-performance portability layer for computations on block-structured grids

From partial differential equations to the convolutional neural networks in deep learning, to matrix operations in dense linear algebra, computations on structured grids dominate high-performance computing and machine learning. The performance of such computations is key to effective utilization of the billions of US dollar’s worth of GPU-accelerated systems such computations are run on. Concurrently, the end of Moore’s law and Dennard scaling are driving the specialization of compute and memory architectures. This specialization often makes performance brittle (small changes in function can have severe ramifications on performance), non-portable (vendors are increasingly motivated to develop their programming models tailored for their specialized architectures), and not performance portable (even a given computation may perform very differently from one architecture to the next). The mismatch between computations that reference data that is logically neighboring in N-dimensional space but physically distant in memory motivated the creation of Bricks — a novel data-structure transformation for multi-dimensional structured grids that reorders data into small, fixed-sized bricks of contiguously-packed data. Whereas a cache-line naturally captures spatial locality in only one dimension of a structured grid, Bricks can capture spatial locality in three or more dimensions. When coupled with a Python interface, a code-generator, and autotuning, the resultant BrickLib software provides not only raw performance, but also performance portability across multiple CPUs and GPUs, scalability in distributed memory, user productivity, and generality across computational domains. In this paper, we provide an overview of BrickLib and provide a series of vignettes on how it delivers on the aforementioned metrics.

Heat flux predictions for hypersonic flows with an overset near body solver on an adaptive block-structured Cartesian off-body grid

The manual volume mesh generation process for body-fitted CFD codes can be a time-intensive process, particularly for complex flight vehicles. The work shown here presents a hybrid body-fitted/Cartesian overset CFD solver capable of automatic volume mesh generation capabilities and accurate surface heating predictions. The solver uses a body-conformal near body solver for capturing boundary layer effects and an off-body Cartesian solver with adaptive mesh refinement for shock and wake tracking within the CHAMPS CFD code. The formulated near body Cartesian solver is the first of its kind to be applied within the context of low and high enthalpy, hypersonic viscous flows. The developed solver is validated on a Mach 9.47 Mars Science Lander and a Mach 6 Orion Crew Exploration Vehicle in a low-enthalpy flow at various non-zero angles of attack followed by a Mach 25.2 AS-202 capsule at a 17.8-degree angle of attack for Earth re-entry. Validation is shown based on grid convergence studies as well as good agreement to available numerical and experimental data in consideration of surface heating and surface pressure predictions.

Discontinuous Galerkin method for the shallow water equations on complex domains using masked block-structured grids

We evaluate masked block-structured grids for ocean domains which allow to represent small-scale geometric features without resorting to very small blocks or excessive mesh resolution. The considered approach aims to combine the geometric flexibility of unstructured meshes with the computational efficiency of stencil-based discretizations and is implemented and tested in a quadrature-free discontinuous Galerkin shallow water solver. We investigate the accuracy and the computational performance of the scheme on a range of realistic ocean domains meshed with blocks of different size and provide some comparisons to results obtained on unmasked block-structured grids and unstructured meshes.

Параметрические исследования течений в микросоплах

The work is devoted to parametric investigations of the krypton flow in a conical micronozzle when flowing into a region with low pressure. The features of the flows are studied at various values of the stagnation pressure in the pre-nozzle volume, including the occurrence of a condensed phase in the flow. Mathematical modeling was carried out on the basis of a numerical solution of the complete system of Navier-Stokes equations, supplemented by the equation for the mass fraction of the condensate. The mathematical model takes into account the change in the coefficients of dynamic viscosity and thermal conductivity depending on the gas temperature. The problem was solved by the control volume method on a block-structured regular grid of quadrangular elements using schemes of the second order of accuracy. The equations were integrated with respect to time using the Runge-Kutta method. The calculations were carried out at stagnation pressures of 5, 10, and 15 atm for single-phase and two-phase flows. The distribution fields of temperature and Mach number in the nozzle and in the space behind it are presented. The axial distribution of pressure, temperature, and Mach number has been studied. It is shown that in the case of a single-phase flow, self-similarity of gas flows is observed. The pressure fields were similar, but in a dimensionless form they coincided to each other. In this case, the identity of the velocity and temperature fields was observed at different values of the stagnation pressure. The self-similarity of the flow is violated in the zone of formation of condensed particles. The dimensions of the zones of local temperature increase, as well as the intensity of heat release, depend on the given stagnation pressure, which is reflected in the velocity characteristics of the flow.

Forecasting the energy characteristics of a reversible hydraulic machine for a head up to 250 m

BACKGROUND: At present, in the pressure zone of 50650 m for pumped storage power plants, hydroelectric units with classic single-stage radial-axial reversible hydraulic machines, which have a relatively simple design of the impeller and cylindrical guide vane, have received the most widespread energy performance, but are relatively quiet, large-sized and metal-intensive hydraulic machines. Numerical research and design of this type of machines are given special attention. The current trend is the design of flow parts based on numerical simulation of the flow. The most well-known commercial software products that implement finite volume numerical simulations are Ansys Fluent, Ansys CFX, StarCD, Numeca, Flow Vision and CADRUN. In the work under consideration, the calculations were performed using the Ansys CFX software package version 2021R1. Today, due to the lack of numerical capacity, the task of developing and using a technique that will allow obtaining an acceptable result with optimal time spent on data preparation and computational studies remains an urgent task.
 AIMS: The aim of the work was to present an economical methodology for numerical simulation of energy characteristics.
 METHODS: The methodology consists in describing the problem statement, the computational grids used, and the assumptions made for the optimal use of computing resources without a significant loss in the accuracy of the results.
 Object of computational research: The presented article investigates the flow part of a radial-axial pump-turbine designed for application to a maximum head of up to 250 m.
 RESULTS: Numerical modeling of power characteristics of on-pump and turbine modes is performed. A brief description of the problem statement, computational grids used, and assumptions made is given. A comparison of calculation results with experimental data of model tests is presented. The comparison results are presented in the relative form for the main parameters: pressure, efficiency, reduced rotational speed and flow rate.
 CONCLUSIONS: It is recommended to use SST model of turbulence in a stationary statement in order to predict the power characteristics of pump-turbines. The use of economical block-structured grids, as well as the performing of calculations only in the region of one blade of the guide vanes, one impeller blade and a suction pipe with the use of preliminary results of calculations in a spiral chamber allow using computational resources optimally without significant loss of accuracy of the results.

The Adaptive Composite Block-Structured Grid Calculation of the Gas-Dynamic Characteristics of an Aircraft Moving in a Gas Environment

This paper considers the problem associated with the numerical simulation of the interaction between the cocurrent stream occurring near a monoblock moving in the gas medium and solid fuel combustion products flowing from a solid fuel rocket engine (SFRE). The peculiarity of the approach used is the description of gas-dynamic processes inside the combustion chamber, in the nozzle block, and the down jet based on a single calculation methodology. In the formulated numerical methodology, the calculation of gas-dynamic parameters is based on the solution of unsteady Navier–Stokes equations and the application of a hybrid computational grid. A hybrid block-structured computational grid makes it possible to calculate gas flow near bodies of complex geometric shapes. The simulation of the main phase of interaction, corresponding to the stationary mode of rocket flight in the Earth’s atmosphere, has been carried out. A conjugated simulation of the internal ballistics of SFRE and interaction of combustion products jets is conducted.

LES-Predicted Flow Patterning in a Newly-Designed Reference Test Sample with Relevance to IC Engine-Related Cooling Channels

<div class="section abstract"><div class="htmlview paragraph">A test sample configuration with a circular cross-section has been conceptualized to reproduce all geometrically relevant flow-guided elements - straight segments, deflections, bifurcations, impingement regions, confluence - as they can also be found in the cooling systems of realistic Internal Combustion (IC) engines. This newly-designed reference test sample is termed as Water Spider Geometry (WSG), with the shape inspired by the flow guidance around an IC engine cylinder head. Computational investigations are carried out within the framework of a BMWi (German Federal Ministry for Economic Affairs and Energy) project by applying a well-resolved, highly comprehensive Large Eddy Simulation aiming at providing a meaningful assessment of the isothermal flow topology within the WSG. The basis forms a fully-hexahedral, block-structured grid arrangement comprising 290 million cells with the results considered to be a reference solution for further investigations.</div></div>

A coupled discontinuous Galerkin-Finite Volume framework for solving gas dynamics over embedded geometries

We present a computational framework for solving the equations of inviscid gas dynamics using structured grids with embedded geometries. The novelty of the proposed approach is the use of high-order discontinuous Galerkin (dG) schemes and a shock-capturing Finite Volume (FV) scheme coupled via an hp adaptive mesh refinement (hp-AMR) strategy that offers high-order accurate resolution of the embedded geometries. The hp-AMR strategy is based on a multi-level block-structured domain partition in which each level is represented by block-structured Cartesian grids and the embedded geometry is represented implicitly by a level set function. The intersection of the embedded geometry with the grids produces the implicitly-defined mesh that consists of a collection of regular rectangular cells plus a relatively small number of irregular curved elements in the vicinity of the embedded boundaries. High-order quadrature rules for implicitly-defined domains enable high-order accuracy resolution of the curved elements with a cell-merging strategy to address the small-cell problem. The hp-AMR algorithm treats the system with a second-order finite volume scheme at the finest level to dynamically track the evolution of solution discontinuities while using dG schemes at coarser levels to provide high-order accuracy in smooth regions of the flow. On the dG levels, the methodology supports different orders of basis functions on different levels. The space-discretized governing equations are then advanced explicitly in time using high-order Runge-Kutta algorithms. Numerical tests are presented for two-dimensional and three-dimensional problems involving an ideal gas. The results are compared with both analytical solutions and experimental observations and demonstrate that the framework provides high-order accuracy for smooth flows and accurately captures solution discontinuities.

Large-Eddy Simulation of a Hydrocyclone with an Air Core Using Two-Fluid and Volume-of-Fluid Models

Large-eddy simulations have been conducted for two-phase flow (water and air) in a hydrocyclone using Two-Fluid (Euler–Euler) and Volume-of-Fluid (VOF) models. Subgrid stresses are modeled using a dynamic eddy–viscosity model, and results are compared to those using the Smagorinsky model. The effects of grid resolutions on the mean flow and turbulence statistics have been thoroughly investigated. Five block-structured grids of 0.72, 1.47, 2.4, 3.81, and 7.38 million elements have been used for the simulations of Hsieh’s 75 mm hydrocyclone Mean velocity profiles and normal Reynolds stresses have been compared with experimental data. Results of the two-fluid model are in good agreement with those of the VOF model. A fine mesh in the axial and radial directions is necessary for capturing the turbulent vortical structure. Turbulence structures in the hydrocyclone are dominated by helical vortices around the air core. Energy spectra are analyzed at different points in the hydrocyclone, and regions of low turbulent kinetic energy are identified and attributed to stabilizing effects of the swirling velocity component.

Удосконалення характеристик кільцевого вхідного пристрою авіаційної силової установки з гвинтовентилятором

The article considers the method of improving the characteristics of the ring inlet device, taking into account the influence of the propeller of an aircraft power plant with a turboprop engine. It is shown that increasing the total pressure loss in the inlet device by 5% increases, approximately, the specific fuel consumption by 3% and reduces engine thrust by 6%, and uneven flow at the inlet to the engine is the cause of unstable compressor of the turboprop engine. It is proposed to improve the characteristics of the input device by modifying the shape of its shell and channel. Evaluation of the influence of the shape of the shell and the channel of the annular axial VP on its main aerodynamic characteristics, taking into account the non-uniformity of the flow on the fan in the calculated mode of operation of the SU is carried out by calculating the full pressure recovery factor. The object of the study is an annular axial input device in front of which is a coaxial fan turboprop fan. The process of modeling the influence of the shape of the shell and the channel on the recovery factor of total pressure, circular and radial non-uniformity of the flow through the input device is implemented in the software system of finite element analysis ANSYS CFX. Geometric models of coaxial screw fan, fairing and inlet device are built in ANSYS SpaceClaim and transferred using the built-in import function in ANSYS Workbench. Block-structured grid models of air propellers of the first and second rows of the fan in the amount of 1.9 million, fairing and inlet device, in the amount of 3.9 million, are built in the ANSYS TurboGrid environment. The standard Stern (Shear Stress Transport) Gamma Theta Transition was used to close the Navier-Stokes equation system. Based on the results of mathematical modeling of flow in coaxial fans and subsonic ring inlet device on the maximum cruising mode of the turboprop engine, the full pressure recovery factor is calculated and it is established that the most influential factor that increases its full pressure recovery factor.

Numerical analysis of high-lift configurations with oscillating flaps

This study presents two-dimensional aerodynamic investigations of various high-lift configuration settings concerning the deflection angles of droop nose, spoiler and flap in the context of enhancing the high-lift performance by dynamic flap movement. The investigations highlight the impact of a periodically oscillating trailing edge flap on lift, drag and flow separation of the high-lift configuration by numerical simulations. The computations are conducted with regard to the variation of the parameters reduced frequency and the position of the rotational axis. The numerical flow simulations are conducted on a block-structured grid using Reynolds Averaged Navier Stokes simulations employing the shear stress transport k-omega turbulence model. The feature Dynamic Mesh Motion implements the motion of the oscillating flap. Regarding low-speed wind tunnel testing for a Reynolds number of 0.5 times 10^{6} the flap movement around a dropped hinge point, which is located outside the flap, offers benefits with regard to additional lift and delayed flow separation at the flap compared to a flap movement around a hinge point, which is located at 15 % of the flap chord length. Flow separation can be suppressed beyond the maximum static flap deflection angle. By means of an oscillating flap around the dropped hinge point, it is possible to reattach a separated flow at the flap and to keep it attached further on. For a Reynolds number of 20 times 10^6, reflecting full scale flight conditions, additional lift is generated for both rotational axis positions.

Simulating penetration problems in incompressible materials using the material point method

This paper presents a methodology for computing the response of a rigid strip footing in incompressible Tresca soil when loaded to large settlements. The numerical simulations were performed using the generalized interpolation material point method (GIMP). For efficiency, a block-structured rectilinear irregular grid was used that moves and compresses as the footing advances. Volumetric locking was prevented using the non-linear extension to the B-bar method. The paper addresses the modifications required for the implementation of the B-bar method in GIMP and demonstrates its efficacy in mitigating locking using two benchmark problems: the Cook’s membrane problem and the limit bearing capacity of a footing in a Tresca soil at small settlements. The response of the footing when loaded to large settlements under both quasi-static and dynamic loads is then presented. A comparison of the solutions obtained using the proposed methodology with other numerical solutions available in the literature illustrates the efficacy of the proposed method.

A local correlation-based zero-equation transition model

In this work, the local correlation-based one-equation transition model (Menter, F.R., Smirnov, P.E., Liu, T. and Avancha, R., A one-equation local correlation-based transition model. Flow, Turbulence and Combustion, vol. 95, no. 4, pp. 583–619, 2015.) is transformed into a zero-equation transition model. The new model provides an attractive choice in terms of quick implementation of a transition model in existing turbulent flow solvers with Menter’s shear-stress transport (SST) turbulence model, as it only introduces three extra source terms in the transport equation of turbulent kinetic energy. The model is validated against a set of benchmark flat-plate test cases: T3 series and SK, and also subsonic flows past two different airfoils: Aerospatiale A-airfoil (Re=2.1 million) and E387 (Re=0.2 million), and finally applied to a transonic flow over 3D DLR-F5 wing (Re=1.5 million). Results show that the proposed model produces similar transition prediction as the one-equation transition model, with a reduced computational effort. The computations are performed with an in-house finite-volume solver for compressible turbulent flows on block-structured grids.

An adaptive multigrid on block-structured grids

An implementation of the multigrid method on conformal block-structured grids is proposed. The algorithm is intended to solve a boundary-value problem for elliptic PDEs. Each block is discretized using a structured hexahedron grid.The set of multilevel grids are arranged in hierarchical levels. In each block the discrete finite volume scheme is constructed on the fine level grid. The coarse level equations are formed by the Galerkin procedure. The presence of the irregular block connections leads to irregular stencils in the nodes which adjoin of several blocks. Instead of a special irregular node treatment we propose to find the solution in these nodes in additive manner. Such an opportunity is provided by the use of the explicit iterative procedures for both the smoothers and the coarsest grid solver. We also present an adaptive technique which adjusts these procedures for achieving the prescribed multigrid convergence rate. The proposed approach enhances the potential of the multigrid method in ultra-parallel computing.

Multi-species collisions for delta-f gyrokinetic simulations: Implementation and verification with GENE

A multi-species linearized collision operator based on the model developed by Sugama et al. has been implemented in the nonlinear gyrokinetic code, GENE. Such a model conserves particles, momentum, and energy to machine precision, and is shown to have negative definite free energy dissipation characteristics, satisfying Boltzmann’s H-theorem, including for realistic mass ratio. Finite Larmor Radius (FLR) effects have also been implemented into the local version of the code. For the global version of the code, the collision operator has been developed to allow for block-structured velocity space grids, allowing for computationally tractable collisional global simulations. The validity of the collision operator has been demonstrated by relaxation and conservation tests, as well as appropriate benchmarks. The newly implemented operator shall be used in future simulations to study magnetically confined fusion plasma turbulence and transport in more extreme regions with higher collisionality.

WaLBerla: A block-structured high-performance framework for multiphysics simulations

Programming current supercomputers efficiently is a challenging task. Multiple levels of parallelism on the core, on the compute node, and between nodes need to be exploited to make full use of the system. Heterogeneous hardware architectures with accelerators further complicate the development process. waLBerla addresses these challenges by providing the user with highly efficient building blocks for developing simulations on block-structured grids. The block-structured domain partitioning is flexible enough to handle complex geometries, while the structured grid within each block allows for highly efficient implementations of stencil-based algorithms. We present several example applications realized with waLBerla, ranging from lattice Boltzmann methods to rigid particle simulations. Most importantly, these methods can be coupled together, enabling multiphysics simulations. The framework uses meta-programming techniques to generate highly efficient code for CPUs and GPUs from a symbolic method formulation.

Modelling binary alloy solidification with adaptive mesh refinement

The solidification of a binary alloy results in the formation of a porous mushy layer, within which spontaneous localisation of fluid flow can lead to the emergence of features over a range of spatial scales. We describe a finite volume method for simulating binary alloy solidification in two dimensions with local mesh refinement in space and time. The coupled heat, solute, and mass transport is described using an enthalpy method with flow described by a Darcy-Brinkman equation for flow across porous and liquid regions. The resulting equations are solved on a hierarchy of block-structured adaptive grids. A projection method is used to compute the fluid velocity, whilst the viscous and nonlinear diffusive terms are calculated using a semi-implicit scheme. A series of synchronization steps ensure that the scheme is flux-conservative and correct for errors that arise at the boundaries between different levels of refinement. We also develop a corresponding method using Darcy's law for flow in a porous medium/narrow Hele-Shaw cell. We demonstrate the accuracy and efficiency of our method using established benchmarks for solidification without flow and convection in a fixed porous medium, along with convergence tests for the fully coupled code. Finally, we demonstrate the ability of our method to simulate transient mushy layer growth with narrow liquid channels which evolve over time.

An embedded boundary approach for efficient simulations of viscoplastic fluids in three dimensions

We present a methodology for simulating three-dimensional flow of incompressible viscoplastic fluids modeled by generalized Newtonian rheological equations. It is implemented in a highly efficient framework for massively parallelizable computations on block-structured grids. In this context, geometric features are handled by the embedded boundary approach, which requires specialized treatment only in cells intersecting or adjacent to the boundary. This constitutes the first published implementation of an embedded boundary algorithm for simulating flow of viscoplastic fluids on structured grids. The underlying algorithm employs a two-stage Runge-Kutta method for temporal discretization, in which viscous terms are treated semi-implicitly and projection methods are utilized to enforce the incompressibility constraint. We augment the embedded boundary algorithm to deal with the variable apparent viscosity of the fluids. Since the viscosity depends strongly on the strain rate tensor, special care has been taken to approximate the components of the velocity gradients robustly near boundary cells, both for viscous wall fluxes in cut cells and for updates of apparent viscosity in cells adjacent to them. After performing convergence analysis and validating the code against standard test cases, we present the first ever fully three-dimensional simulations of creeping flow of Bingham plastics around translating objects. Our results shed new light on the flow fields around these objects.

RANS/ILES Method Optimization for Effective Calculations on Supercomuter

The article discusses the use of the RANS/ILES method for modeling gas-dynamic processes in a combustion chamber, described using an axisymmetric block-structured computational grid. For these calculations, the approaches of the linked border conditions and the fragmentation of the computational grid are used, which allows to effectively calculate this problem on a supercomputer, achieving acceleration by more than two orders of magnitude, using a total of 32 computing nodes.

MHD Stability and Energy Principle for Two-Dimensional Equilibria without Assumption of Nested Magnetic Surfaces

Abandoning the assumption of nested magnetic surfaces in tokamak plasma expands the field of research and opens up new approaches for both theoretical and experimental plasma physics. The computer code KINX for calculations of the ideal MHD stability was developed for studies of doublet plasmas with two magnetic axes and using block-structured grids in each subdomain with nested magnetic surfaces. Then, the MHD_NX code on unstructured grids was developed to calculate the stability of two-dimensional equilibria with an arbitrary topology of magnetic surfaces. The study of equilibrium and stability of equilibrium configurations with toroidal current density reversal and axisymmetric n = 0 islands, which are associated with internal transport barrier and low current density at the magnetic axis, as well as with the operation of tokamaks in the alternating current regime, leads to more general issues of MHD stability of two-dimensional solutions of the Grad−Shafranov equations with islands under other types of symmetry—chain of islands in helical symmetry and cylindrically symmetric m = 0 islands in configurations with the longitudinal field reversal. New ideal MHD unstable modes have been discovered for various types of two-dimensional island configurations. The energy principle with MHD-compatible boundary conditions at open magnetic field lines is necessary for the self-consistent stability analysis of divertor configurations in tokamaks with a finite current density at the separatrix, taking into account the plasma outside the separatrix. Several codes have been developed for calculations of plasma equilibrium and stability, taking into account the influence of currents outside the separatrix, which are ready for integration with other codes for edge plasma modeling.