Parallel Simulation Research Articles

Preventing Workload Interference with Intelligent Routing and Flexible Job Placement Strategy on Dragonfly System

Dragonfly is an indispensable interconnect topology for exascale HPC systems. To link tens of thousands of compute nodes at a reasonable cost, Dragonfly shares network resources with the entire system such that network bandwidth is not exclusive to any single application. Since HPC systems are usually shared among multiple co-running applications at the same time, network competition between co-existing workloads is inevitable. This network contention manifests as workload interference, where a job’s network communication can be severely delayed by other jobs. This study presents a comprehensive examination of leveraging intelligent routing and flexible job placement to mitigate workload interference on Dragonfly systems. Specifically, we leverage the parallel discrete event simulation toolkit, SST, to investigate workload interference on Dragonfly with three contributions. We first present Q-adaptive routing, a multi-agent reinforcement learning routing scheme, and a flexible job placement strategy that, together, can mitigate workload interference based on workload communication characteristics. Next, we enhance SST with Q-adaptive routing and develop an automatic module that serves as the bridge between SST and HPC job scheduler for automatic simulation configuration and automated simulation launching. Finally, we extensively examine the workload interference under various job placement and routing configurations.

Parallel numerical simulation of the 2D acoustic wave equation

Parallel numerical simulation of the 2D acoustic wave equation

Development of the parallel and distributed simulation field

This paper documents the origins and development of the parallel and distributed simulation field, especially as it relates to parallel discrete event simulation (PDES) technology. My focus is on the people and ideas that contributed to the creation and evolution of PDES technology over the past five decades. Much of this paper reviews the research conducted in academia and research laboratories that were the driving forces behind the development of PDES technology as well as subsequent efforts in government and industry to bring the technology to modeling and simulation practitioners. The origins of PDES technology lie in a fundamental technical issue called the synchronization problem. This problem is discussed, as well as the two main solution approaches that were created. The development of these approaches and their evolution to form the time management services of the High Level Architecture (HLA) standard are described, as well as the community of researchers and practitioners that formed in establishing this field. Commercialization efforts are discussed followed by a brief discussion of areas requiring further research. The importance of the Society for Modeling and Simulation International (SCS) and its flagship journal, Simulation, in the development of the field are discussed. Throughout this discussion, I provide my personal observations and take away lessons concerning the development of the field.

Fully implicit bound-preserving discontinuous Galerkin algorithm with unstructured block preconditioners for multiphase flows in porous media

Massively parallel simulation of multiphase flows in porous media is challenging due to the highly nonlinear governing equations with the complexity of various modeling features and the significant heterogeneity of material coefficients. In this paper, we introduce a fully implicit discontinuous Galerkin (DG) reservoir simulator designed for parallel computers to handle large-scale multiphase flow simulations. Our approach formulates a fully implicit DG scheme on 3D unstructured grids to discretize the nonlinear governing equations, which take into account capillary effects, permeability heterogeneity, gravity, and injection/production wells. A vertex-based slope limiter within the fully implicit DG framework is adopted to suppress oscillations near discontinuities and ensure the boundedness of the solution. We address the resulting nonlinear systems from the fully implicit discretization using a family of inexact Newton–Krylov algorithms with an analytic Jacobian. Moreover, we employ an unstructured block-type preconditioner with an efficient overlapping additive Schwarz approximation to accelerate the convergence of iterations and enhance the scalability of the simulator. Large-scale simulations of both benchmark and realistic reservoir problems on 3D unstructured grids are conducted to demonstrate the efficiency and scalability of the proposed reservoir simulator.

Improving relative humidity measurements on Mars: new laboratory calibration measurements

Abstract. In this paper we present new calibration measurements that have been performed with the ground reference models of the relative humidity instruments of the Mars Science Laboratory (MSL), Mars 2020 and ExoMars missions. All instruments are based on capacitive sensor head technology, and they are developed, manufactured and tested by the Finnish Meteorological Institute (FMI). Calibration of capacitive humidity sensors for the Martian environment has been a challenging task and special facilities are needed in order to create Martian conditions including all relevant environmental parameters that can be accurately controlled and measured: low pressure, low temperature, carbon dioxide environment and especially humidity. A measurement campaign was performed at the German Aerospace Center (DLR) PASLAB (Planetary Analog Simulation Laboratory) to determine relative humidity calibration datasets for REMS-H, MEDA HS and METEO-H instruments in temperatures from −30 °C down to −70 °C in low-pressure CO2. In addition to the stable point humidity calibration measurements in CO2, the instrument performance was tested with the actual Martian atmosphere composition and during long continuous measurements. The new calibration dataset has already been used in the flight calibration of the MEDA HS instrument, resulting in successful calibration and excellent accuracy. The results from this campaign will further improve relative humidity measurements on Mars by providing the means to reanalyze the current calibration of the REMS-H flight model and by allowing more accurate comparison between the two instruments currently on the Martian surface.

Atomistic simulation of primary microstructure formation in metals during crystallization from the melt

Additive manufacturing of metallic parts by Selective Laser Melting (SLM) implies high temperature gradients and small volume of the melt bath. These conditions make the process scales close to those available for state-of-the-art massively parallel atomistic simulations. In the paper, the microscopic mechanisms responsible for the formation of primary microstructure during molten metal solidification are investigated using classical molecular dynamics (CMD). The 316L austenitic stainless steel with face centred cubic lattice, which is widely used in industry including SLM applications was chosen as a material for the CMD simulations. It was shown that solidified material inherits substrate defects and catches new ones, which interact with the solidification front thus producing the primary microstructure. Peculiarities of solidification in different crystallographic directions and solidification front interaction with grain boundaries and newly produced defects (mostly twin boundaries) as well as their formation are under study. Resulting microstructures of virtual samples are compared with those of real samples produced by SLM and analysed by the electron backscatter diffraction (EBSD) method. The comparison shows similarities of EBSD and CMD sample patterns and evidences for the capability of the large-scale atomistic simulations to reproduce main features of the microstructures formed in the metallic SLM additive production.

Multistable dynamics and chaos in a system consisting of an inertial neuron coupled to a van der Pol oscillator

Abstract We consider a dynamical system consisting of a van der Pol oscillator linearly coupled to an inertial neuron with two wells potential. Analytical studies are conducted focusing on the energy computation, the dissipation and symmetry, as well as the determination and characterization of the equilibrium points. We define the parameter ranges related to different types of oscillations in the coupled system in order to have an overall idea of the nature of the attractors (hidden or self-excited) that may exist. We apply numerical analysis techniques (2-parameter diagrams, bifurcation analysis, phase portraits, basins of attractions, etc) in accordance with the previous operating range in order to shed light on the plethora of competing dynamics of the model and possible forms of strange attractors as well. Another salient point of this work is the coexistence between five self-excited attractors (limit cycle and chaos) with a hidden attractor (limit cycle). We also examine the impact of symmetry breaking on the system response. An appropriate analog simulator of the coupled system is designed and simulated in PSpice in order to check the results reported during the theoretical analyses. We believe that the results of the present work complement and enrich previously published ones concerning the dynamics of a system composed of a van der pol oscillator coupled to a (non-oscillating) double-well oscillator.

Experimental Study on Mechanical Properties of Artificial Cores Saturated With Ice: An Analogical Simulation to Natural Gas Hydrate Bearing Sediments

ABSTRACTThis work takes the natural gas hydrate (NGH) reservoir in the Shenhu area at the South China Sea as the target. Using the quartz sand, calcite grains, and illite powder as the main mineral composites, and Portland cement as the bonding material, the host skeleton with similar porosity and mechanical properties to the target reservoir were prepared. Ice was used to simulate natural gas hydrates for their high similarity in mechanical properties and distributions within porous host. The ice saturation in the host skeleton is quantitatively controlled by the quality method. As an analogical simulation, artificial samples with different ice saturations were tested by uniaxial compression measurements and the Brazilian tensile tests, aiming to reveal the mechanical behavior of NGH sediments. The results indicated that the plasticity of the artificial sample increases and its damage form transitions from brittleness to ductility as the ice saturation increases. The effects of free water and ice on the strength of saturated samples are quite different. The free water tends to reduce the strength of the sample due to illite hydration and change of internal friction. The bonding effect of ice tends to increase the strength of the sample, while the ice could also reduce the internal friction. When the ice saturation is larger than 30%, the compressive and tensile strengths of the sample increase with ice saturation, which could be regressed as yc = 2.2628S + 0.8322, and yt = 0.411S + 0.0273, respectively. The constitutive model was developed based on the equivalent medium theory and the D‐P criterion, which could describe the experiment data well with deviations less than 10%.

Predicting arbitrary state properties from single Hamiltonian quench dynamics

Analog quantum simulation is an essential routine for quantum computing and plays a crucial role in studying quantum many-body physics. Typically, the quantum evolution of an analog simulator is largely determined by its physical characteristics, lacking the precise control or versatility of quantum gates. This limitation poses challenges in extracting physical properties on analog quantum simulators, an essential step of quantum simulations. To address this issue, we introduce the protocol, which uses a single quench Hamiltonian for estimating arbitrary state properties, eliminating the need for ancillary systems and random unitaries. Additionally, we derive the sample complexity of this protocol and show that it performs comparably to the classical shadow protocol. The Hamiltonian shadow protocol does not require sophisticated control and can be applied to a wide range of analog quantum simulators. We demonstrate its utility through numerical demonstrations with Rydberg atom arrays under realistic parameter settings. The new protocol significantly broadens the application of randomized measurements for analog quantum simulators without precise control and ancillary systems. Published by the American Physical Society 2024

Exploring the spatial effects influencing the EGFR/ERK pathway dynamics with machine learning surrogate models

The fate of cells is regulated by biochemical reactions taking place inside of them, known as intracellular pathways. Cells display a variety of characteristics related to their shape, structure and contained fluid, which influences the diffusion of proteins and their interactions. To gain insights into the spatial effects shaping intracellular regulation, we apply machine learning (ML) to explore a previously developed spatial model of the epidermal growth factor receptor (EGFR) signaling. The model describes the reactions between molecular species inside of cells following the transient activation of EGF receptors. To train our ML models, we conduct 10,000 numerical simulations in parallel where we calculate the cumulative activation of molecules and transcription factors under various conditions such as different diffusion speeds, inactivation rates, and cell structures. We take advantage of the low computational cost of ML algorithms to investigate the effects of cell and nucleus sizes, the diffusion speed of proteins, and the inactivation rate of the Ras molecules on the activation strength of transcription factors. Our results suggest that the predictions by both neural networks and random forests yielded minimal mean square error (MSEs), while linear generalized models displayed a significant larger MSE. The exploration of the surrogate models has shown that smaller cell and nucleus radii as well, larger diffusion coefficients, and reduced inactivation rates increase the activation of transcription factors. These results are confirmed by numerical simulations. Our ML algorithms can be readily incorporated within multiscale models of tumor growth to embed the spatial effects regulating intracellular pathways, enabling the use of complex cell models within multiscale models while reducing the computational cost.

New insights into impact-induced removal of the deposited droplet

This paper presents a comprehensive investigation into the collision dynamics of equal and unequal-sized nanodroplets on a flat surface using molecular dynamics simulations, revealing new insights into scaling laws and energy dissipation mechanisms. The simulations, conducted with the Large-Scale Atomic/Molecular Massively Parallel Simulator software, involved an initially stationary droplet on the surface and a suspended droplet with varying diameter ratios (λ) and impact velocities. The results show that at low Weber numbers (We < 24.15), the droplets tend to deposit after impact, while at higher Weber numbers (We ≥ 24.15), they undergo spreading and retraction, ultimately rebounding. The study reveals that the dimensionless contact time (t*) and maximum spreading factor (βmax*) in collisions between droplets of different sizes do not follow the same scaling relationship observed in single nanodroplet impacts. By redefining the Weber and Reynolds numbers (Re), the new scaling relationships t* ∼ We2/3Re−1/3λ−1/3 and βmax* ∼ We2/3Re−1/3λ−1/3 are proposed and validated. This work represents a further in-depth study of previous research on single nanodroplet impact, introducing for the first time the diameter ratio in unequal droplet impacts into the variation patterns of contact time and maximum spreading diameter. Moreover, these findings highlight the importance of revisiting and potentially revising classical theories to accommodate the unique physical phenomena that emerge at smaller scales.

Molecular Dynamics Simulation of High-Energy Beam Irradiation of SiO2 Single Crystal with Three Different Morphologies: No Free Surface, with a Free Surface, and Thin Films

Abstract High-energy beam irradiation of SiO2 crystal with the α-quartz structure was simulated by the molecular dynamics method. Three types of specimen structures were examined: a single crystal without a free surface, a single crystal with a free surface, and a thin film. After structural relaxation at room temperature, a cylindrical region with diameter of 3.0 nm was set at the center of the specimen and high-thermal energy of Seff = 0.1 – 2.5 keV/nm was added to that region, where Seff is the effective stopping power. Atomic motions were calculated by the molecular dynamics method using the large-scale atomic/molecular massively parallel simulator). Vashishta potential was used for the atomic interaction. The single crystal structure of α-quartz without the free surface was stable up to 1.0 keV/nm and it gradually became amorphous with increasing thermal energy. In contrast, the single crystal structure with the free surface was stable up to 0.5 keV/nm and was amorphous at higher thermal energy. In particular, the atomic structures for the thermal impact Seff ≥ 2.0 keV/nm had a facet-like structure at the impacted surface, which corresponds to actual experimental results. Nano-hole formation was observed in the irradiation process of the film structure.

Reproducing hydrodynamics and elastic objects: A hybrid mesh-free model framework for dynamic multi-phase flows

Reproducing Hydrodynamics and Elastic Objects (RHEO) is a new meshfree modeling framework for simulating complex multi-phase flows of fluids and solids. RHEO is implemented within the open-source Large-Scale Atomic/Molecular Massively Parallel Simulator particle dynamics code and couples a reproducing kernel smoothed particle hydrodynamics scheme for modeling fluid flow and heat transfer with a bonded particle model for modeling breakable elastic bodies. The resulting scheme provides a robust framework for simulating multi-phase material systems with complex and evolving boundaries and interfaces. RHEO collects many advanced mesh-free modeling features into a centralized, modular, and easily extensible package, including heat transport, reversible melting and solidification, particle distribution regularization, and user adjustable kernel accuracy. We report comprehensive tests of RHEO's accuracy and convergence for common benchmark flows of bulk fluids, boundary driven flows, and complex fluid/solid systems. A series of multi-phase systems are highlighted, including a bouncing water balloon, the fracture and flow of a brittle egg, melting of a free-standing solid beam, and conductive cooling and solidification of an extruded polymer during 3D printing.

Parallel Simulations of the Sharp Wave-Ripples of the Hippocampus on Multicore CPUs and GPUs

The simulation of realistic systems plays a crucial role in modern sciences. Complex organs such as the brain can be described by mathematical models to reproduce biological behaviors. In the brain, the hippocampus is a critical region for memory and learning. In the literature, a model to reproduce the memory consolidation mechanism has been proposed. This model exhibits a high degree of biological realism, though it is accompanied by a significant increase in computational complexity. This paper proposes the development of parallel simulation targeting different devices, namely multicore CPUs and GPUs. The experiments highlighted that the biological realism is maintained, together with a significant decrease of the processing times. Finally, the conducted analysis highlights that the GPU is one of the most suitable technologies for this kind of simulation.

Investigating the role of temperature and moisture on the degradation of 3D-printed polymethyl methacrylate dental materials through molecular dynamics simulations

This study aimed to comprehensively investigate the degradation behavior of 3D printed polymethyl methacrylate (PMMA) dental materials, with a specific focus on the influential factors of temperature and moisture, by employing molecular dynamics simulations. Owing to their aesthetic properties, 3D-printed PMMA dental materials play a pivotal role in dental applications. However, understanding their degradation mechanisms, particularly in the context of temperature and moisture variations, is crucial for their long-term durability. A Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) was utilized for the molecular dynamics simulations. The simulation setup included temperature variations from 300 to 600 K and relative humidity (RH) levels ranging from 20 to 100%. Various mechanical properties and structural changes were analyzed to determine the degradation behavior. Energetic profiling during equilibration and the subsequent temperature variations were studied. The spatial distribution of the mean squared displacement, non-bond energy, Young’s modulus, bending stress, and volume expansion coefficient of the particles were quantitatively analyzed, revealing temperature- and moisture-dependent trends. The study concluded that temperature and moisture significantly affected the degradation behavior of 3D-printed PMMA dental materials. Higher temperatures and increased humidity levels contribute to reduced mechanical strength and altered structural properties, emphasizing the importance of controlling environmental conditions during fabrication.

Integrating machine learning interatomic potentials with hybrid reverse Monte Carlo structure refinements in RMCProfile.

Structure refinement with reverse Monte Carlo (RMC) is a powerful tool for interpreting experimental diffraction data. To ensure that the under-constrained RMC algorithm yields reasonable results, the hybrid RMC approach applies interatomic potentials to obtain solutions that are both physically sensible and in agreement with experiment. To expand the range of materials that can be studied with hybrid RMC, we have implemented a new interatomic potential constraint in RMCProfile that grants flexibility to apply potentials supported by the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) molecular dynamics code. This includes machine learning interatomic potentials, which provide a pathway to applying hybrid RMC to materials without currently available interatomic potentials. To this end, we present a methodology to use RMC to train machine learning interatomic potentials for hybrid RMC applications.

A parallel compositional reservoir simulator for large-scale CO2 geological storage modeling and assessment

Storing CO2 in deep aquifers and depleted gas reservoirs is an effective way to achieve carbon neutrality. However, the numerical simulation of CO2 storage in these formations is challenging due to the complexity of gases-brine systems. The number of gas species included in the gases-brine fluid models of existing simulators cannot meet the rapidly evolving CO2 sequestration scenarios. To address this intricate issue, we developed a three-dimensional fully implicit parallel CO2 geological storage simulator (PRSI-CGCS) on distributed-memory computers based on our in-house parallel platform. This simulator uses a compositional fluid model with a diverse range of gas species, including CO2, C1 ~ C3, N2, H2S, as well as newly added gases H2 and O2, which may be encountered in geological CO2 storages. Besides, we provide more suitable scaling factors for different gases in the stability analysis bypassing (SAB) method to accelerate the gases-brine phase equilibrium calculations. PRSI-CGCS does not incorporate energy conservation equations, and salt precipitation or dissolution is also not considered. Numerical experiments show that our simulator is scalable, robust and validated to simulate large-scale CO2 storage problems with hundreds of millions of grid blocks on a parallel supercomputer cluster. Besides, after our modification on scaling factors, the SAB method can reduce the number of stability analyses by 61.39 % to 88.71 %, thereby reducing simulation time. Furthermore, case studies indicate that injecting O2 and H2 along with CO2 reduces the stability or capacity of CO2 storage and increases the pressure required for injection. However, this impact is not significant when the impurity content is less than 10 %.

Driving Control Strategy and Specification Optimization for All-Wheel-Drive Electric Vehicle System with a Two-Speed Transmission

Equipping electric vehicles with a two-speed gearbox allows for achieving high torque and maximum speed through appropriate gear ratio adjustments. Additionally, tuning motor operating points to efficient zones, considering energy efficiency, significantly enhances the vehicle’s overall performance. This paper presents an AWD system configuration method, integrating a two-speed transmission to improve energy efficiency and driving performance through front and rear motor torque distribution and powertrain specification optimization. Based on vehicle simulations conducted using MATLAB/Simulink, a strategy for torque distribution between the front/rear axles was established using fuzzy logic, considering energy efficiency and driving stability. Furthermore, a multi-objective optimization was performed using a surrogate model trained through MATLAB parallel simulations. When the optimization results were applied to various vehicle specifications, it was observed that energy efficiency was improved, and acceleration performance was increased compared to a baseline vehicle without optimization.

Overall plan optimization of BWB UAV based on comprehensive evaluation

Based on the shape and performance characteristics of the wing-body fusion layout UAV, a multidisciplinary comprehensive analysis model, comprehensive evaluation model and optimization model suitable for the overall scheme of the UAV in the conceptual design stage are established, and the corresponding tools are developed. The geometric, weight, aerodynamic and stealth characteristics of the overall scheme of the wing-body fusion layout UAV are analyzed by combining numerical analysis and engineering methods. The improved TOPSIS method is used to establish a comprehensive evaluation model from the perspective of the loading capacity, economy and adaptability of the UAV. The rationality of the analysis model is verified by taking multiple UAV schemes as examples. Using the parallel subset simulation optimization algorithm, the traditional multi-objective optimization is completed with the maximum cruise lift-to-drag ratio and the minimum forward RCS as the goal, and the scheme optimization based on comprehensive evaluation is completed with the strongest competitiveness as the goal. Based on all the schemes generated by the multi-objective optimization process, the parameter correlation analysis is completed, and the influence of the span, sweep angle and twist angle on the aerodynamic and stealth characteristics is verified. The RCS of the final multi-objective optimization scheme is reduced 41.6%, and the cruise lift-to-drag ratio is improved 3.5% ; the competitiveness score of the comprehensive evaluation optimization scheme is improved 44.0%.

Signal-to-noise trade-offs between magnet diameter and shield-to-coil distance for cylindrical Halbach-based portable MRI systems for neuroimaging.

To investigate the trade-off between magnet bore diameter and the distance between the conductive Faraday shield and RF head coil for low-field point-of-care neuroimaging systems. Electromagnetic simulations were performed for three different Faraday shield geometries and two commonly used RF coil designs (spiral and solenoid) to assess the effects of a close-fitting shield on the RF coil's transmit and receive efficiencies. Experimental measurements were performed to confirm the accuracy of the simulations. Parallel simulations were performed to assess the static magnet ( ) field as a function of the magnet bore diameter. The obtainable SNR was then calculated as a function of these two related variables. Simulations of the RF coil characteristics and transmit efficiencies agreed well with corresponding experimentally determined parameters. Overall, the RF coil transmit efficiency was, as expected, higher when the gap between the shield and coil increased. The calculated intrinsic SNR showed that maximum SNR would be obtained for a cylindrical shield of diameter 310mm with an inner diameter of the magnet of 320mm (assuming 10mm for the gradient coils). This work presents an overview of the trade-offs in transmit efficiencies for RF coils used for POC MRI neuroimaging as a function of coil-to-shield distance and inner diameter of the Halbach magnet. Results show that there is a relatively shallow optimum between a magnet diameter of 290 and 330mm, with values falling more than 10% if either smaller or larger magnets are used.