Parallel Simulation Research Articles

Acceleration mechanisms of energetic ion debris in laser-driven tin plasma EUV sources

Laser-driven tin plasmas are driving new-generation nanolithography as sources of extreme ultraviolet (EUV) radiation centered at 13.5 nm. A major challenge facing industrial EUV source development is predicting energetic ion debris produced during the plasma expansion that may damage the sensitive EUV channeling multilayer optics. Gaining a detailed understanding of the plasma dynamics and ion acceleration mechanisms in these sources could provide critical insights for designing debris mitigation strategies in future high-power EUV sources. We develop a fully kinetic model of tin-EUV sources using one-dimensional particle-in-cell simulations to study ion debris acceleration, which will be valuable for cross-validation of radiation-hydrodynamic simulations. An inverse-bremsstrahlung heating operator is used to model the interaction of a tin target with an Nd:YAG laser, and thermal conduction is included through a Monte Carlo Coulomb collision operator. While the large-scale evolution is in reasonable agreement with analogous hydrodynamic simulations, the significant timescale for collisional equilibration between electrons and ions allows for the development of prominent two-temperature features. A collimated flow of energetic ions is produced with a spectrum that is significantly enhanced at high energies compared to fluid simulations. The dominant acceleration mechanism is found to be a large-scale electric field supported mainly by the electron pressure gradient, which is enhanced in the kinetic simulations due to the increased electron temperature. We discuss the implications of these results for future modeling of tin-EUV sources and the development of debris mitigation schemes.

Transferring predictions of formation energy across lattices of increasing size*

In this study, we show the transferability of graph convolutional neural network (GCNN) predictions of the formation energy of the nickel-platinum solid solution alloy across atomic structures of increasing sizes. The original dataset was generated with the large-scale atomic/molecular massively parallel simulator using the second nearest-neighbor modified embedded-atom method empirical interatomic potential. Geometry optimization was performed on the initially randomly generated face centered cubic crystal structures and the formation energy has been calculated at each step of the geometry optimization, with configurations spanning the whole compositional range. Using data from various steps of the geometry optimization, we first trained our open-source, scalable implementation of GCNN called HydraGNN on a lattice of 256 atoms, which accounts well for the short-range interactions. Using this data, we predicted the formation energy for lattices of 864 atoms and 2048 atoms, which resulted in lower-than-expected accuracy due to the long-range interactions present in these larger lattices. We accounted for the long-range interactions by including a small amount of training data representative for those two larger sizes, whereupon the predictions of HydraGNN scaled linearly with the size of the lattice. Therefore, our strategy ensured scalability while reducing significantly the computational cost of training on larger lattice sizes.

An Axiomatic Theory for Reversible Computation

Undoing computations of a concurrent system is beneficial in many situations, such as in reversible debugging of multi-threaded programs and in recovery from errors due to optimistic execution in parallel discrete event simulation. A number of approaches have been proposed for how to reverse formal models of concurrent computation, including process calculi such as CCS, languages like Erlang, and abstract models such as prime event structures and occurrence nets. However, it has not been settled as to what properties a reversible system should enjoy, nor how the various properties that have been suggested, such as the parabolic lemma and the causal-consistency property, are related. We contribute to a solution to these issues by using a generic labelled transition system equipped with a relation capturing whether transitions are independent to explore the implications between various reversibility properties. In particular, we show how all properties we consider are derivable from a set of axioms. Our intention is that when establishing properties of some formalism, it will be easier to verify the axioms rather than proving properties such as the parabolic lemma directly. We also introduce two new properties related to causal-consistent reversibility, namely causal liveness and causal safety, stating, respectively, that an action can be undone if (causal liveness) and only if (causal safety) it is independent from all of the following actions. These properties come in three flavours: defined in terms of independent transitions, independent events, or via an ordering on events. Both causal liveness and causal safety are derivable from our axioms.

Fabrication of MOF-rGO aerogels to enhance electromagnetic wave absorption by adjusting the morphology and structure of MOFs by electron transfer

Graphene and its derivatives have become the star materials in the field of electromagnetic wave (EMW) absorption, and have been widely studied because of their unique structure and excellent properties. However, the high electrical conductivity of graphene-based material leads to poor impedance matching performances, so it often needs to be annealed at high temperatures, which is a intricate procedure and cause the waste of energy. Herein, we develop a convenient strategy to synthesize metal organic framework-reduced graphene oxide (MOF-rGO) aerogels through facile hydrothermal and freeze drying procedure. Firstly, the excessive electrons on graphene can be transferred to MOF, in this way, no annealing is needed. Secondly, the morphology and structure of MOF can be regulated electronically to endow the material with rich heterojunction interfaces and enhance its interfacial polarization. Thirdly, the synthesized aerogels satisfy the requirements of thin thickness, low density and light weight. The fabricated Ni-MOF-rGO and FeNi-MOF-rGO aerogels exhibit remarkable EMW absorption performances with strong reflection loss (RL) and broad effective absorption band (EAB) at the thickness of 2.4 (−47.8 dB and 7.84 GHz) and 2.9 mm (−48.3 dB and 8.32 GHz) whereas the filling contents are just 5 %. Moreover, the attenuation capacity of the fabricated aerogels are verified through the radar-section simulation and the interfacial polarization loss are confirmed by the COMSOL analogue simulation. In a word, this work provides a new afflatus and inspiration for the design of graphene-based EMW absorption materials.

Enhancing piezoelectric performance of CNTs through B and N substitution under combined mechanical loads: insights from MD simulations

The piezoelectric properties of carbon nanotubes (CNTs) doped with boron (B) and nitrogen (N) were studied using the classical molecular dynamics (MD) simulation software package large scale atomic/molecular massively parallel simulator. The interactions among the nanotube atoms C, N, and B were calculated using the Tersoff potential. MD simulations were performed to observe the changes in the piezoelectric coefficient of the doped CNTs under loading conditions like tension, torsion, and a combination of both. We considered a wide range of chirality to determine the influence of structural variation on the piezoelectric effect. The study revealed that B-CNTs exhibit superior piezoelectric coefficients compared to N-CNTs, indicating the significant role of dopant type. Moreover, under tensile loading, zigzag-oriented B-CNTs showed higher piezoelectric coefficients with a maximum e33 = 0.2441 C m−2, whereas under torsional loading, armchair-oriented B-CNTs showed enhanced response with a maximum e36 = 0.0564 C m−2. A notable observation was that under combined loading conditions (tensile and torsional), the piezoelectric behavior of the B-CNTs was dependent on the nanotube’s chirality and did not yield a linear additive response. The polarization induced under combined loading in most of the doped CNTs is significantly higher than the sum of polarization generated under tensile and torsional loading conditions. This behavior suggests that the overall piezoelectric effect under combined loading can be enhanced, which emphasizes the need for an approach to optimize the mechanical loading condition. The results showcase the potential of B-/N-CNTs to be engineered for efficient performance by demonstrating that tailored mechanical loading can enhance the piezoelectric responses in doped CNTs, opening a pathway for highly functional and efficient nanoscale piezoelectric devices.

Parallel Simulation of Quantum Networks with Distributed Quantum State Management

Quantum network simulators offer the opportunity to cost-efficiently investigate potential avenues for building networks that scale with the number of users, communication distance, and application demands by simulating alternative hardware designs and control protocols. Several quantum network simulators have been recently developed with these goals in mind. As the size of the simulated networks increases, however, sequential execution becomes time-consuming. Parallel execution presents a suitable method for scalable simulations of large-scale quantum networks, but the unique attributes of quantum information create unexpected challenges. In this work, we identify requirements for parallel simulation of quantum networks and develop the first parallel discrete-event quantum network simulator by modifying the existing serial simulator SeQUeNCe. Our contributions include the design and development of a quantum state manager (QSM) that maintains shared quantum information distributed across multiple processes. We also optimize our parallel code by minimizing the overhead of the QSM and decreasing the amount of synchronization needed among processes. Using these techniques, we observe a speedup of 2 to 25 times when simulating a 1,024-node linear network topology using 2 to 128 processes. We also observe an efficiency greater than 0.5 for up to 32 processes in a linear network topology of the same size and with the same workload. We repeat this evaluation with a randomized workload on a caveman network. We also introduce several methods for partitioning networks by mapping them to different parallel simulation processes. We have released the parallel SeQUeNCe simulator as an open source tool alongside the existing sequential version.

High energy radiation tolerance of iron phosphate glasses: Molecular dynamics study

We report the results of massive parallel molecular dynamics simulations of high-energy radiation damage in phosphate glasses. This damage is created by overlapping multiple 70 keV collision cascades. We quantify different aspects of radiation-induced structural changes including at different stages of damage development, including coordination numbers, cluster sizes and density. The overall trend is that radiation damage causes polymerisation of the glass network and the loss of small and isolated clusters, however the details of this response vary with different glass compositions.

A parallel and performance portable implementation of a full-field crystal plasticity model

We have developed a parallel implementation of an Elasto-Viscoplastic Fast Fourier Transform-based (EVPFFT) micromechanical solver to enable computationally efficient crystal plasticity modeling for polycrystalline materials. Our primary focus lies in achieving performance portability, allowing a single EVPFFT implementation to run optimally on various homogeneous architectures, including multi-core Central Processing Units (CPUs), as well as on heterogeneous computer architectures comprising multi-core CPUs and Graphics Processing Units (GPUs) from different vendors. To accomplish this goal, we have leveraged MATAR, a C++ software library that simplifies the creation and utilization of multidimensional dense or sparse matrix and array data structures. These data structures are designed to be portable across diverse architectures through the use of Kokkos, a performance-portable library. Additionally, we have employed the Message Passing Interface (MPI) to efficiently distribute the computational workload among processors. The heFFTe (Highly Efficient FFT for Exascale) library is used to facilitate the performance portability of the fast Fourier transforms (FFTs) computation. The computational performance of EVPFFT is evaluated and presented in terms of parallel scalability and simulation runtime on different high-performance computing (HPC) architectures. The utility of the developed framework to efficiently simulate the micro-mechanical fields in polycrystalline microstructures in engineering applications is discussed. Program summaryProgram Title: EVPFFTCPC Library link to program files:https://doi.org/10.17632/2k8579fyyv.1Developer's repository link:https://github.com/lanl/FierroLicensing provisions: BSD 3-Clause LicenseProgramming language: C++External routines/libraries: MPI, Kokkos, MATAR, HeFFTe, HDF5Nature of problem: EVPFFT is a crystal plasticity code designed to compute micro-mechanical fields within a polycrystalline representative volume element (RVE) and predict the macroscale response of the RVE.Solution method: EVPFFT uses the periodic Green's function method in Fourier space to solve the field equations of static stress equilibrium in a periodic spatial domain.

Dynamic characteristics testing and modeling in electromagnetic-thermal-force coupling of electromagnetic devices using parallel finite element simulation

Electromagnetic systems of Electromagnetic devices involve multiphysics coupling of electromagnetic fields, electrical circuits, mechanical motion, and temperature fields. When the digital prototype model has a large number of meshes, the commercial finite element software also has the disadvantage of long computation time in the simulation of dynamic characteristics of electrical apparatus. This paper proposes a transmission line method (TLM) based digital modeling method for the electromagnetic-thermal-force coupling to test and evaluate the static and dynamic characteristics of electromagnetic devices more accurately and efficiently. The case study is developed on a direct current (DC) contactor, and the dynamic characteristics testing and modeling in the electromagnetic-thermal-force coupling of the contactor are investigated. The numerical results of the finite element method (FEM) model for the electromagnetic system of the contactor are compared with the commercial FEM software Altair Flux and the experimental results, which demonstrate the exceptional accuracy of the proposed digital modeling method for the multiphysics coupling of the DC contactor. Meanwhile, the dynamic characteristics of the DC contactor at different ambient temperatures are measured and evaluated, and the comparison with the experimental measurements shows the applicability of the proposed digital prototype model at different ambient temperatures. Compared with the conventional Newton-Raphson (NR) iteration, when dividing fine mesh and extremely fine mesh, the TLM parallel computing significantly improves the efficiency of solving the dynamic characteristics of multiphysics coupling by about 35 % and 44 % with the same computational accuracy, respectively. The dynamic characteristics evaluation method combined with digital modeling technology and experimental measurements proposed in this paper demonstrates high accuracy and solving efficiency, making a notable contribution to the field of design and optimization of electromagnetic devices.

The analysis of extension evolution law and damage behavior characteristic for line microcracks on Si3N4 bearing balls

To predict the relation between extension evolution law and damage of line microcracks on silicon nitride bearing balls. Firstly, the 3D point cloud features of line microcracks are obtained based on multi-image 3D reconstruction. Next, the point cloud diagram data is optimized to fit the molecular dynamics geometric model of line microcracks extension process. Then, according to the Si3N4 molecular dynamics deep learning potential, the molecular dynamics mechanical model is established. Finally, Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) is used to numerically solve the extension evolution of line microcracks. The experiment result shows that from the change analysis of single-layer surface mesh, the microcracks extension is concentrated in the central region of the indentation. The new microcracks spread along both sides of the original microcracks. After uniformly distributed load is applied to line microcracks, there is a phenomenon of concentrated swirling microcracks formation under the indentation region. By analyzing the atomic coordination number nephogram, the microcracks extension occurs when the proportion of amorphous atoms is up to 7 % during loading. And the plastic deformation occurs when the proportion of amorphous atoms is dropped to 6 % during unloading. In addition, in the process of indentation loading, plastic deformation and slip zone occur on Si3N4 bearing balls surface. However, the strain is accumulated in the middle region during unloading. The slip zone moves inward and becomes thin, and the elastic deformation partially is recovered.

Colored Noise in PPO: Improved Exploration and Performance through Correlated Action Sampling

Proximal Policy Optimization (PPO), a popular on-policy deep reinforcement learning method, employs a stochastic policy for exploration. In this paper, we propose a colored noise-based stochastic policy variant of PPO. Previous research highlighted the importance of temporal correlation in action noise for effective exploration in off-policy reinforcement learning. Building on this, we investigate whether correlated noise can also enhance exploration in on-policy methods like PPO. We discovered that correlated noise for action selection improves learning performance and outperforms the currently popular uncorrelated white noise approach in on-policy methods. Unlike off-policy learning, where pink noise was found to be highly effective, we found that a colored noise, intermediate between white and pink, performed best for on-policy learning in PPO. We examined the impact of varying the amount of data collected for each update by modifying the number of parallel simulation environments for data collection and observed that with a larger number of parallel environments, more strongly correlated noise is beneficial. Due to the significant impact and ease of implementation, we recommend switching to correlated noise as the default noise source in PPO.

3.0 - MOOSE: Enabling massively parallel multiphysics simulations

The development of MOOSE has kept accelerating since the last release, with over 2,100 pull requests merged over the last 30 months that involved nearly fifty contributors across close to a dozen institutions internationally. The growth in MOOSE's capabilities and downstream applications is reflected in the growth of the community. User support provided on the GitHub discussions forum has steadily increased to nearly 50 daily interactions. New simulation projects, notably to model advanced nuclear reactor and fusion devices, are driving a significant expansion of the capabilities. This paper reports on these developments, with several major released features, new physics modules, and key improvements to the user experience and simulation workflow.

Rotor Performance Predictions for Urban Air Mobility: Single vs. Coaxial Rigid Rotors

This work details the development and validation of a methodology for high-resolution rotor models used in hybrid Blade Element Momentum Theory Unsteady Reynolds Averaged Navier–Stokes (BEMT-URANS) CFD. The methodology is shown to accurately predict single and coaxial rotor performance in a fraction of the time required by conventional CFD methods. The methodology has three key features: (1) a high-resolution BEMT rotor model enabling large reductions in grid size, (2) a discretized set of momentum sources to interface between the BEMT rotor model and the structured URANS flow solver, and (3) leveraging of the first two features to enable highly parallelized GPU-accelerated multirotor CFD simulations. The hybrid approach retains high-fidelity rotor inflow, wake propagation, and rotor–rotor interactional effects at a several orders of magnitude lower computational cost compared to conventional blade-resolved CFD while retaining high accuracy on steady rotor performance metrics. Rotor performance predictions of thrust and torque for both single and coaxial rotor configurations are compared to test the data that the authors obtained at the NASA Langley 14- by 22-ft. Subsonic Tunnel Facility. Simulations were run with both fully turbulent and free-transition airfoil performance tables to quantify the associated uncertainty. Single rotor thrust and torque were predicted on average within 4%. Coaxial thrust and power were predicted within an average of 5%. A vortex ring state (VRS) shielding phenomenon for coaxial rotor systems is also presented and discussed. The results support that this hybrid BEMT-URANS CFD methodology can be highly parallelized on GPU machines to obtain accurate rotor performance predictions across the full spectrum of possible UAM flight conditions in a fraction of the time required by conventional higher-fidelity methods. This strategy can be used to rapidly create look-up tables with hundreds to thousands of flight conditions using a three-dimensional multirotor CFD for UAM.

Simulation Study of Polyethylene Terephthalate Hydrolase Adsorption on Self-Assembled Monolayers.

Polyethylene terephthalate (PET) hydrolase, discovered in Ideonella sakaiensis (IsPETase), is a promising agent for the biodegradation of PET under mild reaction conditions, yet the thermal stability is poor. The efficient immobilization and orientation of IsPETase on different solid substrates are essential for its application. In this work, the combined parallel tempering Monte Carlo simulation with the all-atom molecular dynamics simulation approach was adopted to reveal the adsorption mechanism, orientation, and conformational changes of IsPETase adsorbed on charged self-assembled monolayers (SAMs), including COOH-SAM and NH2-SAM with different surface charge densities (SCDs). The results show that the protein adsorption orientation was determined not only by attraction interactions but also by repulsion interactions. IsPETase is adsorbed on the COOH-SAM surface with an "end-on" orientation, which favors the exposure of the catalyzed triplet to the solution. In addition, the entrance to the catalytic active center is larger on the COOH-SAM surface with a low SCD. This work reveals the controlled orientation and conformational information on IsPETase on charged surfaces at the atomistic level. This study would certainly promote our understanding of the mechanism of IsPETase adsorption and provide theoretical support for the design of substrates for IsPETase immobilization.

A universal parallel simulation framework for energy pipeline networks on high-performance computers

Energy distribution networks represent crucial infrastructures for modern society, and various simulation tools have been widely used by energy suppliers to manage these intricate networks. However, simulation calculations include a large number of fluid control equations, and computational overhead limits the performance of simulation software. This paper proposes a universal parallel simulation framework for energy pipeline networks that takes advantages of data parallelism and computational independence between network elements. A non-pipe model of an energy supply network is optimized, and the input and output of the network model in the proposed framework are modified, which can reduce the development burden during the numerical computations of the pipeline network and weaken the computational correlation between different simulated components. In addition, independent computations can be performed concurrently through periodic data exchange procedures between component instances, improving the parallelism and efficiency of simulation computations. Further, a parallel water pipelines network simulation computing paradigm based on a heterogeneous computer hardware architecture is used to evaluate the proposed framework’s performance. A series of tests are conducted to verify the accuracy of the proposed framework, and simulation errors of less than 5% are achieved. The results of multi-threaded simulation experiments have demonstrated the feasibility of the proposed framework in a parallel computing approach. Moreover, an Advanced Micro Devices (AMD) Deep Computing Unit (DCU)-parallel program is implemented into a water supply network simulation system; the computational efficiency of this system is compared with that of its serial counterpart. The experimental results show that the proposed framework is appropriate for high-performance computer architectures, and the 18x speed-up ratio demonstrates that the parallel program based on the proposed universal framework outperforms the serial program. That provides the basis for the application of pipe network simulation on high-performance computers.

Molecular dynamics to model carbon infiltration into a porous silicon matrix: An experimental and computational approach

In this work, a porous silicon matrix (PSm) and its functionalization with carbon atoms are modeled by molecular dynamics (MD); the simulation is correlated experimentally to analyze the model results. Computationally, the Tersoff potential for the silicon carbon system is used to build a model by MD using Large-scale Atomic/Molecular Massively Parallel Simulator free-source code (LAMMPS). First, a model of a PSm with an aleatory distributed macroporous was built; then, C atoms were injected into the PSm model to functionalize the matrix; finally, the model shows the functionalization of the PSm, forming carbon-silicon composites, which are Si clusters surrounded by C atoms. Experimentally, PSm was fabricated using samples of a p-type crystalline silicon wafer by electrochemical etching. A colloidal suspension of graphene oxide (GOs) was used as a source of carbon atoms: the GOs was added to the electrolyte, during the etching process, to functionalize the PSm with C atoms. AFM images show that the PSm pores are not obstructed after functionalization. Carbon-silicon composites were formed into the PSm samples according to X-ray diffraction and EDS characterizations. The results of the modeling agree with the results of the experiments.

Modeling and improvement for the thermal stability of ring-shaped workpieces with shape boundary constraints

Ring-shaped workpieces are commonly utilized in high-precision measuring instruments, and their thermal deformation affects the instruments’ measuring accuracy. A novel method for establishing the thermal deformation model of ring-shaped workpieces is proposed in this article. The mechanism of thermal deformation caused by shape boundaries is investigated using the principle of molecular dynamics. A mathematical model between deformation degree and height, diameter ratio, or temperature is created using a large-scale atomic/molecular massively parallel simulator. The established model is verified by measuring the thermal deformation of ring-shaped workpieces. The model is used to optimize laser collimation systems, and stability experiments for laser collimation systems in different sizes are performed. The stability of optimized systems can be improved by 50%, 50%, and 48% with temperature increases of 10 °C, 20 °C, and 30 °C, respectively. The experimental results indicate the obtained model can be utilized to improve the stability of instruments.

Promoting nanoscale deformation twinning through FCC phase decomposition in AlCoCrFeMo0.05Ni2 high entropy alloy

The high temperature plasticity of AlCoCrFeMo0.05Ni2 high entropy alloy (HEA) is significantly enhanced after annealing. However, the specific mechanisms are not fully understood. In this study, a polycrystalline HEA model with dimensions of 18 nm×18 nm×26 nm was used to analyze stress mapping under uniaxial tension through Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS). Complemented by experimental results, one can see that the decomposition of the FCC phase promotes the formation of nanoscale deformation twins. The formation of deformation twinning is most likely one of the primary reasons for the substantial improvement in high temperature plasticity. The findings provide valuable insights for controlling the high temperature performance of the alloy.

Performance Evaluation of Spintronic-Based Spiking Neural Networks using Parallel Discrete-Event Simulation

Spintronics devices that use the spin of electrons as the information state variable have the potential to emulate neuro-synaptic dynamics and can be realized within a compact form-factor, while operating at ultra-low energy-delay point. In this paper, we benchmark the performance of a spintronics hardware platform designed for handling neuromorphic tasks. To explore the benefits of spintronics-based hardware on realistic neuromorphic workloads, we developed a Parallel Discrete-Event Simulation model called Doryta, which is further integrated with a materials-to-systems benchmarking framework. The benchmarking framework allows us to obtain quantitative metrics on the throughput and energy of spintronics-based neuromorphic computing and compare these against standard CMOS-based approaches. Although spintronics hardware offers significant energy and latency advantages, we find that for larger neuromorphic circuits, the performance is limited by the interconnection networks rather than the spintronics-based neurons and synapses. This limitation can be overcome by architectural changes to the network. Through Doryta we are also able to show the power of neuromorphic computing by simulating Conway’s Game of Life (GoL), thus showing that it is Turing complete. We show that Doryta obtains over 300 × speedup using 1,024 CPU cores when tested on a convolutional, sparse, neural architecture. When scaled-up 64 times, to a 200 million neuron model, the simulation ran in 3:42 minutes for a total of 2000 virtual clock steps. The conservative approach of execution was found to be faster in most cases than the optimistic approach, even when a tie-breaking mechanism to guarantee deterministic execution, was deactivated.

HydroCompute: An open-source web-based computational library for hydrology and environmental sciences

We present HydroCompute, a high-performance client-side computational library specifically designed for web-based hydrological and environmental science applications. Leveraging state-of-the-art technologies in web-based scientific computing, the library facilitates both sequential and parallel simulations, optimizing computational efficiency. Employing multithreading via web workers, HydroCompute enables the porting and utilization of various engines, including WebGPU, Web Assembly, and native JavaScript code. Furthermore, the library supports local data transfers through peer-to-peer communication using WebRTC. The flexible architecture and open-source nature of HydroCompute provide effective data management and decision-making capabilities, allowing users to integrate their own code into the framework. To demonstrate the capabilities of the library, we conducted two case studies: a benchmarking study assessing the performance of different engines and a real-time data processing and analysis application for the state of Iowa. The results exemplify HydroCompute's potential to enhance computational efficiency and contribute to the interoperability and advancement of hydrological and environmental sciences.