Efficient Large-scale Simulations Research Articles

Anisotropic Interlayer Force Field for Group-VI Transition Metal Dichalcogenides.

An anisotropic interlayer force field that describes the interlayer interactions in homogeneous and heterogeneous interfaces of group-VI transition metal dichalcogenides (MX2, where M = Mo, W, and X = S, Se) is presented. The force field is benchmarked against density functional theory calculations for bilayer systems within the Heyd-Scuseria-Ernzerhof hybrid density functional approximation, augmented by a nonlocal many-body dispersion treatment of long-range correlation. The parametrization yields good agreement with the reference calculations of binding energy curves and sliding potential energy surfaces. It is found to be transferable to transition metal dichalcogenide (TMD) junctions outside of the training set that contain the same atom types. Calculated bulk moduli agree with most previous dispersion-corrected density functional theory predictions, which underestimate the available experimental values. Calculated phonon spectra of the various junctions under consideration demonstrate the importance of appropriately treating the anisotropic nature of the layered interfaces. Considering our previous parametrization for MoS2, the anisotropic interlayer potential enables accurate and efficient large-scale simulations of the dynamical, tribological, and thermal transport properties of a large set of homogeneous and heterogeneous TMD interfaces.

Modelling opinion dynamics under the impact of influencer and media strategies

Digital communication has made the public discourse considerably more complex, and new actors and strategies have emerged as a result of this seismic shift. Aside from the often-studied interactions among individuals during opinion formation, which have been facilitated on a large scale by social media platforms, the changing role of traditional media and the emerging role of “influencers” are not well understood, and the implications of their engagement strategies arising from the incentive structure of the attention economy even less so. Here we propose a novel framework for opinion dynamics that can accommodate various versions of opinion dynamics as well as account for different roles, namely that of individuals, media and influencers, who change their own opinion positions on different time scales. Numerical simulations of instances of this framework show the importance of their relative influence in creating qualitatively different opinion formation dynamics: with influencers, fragmented but short-lived clusters emerge, which are then counteracted by more stable media positions. The framework allows for mean-field approximations by partial differential equations, which reproduce those dynamics and allow for efficient large-scale simulations when the number of individuals is large. Based on the mean-field approximations, we can study how strategies of influencers to gain more followers can influence the overall opinion distribution. We show that moving towards extreme positions can be a beneficial strategy for influencers to gain followers. Finally, our framework allows us to demonstrate that optimal control strategies allow other influencers or media to counteract such attempts and prevent further fragmentation of the opinion landscape. Our modelling framework contributes to a more flexible modelling approach in opinion dynamics and a better understanding of the different roles and strategies in the increasingly complex information ecosystem.

Two novel families of multiscale staggered patch schemes efficiently simulate large-scale, weakly damped, linear waves

Many multiscale wave systems exhibit macroscale emergent behaviour, for example, the fluid dynamics of floods and tsunamis. Resolving a large range of spatial scales typically requires prohibitively high computational costs. The small dissipation in wave systems poses a significant challenge to further developing multiscale modelling methods in multiple dimensions. This article develops and evaluates two families of equation-free multiscale methods on novel 2D staggered patch schemes, and demonstrates the power and utility of these multiscale schemes for weakly damped linear waves. A detailed study of sensitivity to numerical roundoff errors establishes the robustness of developed staggered patch schemes. Comprehensive eigenvalue analysis over a wide range of parameters establishes the stability, accuracy, and consistency of the multiscale schemes. Analysis of the computational complexity shows that the measured compute times of the multiscale schemes may be 100,000 times smaller than the compute time for the corresponding (same resolution) full-domain computation. This work provides the essential foundation for efficient large-scale simulation of challenging nonlinear multiscale waves.

Large-scale benchmarks of the time-warp/graph-theoretical kinetic Monte Carlo approach for distributed on-lattice simulations of catalytic kinetics.

Motivated by the need to perform large-scale kinetic Monte Carlo (KMC) simulations, in the context of unravelling complex phenomena such as catalyst reconstruction and pattern formation, we extend the work of Ravipati et al. [S. Ravipati, G. D. Savva, I.-A. Christidi, R. Guichard, J. Nielsen, R. Réocreux and M. Stamatakis, Comput. Phys. Commun., 2022, 270, 108148] in benchmarking the performance of a distributed-computing, on-lattice KMC approach. The latter, implemented in our software package Zacros, combines the graph-theoretical KMC framework with the Time-Warp algorithm for parallel discrete event simulations, and entails dividing the lattice into subdomains, each assigned to a processor. The cornerstone of the Time-Warp algorithm is the state queue, to which snapshots of the simulation state are saved regularly, enabling historical KMC information to be corrected when conflicts occur at subdomain boundaries. Focusing on three model systems, we highlight the key Time-Warp parameters that can be tuned to optimise performance. The frequency of state saving, controlled by the state saving interval, δsnap, is shown to have the largest effect on performance, which favours balancing the overhead of re-simulating KMC history with that of writing state snapshots to memory. Also important is the global virtual time (GVT) computation interval, ΔτGVT, which has little direct effect on the progress of the simulation but controls how often the state queue memory can be freed up. We also find that pre-allocating memory for the state queue data structure favours performance. These findings will guide users in maximising the efficiency of Zacros or other distributed KMC software, which is a vital step towards realising accurate, meso-scale simulations of heterogeneous catalysis.

Gradient strengthening effects in mode I tearing of ductile plate at the engineering scale

The present study aims to quantify the influence of void size effect on crack initiation and growth in mode I tearing of large ductile plates. Specifically, the aim is to demonstrate how size effects influence the transition from crack initiation to steady-state tearing and reveal the evolution in both the traction-separation relation and the fracture energy dissipated in the tearing process when related to large-scale cohesive modeling. The study investigates mode I tearing by adopting the gradient enriched Gurson–Tvergaard–Needleman (GTN) model by Niordson and Tvergaard (2019), where the novelty lies in accounting for strain gradient hardening near growing micro-voids in relation to large-scale plate tearing. The steady-state crack growth conditions are compared to a 2D plane strain model setup, while the transition from crack initiation to steady-state is analyzed using full 3D plate tearing calculations. A discussion about the 2D versus 3D models’ capability, their differences, and agreement is presented. The results are related to the traction-separation relations fit for efficient large-scale simulations, e.g., using cohesive zone modeling. The present study shows that the material length parameter incorporated into the constitutive model has a negligible impact on the onset of plate thinning (or necking) but delays the onset of shear localization and, thus, increases the total cohesive fracture energy.

Topological and geometric patterns in optimal bang-bang protocols for variational quantum algorithms: Application to the XXZ model on the square lattice

In this work, we address the challenge of uncovering patterns in variational optimal protocols for taking the system to ground states of many-body Hamiltonians, using variational quantum algorithms. We develop highly optimized classical Monte Carlo (MC) algorithms to find the optimal protocols for transformations between the ground states of the square-lattice XXZ model for finite systems sizes. The MC method obtains optimal bang-bang protocols, as predicted by Pontryagin's minimum principle. We identify the minimum time needed for reaching an acceptable error for different system sizes as a function of the initial and target states and uncover correlations between the total time and the wave-function overlap. We determine a dynamical phase diagram for the optimal protocols, with different phases characterized by a topological number, namely the number of on-pulses. Bifurcation transitions as a function of initial and final states, associated with new jumps in the optimal protocols, demarcate these different phases. The number of pulses correlates with the total evolution time. In addition to identifying the topological characteristic above, i.e., the number of pulses, we introduce a correlation function to characterize bang-bang protocols' quantitative geometric similarities. We find that protocols within one phase are indeed geometrically correlated. Identifying and extrapolating patterns in these protocols may inform efficient large-scale simulations on quantum devices.

OpenSBLI: Automated code-generation for heterogeneous computing architectures applied to compressible fluid dynamics on structured grids

OpenSBLI is an open-source code-generation system for compressible fluid dynamics (CFD) on heterogeneous computing architectures. Written in Python, OpenSBLI is an explicit high-order finite-difference solver on structured curvilinear meshes. Shock-capturing is performed by a choice of high-order Weighted Essentially Non-Oscillatory (WENO) or Targeted Essentially Non-Oscillatory (TENO) schemes. OpenSBLI generates a complete CFD solver in the Oxford Parallel Structured (OPS) domain specific language. The OPS library is embedded in C code, enabling massively-parallel execution of the code on a variety of high-performance-computing architectures, including GPUs. The present paper presents a code base that has been completely rewritten from the earlier proof of concept Jacobs et al. (2017) [7], allowing shock capturing, coordinate transformations for complex geometries, and a wide range of boundary conditions, including solid walls with and without heat transfer. A suite of validation and verification cases are presented, plus demonstration of a large-scale Direct Numerical Simulation (DNS) of a transitional Shockwave Boundary Layer Interaction (SBLI). The code is shown to have good weak and strong scaling on multi-GPU clusters. We demonstrate that code-generation and domain specific languages are suitable for performing efficient large-scale simulations of complex fluid flows on emerging computing architectures. Program summaryProgram Title: OpenSBLI code-generation framework for compressible fluid dynamics on heterogeneous architecturesCPC Library link to program files:https://doi.org/10.17632/3sdb6hck2c.1Developer's repository link:https://github.com/opensbli/opensbliLicensing provisions: GPLv3Programming languages: Python, C/C++, OPS DSLNature of problem: The compressible 3D Navier-Stokes equations are solved via Implicit Large Eddy Simulation (ILES) or Direct Numerical Simulation (DNS).Solution method: OpenSBLI [1,2] is a Python-based code-generation system that uses symbolic algebra to generate a complete CFD solver in C/C++. The basic algorithm is a stencil-based finite-difference solver on structured curvilinear meshes. Shock-capturing is performed by a selection of high-order Weighted/Targeted Essentially Non-Oscillatory (WENO/TENO) schemes. Explicit low-storage Runge-Kutta schemes are used for time-advancement.Additional comments including restrictions and unusual features: The generated code is compliant with the Oxford Parallel Structured (OPS) [3] software library. OpenSBLI/OPS executables can be generated for the OpenMP, MPI, CUDA, OpenCL, and OpenACC parallel programming paradigms. Multi-GPU support is available via combinations of MPI with CUDA, OpenCL or OpenACC.

An Implementation of Replica Exchange with Dynamical Scaling for Efficient Large-Scale Simulations.

An implementation of the replica exchange with dynamical scaling (REDS) method in the commonly used molecular dynamics program GROMACS is presented. REDS is a replica exchange method that requires fewer replicas than conventional replica exchange while still providing data over a range of temperatures and can be used in either constant volume or constant pressure ensembles. Details for running REDS simulations are given, and an application to the human islet amyloid polypeptide (hIAPP) 11-25 fragment shows that the model efficiently samples conformational space.

Hermes: Enabling efficient large-scale simulation in MATSim

Large scale simulations of transportation networks can yield highly valuable insights into the design decisions needed to deliver the best possible transportation service. Current technologies, however, are generally unable to provide support for large scale simulations as they often require impractical amounts of time to compute results. To address this issue, we have developed Hermes, a novel simulation core for MATSim designed as an alternative to QSim. Using an event-driven approach, as well as numerous optimizations to the data structures and the simulation logic, Hermes is capable of completing simulations of larger scale scenarios within feasible time frames. We demonstrate Hermes’ capabilities through extensive experiments as well as through production level results obtained from the Swiss Federal Railways (SBB), where Hermes is currently being used in production.

Scaling of a Large-Scale Simulation of Synchronous Slow-Wave and Asynchronous Awake-Like Activity of a Cortical Model With Long-Range Interconnections.

Cortical synapse organization supports a range of dynamic states on multiple spatial and temporal scales, from synchronous slow wave activity (SWA), characteristic of deep sleep or anesthesia, to fluctuating, asynchronous activity during wakefulness (AW). Such dynamic diversity poses a challenge for producing efficient large-scale simulations that embody realistic metaphors of short- and long-range synaptic connectivity. In fact, during SWA and AW different spatial extents of the cortical tissue are active in a given timespan and at different firing rates, which implies a wide variety of loads of local computation and communication. A balanced evaluation of simulation performance and robustness should therefore include tests of a variety of cortical dynamic states. Here, we demonstrate performance scaling of our proprietary Distributed and Plastic Spiking Neural Networks (DPSNN) simulation engine in both SWA and AW for bidimensional grids of neural populations, which reflects the modular organization of the cortex. We explored networks up to 192 × 192 modules, each composed of 1,250 integrate-and-fire neurons with spike-frequency adaptation, and exponentially decaying inter-modular synaptic connectivity with varying spatial decay constant. For the largest networks the total number of synapses was over 70 billion. The execution platform included up to 64 dual-socket nodes, each socket mounting 8 Intel Xeon Haswell processor cores @ 2.40 GHz clock rate. Network initialization time, memory usage, and execution time showed good scaling performances from 1 to 1,024 processes, implemented using the standard Message Passing Interface (MPI) protocol. We achieved simulation speeds of between 2.3 × 109 and 4.1 × 109 synaptic events per second for both cortical states in the explored range of inter-modular interconnections.

On the accurate large-scale simulation of ferrofluids

We present an approach to the accurate and efficient large-scale simulation of the complex dynamics of ferrofluids based on physical principles. Ferrofluids are liquids containing magnetic particles that react to an external magnetic field without solidifying. In this contribution, we employ smooth magnets to simulate ferrofluids in contrast to previous methods based on the finite element method or point magnets. We solve the magnetization using the analytical solution of the smooth magnets' field, and derive the bounded magnetic force formulas addressing particle penetration. We integrate the magnetic field and force evaluations into the fast multipole method allowing for efficient large-scale simulations of ferrofluids. The presented simulations are well reproducible since our approach can be easily incorporated into a framework implementing a Fast Multipole Method and a Smoothed Particle Hydrodynamics fluid solver with surface tension. We provide a detailed analysis of our approach and validate our results against real wet lab experiments. This work can potentially open the door for a deeper understanding of ferrofluids and for the identification of new areas of applications of these materials.

Routine million-particle simulations of epoxy curing with dissipative particle dynamics

Mesoscale simulation techniques have helped to bridge the length scales and time scales needed to predict the microstructures of cured epoxies, but gaps in computational cost and experimental relevance have limited their impact. In this work, we develop an open-source plugin epoxpy for HOOMD-Blue that enables epoxy crosslinking simulations of millions of particles to be routinely performed on a single modern graphics card. We demonstrate the first implementation of custom temperature-time curing profiles with dissipative particle dynamics and show that reaction kinetics depend sensitively on the stochastic bonding rates. We provide guidelines for modeling first-order reaction dynamics in a classic epoxy/hardener/toughener system and show structural sensitivity to the temperature-time profile during cure. We conclude with a discussion of how these efficient large-scale simulations can be used to evaluate ensembles of epoxy processing protocols to quantify the sensitivity of microstructure on processing.

Linear Time Algorithms to Construct Populations Fitting Multiple Constraint Distributions at Genomic Scales.

Computer simulations can be used to study population genetic methods, models, and parameters, as well as to predict potential outcomes. For example, in plant populations, predicting the outcome of breeding operations can be studied using simulations. In-silico construction of populations with pre-specified characteristics is an important task in breeding optimization and other population genetic studies. We present two linear time Simulation using Best-fit Algorithms (SimBA) for two classes of problems where each co-fits two distributions: SimBA-LD fits linkage disequilibrium and minimum allele frequency distributions, while SimBA-hap fits founder-haplotype and polyploid allele dosage distributions. An incremental gap-filling version of previously introduced SimBA-LD is here demonstrated to accurately fit the target distributions, allowing efficient large scale simulations. SimBA-hap accuracy and efficiency is demonstrated by simulating tetraploid populations with varying numbers of founder haplotypes, we evaluate both a linear time greedy algoritm and an optimal solution based on mixed-integer programming. SimBA is available on http://researcher.watson.ibm.com/project/5669.

General-purpose coordinator–master–worker model for efficient large-scale simulation over heterogeneous infrastructure

In this work, we propose a general-purpose coordinator–master–worker (GP-CMW) model to enable efficient and scalable simulation. The model supports distributed and parallel simulation over a heterogeneous computing node architecture with both multi-core CPUs and GPUs. The model aims at maximizing the hardware activity rate while reducing the overall management overhead. The proposed model includes five components: coordinator, priority abstraction layer, master, hardware abstraction layer, and worker. The proposed model is mainly optimized for large-scale simulation that relies on massive parallelizable events. Extensive set of experimental results shows that GP-CMW provides a significant gain from medium to intensive simulation load by exploiting heterogeneous computing resources including CPU and GPU. Regarding simulation runtime, the proposed GP-CMW model delivers a speedup that is 3.6 times faster than the CMW model.

Multiscale Modeling of Cardiovascular Flows for Clinical Decision Support

Patient-specific cardiovascular simulations can provide clinicians with predictive tools, fill current gaps in clinical imaging capabilities, and contribute to the fundamental understanding of disease progression. However, clinically relevant simulations must provide not only local hemodynamics, but also global physiologic response. This necessitates a dynamic coupling between the Navier–Stokes solver and reduced-order models of circulatory physiology, resulting in numerical stability and efficiency challenges. In this review, we discuss approaches to handling the coupled systems that arise from cardiovascular simulations, including recent algorithms that enable efficient large-scale simulations of the vascular system. We maintain particular focus on multiscale modeling algorithms for finite element simulations. Because these algorithms give rise to an ill-conditioned system of equations dominated by the coupled boundaries, we also discuss recent methods for solving the linear system of equations arising from these systems. We then review applications that illustrate the potential impact of these tools for clinical decision support in adult and pediatric cardiology. Finally, we offer an outlook on future directions in the field for both modeling and clinical application.

A virtual environment for the interrogation of 3D polycrystalline microstructures including grain size effects

A finite element analysis of the large deformation of three-dimensional polycrystals is presented using pixel-based finite elements as well as finite elements conforming with grain boundaries. The macroscopic response is obtained through volume-averaging laws. A constitutive framework for elasto-viscoplastic response of single crystals is utilized along with a fully-implicit Lagrangian finite element algorithm for modeling microstructure evolution. The effect of grain size is included by considering a physically motivated measure of lattice incompatibility which provides an updated shearing resistance within grains. A domain decomposition approach is adopted for parallel computation to allow efficient large scale simulations. Conforming grids are adopted to simulate flexible and complex shapes of grains. The computed mechanical properties of polycrystals are shown to be consistent with experimental results for different grain sizes.

Efficient large-scale simulations of a uniformly driven system.

We present results from an efficient algorithm for simulating systems of locally connected elements that are subject to uniformly increasing stresses and that discharge when these stresses reach some threshold. Previously, large-scale simulations of such systems have been hindered by the very-time-consuming search for those elements that are going to discharge next. We avoid this by using a suitable data structure, reducing computer CPU times by several orders of magnitude in typical cases. In particular, we present simulations for a simple version of the Burridge-Knopoff model introduced by Olami, Feder, and Christensen [Phys. Rev. Lett. 68, 1244 (1992)]. Due to the substantially larger lattices and longer simulation times presently used, we find that the conclusions of these authors have to be modified considerably.