Total Wall Clock Time Research Articles

Sweep interpolation: a cost-effective semi-Lagrangian scheme in the Global Environmental Multiscale model

Abstract. The interpolation process is the most computationally expensive step of the semi-Lagrangian (SL) approach for solving advection and is commonly used in numerical weather prediction (NWP) models. It has a significant impact on the accuracy of the solution and can potentially be the most expensive part of model integration. The sweep algorithm, which was first described by Mortezazadeh and Wang (2017), performs SL interpolation with the same computational cost as a third-order polynomial scheme but at the accuracy of the fourth order. This improvement is achieved by using two third-order backward and forward polynomial interpolation schemes in two consecutive time steps. In this paper, we present a new application of the sweep algorithm within the context of global forecasts produced with Environment Climate Change Canada's Global Environmental Multiscale (GEM) model. Results show that the SL scheme with sweep interpolation is computationally more efficient compared to a conventional SL scheme with fourth-order polynomial interpolation, especially when a very large number of passive tracers are advected. An additional advantage of this new approach is that its implementation in a chemical and weather forecast model requires minimum modifications of the interpolation weighting coefficients. An analysis of the computational performance for a set of theoretical benchmarks as well as a global ozone forecast experiment show that up to 15 % reduction in total wall clock time is achieved. Forecasting experiments using the global version of the GEM model and the new interpolation show that the sweep interpolation can perform very well in predicting ozone distribution, especially in the tropopause region, where transport processes play a significant role.

Implementation of OpenMP Parallelization of Rate-Dependent Ceramic Peridynamic Model

A rate-dependent peridynamic ceramic model, considering the brittle tensile response, compressive plastic softening and strain-rate dependence, can accurately represent the dynamic response and crack propagation of ceramic materials. However, it also considers the strain-rate dependence and damage accumulation caused by compressive plastic softening during the compression stage, requiring more computational resources for the bond force evaluation and damage evolution. Herein, the OpenMP parallel optimization of the rate-dependent peridynamic ceramic model is investigated. Also, the modules that compute the interactions between material points and update damage index are vectorized and parallelized. Moreover, the numerical examples are carried out to simulate the dynamic response and fracture of the ceramic plate under normal impact. Furthermore, the speed-up ratio and computational efficiency by multi-threads are evaluated and discussed to demonstrate the reliability of parallelized programs. The results reveal that the total wall clock time has been significantly reduced after optimization, showing the promise of parallelization process in terms of accuracy and stability.

Pseudospectral implementations of long-range corrected density functional theory.

We have implemented pseudospectral density-functional theory (DFT) with long-range corrected DFT functionals (PS-LRC) in quantum mechanics package Jaguar, and applied it in the calculations of geometry optimizations, dimmer interaction energies, polarizabilities and first-order hyperpolarizabilities, harmonic vibrational frequencies, S1 and T1 excitation energies, singlet-triplet gaps, charge transfer numbers, oscillator strengths, reaction barrier heights, electron-transfer couplings, and charge-transfer excitation energies. From our accuracy benchmark analysis, PS grids, PS dealiasing functions, PS atomic corrections, PS multigrid strategy, PS length scales, and PS cutoff scheme perform well in PS DFT with LRC density functionals with very small and ignorable deviations when compared to the conventional spectral (CS) method. The timing benchmark study of S1 excitation energy calculations of fullerenes (Cn , n up to 540) demonstrates that PS-LRC achieves 1.4-8.4-fold speedups in SCF, 22-32-fold speedups in Tamm-Dancoff approximation, and 6-15-fold speedups in total wall clock time with an average error 0.004 eV of excitation energies compared to the CS method.

Highly efficient implementation of the analytical gradients of pseudospectral time-dependent density functional theory.

The accuracy and efficiency of time-dependent density functional theory (TDDFT) excited state gradient calculations using the pseudospectral method are presented. TDDFT excited state geometry optimizations of the G2 test set molecules, the organic fluorophores with large Stokes shifts, and the Pt-complexes show that the pseudospectral method gives average errors of 0.01-0.1 kcal/mol for the TDDFT excited state energy, 0.02-0.06pm for the bond length, and 0.02-0.12° for the bond angle when compared to the results from conventional TDDFT. TDDFT gradient calculations of fullerenes (Cn, n up to 540) with the B3LYP functional and 6-31G** basis set show that the pseudospectral method provides 8- to 14-fold speedups in the total wall clock time over the conventional methods. The pseudospectral TDDFT gradient calculations with a diffuse basis set give higher speedups than the calculations for the same basis set without diffuse functions included.

Deep learning for NLTE spectral opacities

Computer simulations of high energy density science experiments are computationally challenging, consisting of multiple physics calculations including radiation transport, hydrodynamics, atomic physics, nuclear reactions, laser–plasma interactions, and more. To simulate inertial confinement fusion (ICF) experiments at high fidelity, each of these physics calculations should be as detailed as possible. However, this quickly becomes too computationally expensive even for modern supercomputers, and thus many simplifying assumptions are made to reduce the required computational time. Much of the research has focused on acceleration techniques for the various packages in multiphysics codes. In this work, we explore a novel method for accelerating physics packages via machine learning. The non-local thermodynamic equilibrium (NLTE) package is one of the most expensive calculations in the simulations of indirect drive inertial confinement fusion, taking several tens of percent of the total wall clock time. We explore the use of machine learning to accelerate this package, by essentially replacing the physics calculation with a deep neural network that has been trained to emulate the physics code. We demonstrate the feasibility of this approach on a simple problem and perform a side-by-side comparison of the physics calculation and the neural network inline in an ICF Hohlraum simulation. We show that the neural network achieves a 10× speed up in NLTE computational time while achieving good agreement with the physics code for several quantities of interest.

Challenge data set for macromolecular multi-microcrystallography.

A synthetic data set demonstrating a particularly challenging case of indexing ambiguity in the context of radiation damage was generated. This set shall serve as a standard benchmark and reference point for the ongoing development of new methods and new approaches to robust structure solution when single-crystal methods are insufficient. Of the 100 short wedges of data, only the first 36 are currently necessary to solve the structure by `cheating', or using the correct reference structure as a guide. The total wall-clock time and number of crystals required to solve the structure without cheating is proposed as a metric for the efficacy and efficiency of a given multi-crystal automation pipeline.

Accelerating multi-dimensional combustion simulations using GPU and hybrid explicit/implicit ODE integration

Simulating multi-dimensional combustion with detailed kinetics often requires solving a large number of ordinary differential equation (ODE) problems at each global time step. In many cases, the ODE integrations account for the bulk of the total wall-clock time for the simulation. This paper introduces CHEMEQ2-GPU – a new explicit stiff ODE solver (based on the existing CHEMEQ2 solver) that exploits the parallel architecture of the modern graphics processing unit (GPU) to accelerate ODE integration in multi-dimensional combustion simulations. We also demonstrate efficient application of the CPU and GPU as co-processors, for further speedup. We describe a hybrid explicit/implicit ODE solver approach that combines the strengths of both solver types running simultaneously on the GPU and CPU, respectively. A dynamic load balancing scheme was used to assign the kinetics ODE integrations over all grid points to either the CPU-based implicit solver DVODE (which is the more efficient solver for highly stiff grid points) or CHEMEQ2-GPU (more efficient for moderately stiff or non-stiff grid points). We demonstrate CHEMEQ2-GPU and the hybrid approach in 3-D simulations of homogeneous charge compression ignition (HCCI) engines. The test cases applied two different n-heptane reaction mechanisms (a large detailed model and a small skeletal model) and three different mesh sizes. Engine simulations were performed using KIVA-CHEMKIN. CHEMEQ2 was about 2–3 times faster than DVODE, with similar prediction accuracy. The CHEMEQ2-GPU speedup relative to CHEMEQ2 increased linearly with the number of grid points for the range of meshes tested in this work. Assuming ideal linear scaling of simulation time with number of processors, the speed of CHEMEQ2-GPU on the Tesla C2050 GPU was equivalent to CHEMEQ2 running on approximately 13 parallel 2.8GHz CPU processors for the finest mesh; and the hybrid solver approach was equivalent to CHEMEQ2 on ∼15 such CPU processors. In summary, CHEMEQ2-GPU provided the additional computing power of 14 parallel CPU processors (for the finest mesh tested) and the hybrid solver approach demonstrated a method to efficiently apply these additional co-processors with existing CPU cores for combustion simulations. CHEMEQ2-GPU scales favorably with the number of grid points and is available by request to the authors. This work presents opportunities for further development, particularly in CPU/GPU load balancing algorithms.

Efficient and accurate approximations to the local coupled cluster singles doubles method using a truncated pair natural orbital basis

A production level implementation of the closed-shell local quadratic configuration interaction and coupled cluster methods with single and double excitations (QCISD and CCSD) based on the concept of pair natural orbitals [local pair natural orbital LPNO-QCISD and LPNO-CCSD) is reported, evaluated, and discussed. This work is an extension of the earlier developed LPNO coupled-electron pair approximation (LNPO-CEPA) method [F. Neese et al., Chem. Phys. 130, 114108 (2009)] and makes extended use of the resolution of the identity (RI) or density fitting (DF) approximation. Two variants of each method are compared. The less accurate approximations (LPNO2-QCISD/LPNO2-CCSD) still recover 98.7%-99.3% of the correlation energy in the given basis and have modest disk space requirements. The more accurate variants (LPNO1-QCISD/LPNO1-CCSD) typically recover 99.75%-99.95% of the correlation energy in the given basis but require the Coulomb and exchange operators with up to two-external indices to be stored on disk. Both variants have comparable computational efficiency. The convergence of the results with respect to the natural orbital truncation parameter (T(CutPNO)) has been studied. Extended numerical tests have been performed on absolute and relative correlation energies as function of basis set size and T(CutPNO) as well as on reaction energies, isomerization energies, and weak intermolecular interactions. The results indicate that the errors of the LPNO methods compared to the canonical QCISD and CCSD methods are below 1 kcal/mol with our default thresholds. Finally, some calculations on larger molecules are reported (ranging from 40-86 atoms) and it is shown that for medium sized molecules the total wall clock time required to complete the LPNO-CCSD calculations is only two to four times that of the preceding self-consistent field (SCF). Thus these methods are highly suitable for large-scale computational chemistry applications. Since there are only three thresholds involved that have been given conservative default values, the methods can be confidentially used in a "black-box" fashion in the same way as their canonical counterparts.