Weak Scale Research Articles

A direction splitting scheme for Navier–Stokes–Boussinesq system in spherical shell geometries

AbstractThis article introduces a second‐order direction splitting method for solving the incompressible Navier–Stokes–Boussinesq system in a spherical shell region. The equations are solved on overset Yin–Yang grids, combined with spherical coordinate transforms. This approach allows to avoid the singularities at the poles and keeps the grid size relatively uniform. The downside is that the spherical shell is subdivided into two equally sized, overlapping subdomains that requires the use of Schwarz‐type iterations. The temporal second‐order accuracy is achieved via an artificial compressibility scheme with bootstrapping. The spatial discretization is based on second‐order finite differences on the Marker‐And‐Cell stencil. The entire scheme is implemented in parallel using a domain decomposition iteration and a direction splitting approach for the local solves. The stability, accuracy, and weak scalability of the method is verified on a manufactured solution of the Navier–Stokes–Boussinesq system while its practicality is demonstrated on the natural convection problem in the gap between two concentric spheres.

Efficient Parallel 3D Computation of the Compressible Euler Equations with an Invariant-domain Preserving Second-order Finite-element Scheme

We discuss the efficient implementation of a high-performance second-order collocation-type finite-element scheme for solving the compressible Euler equations of gas dynamics on unstructured meshes. The solver is based on the convex-limiting technique introduced by Guermond et al. (SIAM J. Sci. Comput. 40, A3211–A3239, 2018). As such, it is invariant-domain preserving ; i.e., the solver maintains important physical invariants and is guaranteed to be stable without the use of ad hoc tuning parameters. This stability comes at the expense of a significantly more involved algorithmic structure that renders conventional high-performance discretizations challenging. We develop an algorithmic design that allows SIMD vectorization of the compute kernel, identify the main ingredients for a good node-level performance, and report excellent weak and strong scaling of a hybrid thread/MPI parallelization.

Domain decomposition in the GPU-accelerated Shift Monte Carlo code

The GPU solver within the Shift continuous-energy Monte Carlo neutron transport code has been extended to provide domain decomposition in addition to domain replication to enable the solution of problems with memory requirements exceeding the capacity of a single GPU. The strategy follows the Multiple Set, Overlapping Domain (MSOD) approach that is used in Shift’s CPU solver and integrates into the event-based algorithm used for Shift’s GPU solver. Furthermore, the ability to assign processors to spatial domains non-uniformly has been maintained. Two different approaches for communicating particle data between domains are considered, and multiple criteria for load balancing problems have been investigated. Numerical results are presented for both fresh and depleted small modular nuclear reactor (SMR) cores. A parallel efficiency of approximately 80% was achieved with up to 16 spatial domains measured relative to full domain replication. A scaling study on the Summit supercomputer demonstrates a weak scaling parallel efficiency of over 90% on over 24000 GPUs.

A FETI approach to domain decomposition for meshfree discretizations of nonlocal problems

We propose a domain decomposition method for the efficient simulation of nonlocal problems. Our approach is based on a multi-domain formulation of a nonlocal diffusion problem where the subdomains share “nonlocal” interfaces of the size of the nonlocal horizon. This system of nonlocal equations is first rewritten in terms of minimization of a nonlocal energy, then discretized with a meshfree approximation and finally solved via a Lagrange multiplier approach in a way that resembles the finite element tearing and interconnect method. Specifically, we propose a distributed projected gradient algorithm for the solution of the Lagrange multiplier system, whose unknowns determine the nonlocal interface conditions between subdomains. Several two-dimensional numerical tests on problems as large as 191 million unknowns illustrate the strong and the weak scalability of our algorithm, which outperforms the standard approach to the distributed numerical solution of the problem. This work is the first rigorous numerical study in a two-dimensional multi-domain setting for nonlocal operators with finite horizon and, as such, it is a fundamental step towards increasing the use of nonlocal models in large scale simulations.

Is the severity of initial sensitization to diphenylcyclopropenone in alopecia areata patients predictive of the final clinical response?

Immunotherapy by diphenylcyclopropenone (DPCP) is generally started with 2% DPCP sensitization, however in recent years studies have questioned the necessity of sensitization that may cause patients severe reactions and troubles at the onset of therapy. The purpose of the present study was to evaluate the association between the severity of initial reaction to 2% DPCP sensitization in AA patients and clinical response. In this retrospective study, 110 AA patients who continued therapy for at least 6months were enrolled. Hair loss and hair regrowth rates were calculated based on the Severity of Alopecia Tool (SALT) scoring system. Initial reaction to 2% DPCP sensitization after 2 weeks was graded as negative reaction (absence of any reaction), doubtful reaction (mild erythema, pruritus, and irritation for minutes after test), weak (erythema, mild edema, and scaling), and strong to extreme reaction (vesicles, bullae, ulcer, and discharge). The degrees of the initial reaction to 2% DPCP after 2 weeks were negative reaction 13 (11.81%), doubtful reaction 40 (36.36%), weak reaction 33 (30%), and strong to extreme reaction 24 (21.81%). Patients were divided into two groups: (A) patients with less than 12-month therapy (75 of 110), (B) patients with more than 12-month therapy (35 of 110). Initial reaction to 2% DPCP sensitization was not correlated with hair regrowth rate in either group (group A: Spearman's rho=0.194, p =0.095; group B: Spearman's rho=0.063 p =0.720). After 12-month treatment with DPCP, hair regrowth rate was significantly greater than 6-months therapy (group A: 17.03 ± 37.78, group B: 49.26 ± 36.34; p =0.003). The severity of hair loss at the onset of treatment was significantly associated with the response rate in both groups (p-value <0.002). Based on our results, it is the initial severity of the disease and not the initial reaction to 2% DPCP sensitization that predicts the clinical response to DPCP immunotherapy.

Achieving performance portability in Gaussian basis set density functional theory on accelerator based architectures in NWChemEx

The numerical integration of the exchange–correlation (XC) potential is one of the primary computational bottlenecks in Gaussian basis set Kohn–Sham density functional theory (KS-DFT). To achieve optimal performance and accuracy, care must be taken in this numerical integration to preserve local sparsity as to allow for near linear weak scaling with system size. This leads to an integration scheme with several performance critical kernels which must be hand optimized for each architecture of interest. As the set of available accelerator hardware goes more diverse, a key challenge for developers of KS-DFT software is to maintain performance portability across a wide range of computational architectures. In this work, we examine a modular software design pattern which decouples the implementation details of performance critical kernels from the expression of high-level algorithmic workflows in a device-agnostic language such as C++; thus allowing for developers to target existing and emerging accelerator hardware within a single code base. We consider the efficacy of such a design pattern in the numerical integration of the XC potential by demonstrating its ability to achieve performance portability across a set of accelerator architectures which are representative of those on current and future U.S. Department of Energy Leadership Computing Facilities.

A High-Order Finite-Difference Method on Staggered Curvilinear Grids for Seismic Wave Propagation Applications with Topography

ABSTRACTWe developed a 3D elastic wave propagation solver that supports topography using staggered curvilinear grids. Our method achieves comparable accuracy to the classical fourth-order staggered grid velocity–stress finite-difference method on a Cartesian grid. We show that the method is provably stable using summation-by-parts operators and weakly imposed boundary conditions via penalty terms. The maximum stable timestep obeys a relationship that depends on the topography-induced grid stretching along the vertical axis. The solutions from the approach are in excellent agreement with verified results for a Gaussian-shaped hill and for a complex topographic model. Compared with a Cartesian grid, the curvilinear grid adds negligible memory requirements, but requires longer simulation times due to smaller timesteps for complex topography. The code shows 94% weak scaling efficiency up to 1014 graphic processing units.

Implementing a neural network interatomic model with performance portability for emerging exascale architectures

The two main thrusts of computational science are increasingly accurate predictions and faster calculations; to this end, the zeitgeist in molecular dynamics (MD) simulations is pursuing machine learned and data driven interatomic models, e.g. neural network potentials, and novel hardware architectures, e.g. GPUs. Current implementations of neural network potentials are orders of magnitude slower than traditional interatomic models and while looming exascale computing offers the ability to run large, accurate simulations with these models, achieving portable performance for MD with new and varied exascale hardware requires rethinking traditional algorithms, using novel data structures, and library solutions. We re-implement a neural network interatomic model in CabanaMD, an MD proxy application, built on libraries developed for performance portability. Our implementation shows significantly improved thread scaling in this complex kernel as compared to a current LAMMPS implementation, across both strong and weak scaling. Our single-source solution enables simulations up to 20 million atoms on a single CPU node and 4 million atoms with improved performance on a single GPU. We also explore parallelism and data layout choices (using flexible data structures called AoSoAs) and their effect on performance, seeing up to ∼50% and ∼5% improvements in performance on a GPU by choosing the right level of parallelism and data layout respectively. Program summaryProgram title: CabanaMD-NNPCPC Library link to program files:https://doi.org/10.17632/x948kyy7jh.1Developer's repository link:https://github.com/ECP-CoPA/CabanaMD, https://github.com/CompPhysVienna/n2p2Licensing provisions: BSD3-Clause, GPL-3.0Programming Language: C++Nature of problem: Developing a performance portable implementation of a neural network potential for exascale architectures.Solution method: CabanaMD-NNP uses algorithms and data-structures from the Kokkos [1] and Cabana [2] libraries to re-implement the computations in Behler-Parrinello neural network potentials [3, 4] for performance portability across hardware. All molecular dynamics data is stored in performance portable data structures, with atomic properties in array-of-structs-of-arrays (Cabana::AoSoAs), and auxiliary values including potential parameters in arrays (Kokkos::Views). All computation is also done in a performance portable way: neural network propagation uses Kokkos parallel kernels (Kokkos::parallel_for), while calculations performed for each atom and neighbor, evaluation of descriptors (symmetry functions) and forces, use Cabana extensions of Kokkos constructs (Cabana::neighbor_parallel_for). These choices provide our implementation with significant speedups both on CPUs and GPUs for large systems, additionally allowing flexibility for parallelism and data layout for further optimizations.Additional comments including restrictions and unusual features: The previously developed n2p2 package [4] contains an interface to LAMMPS [5], which we compare to throughout the paper. We primarily extend the n2p2 library directly (https://github.com/CompPhysVienna/n2p2) and also add an interface to that extension within CabanaMD (https://github.com/ECP-CoPA/CabanaMD), to obtain the main results with an identical LAMMPS input file.

Strong Ties and Weak Ties Rationality: Theory and Scale Development.

Strong Ties and Weak Ties Rationality Scale (STWTRS) is a theory-driven questionnaire designed to capture cultural differences in reasoning about the world. It is intended to demonstrate empirically the heuristic value of the ontological turn that shifts the focus of cultural analysis from the down-stream values, beliefs, and behaviors to the upstream process of thinking and reasoning that is rooted in the local ways of being. This paper will present theory development, preliminary results, and potential contributions of this scale toward better understanding of the culturally different other.

MERF v3.0, a highly computationally efficient non-hydrostatic ocean model with implicit parallelism: Algorithms and validation experiments

Non-hydrostatic ocean models are widely required in coastal and estuarine studies but are limited in their application due to their very high computational expense. To alleviate this issue, this study developed a highly computationally efficient non-hydrostatic ocean model named MERF v3.0 (the third version of the Marine Environment Research and Forecasting model) in which a semi-implicit and variable layers (SIVL) scheme is proposed for dynamic pressure calculation. The SIVL scheme provides freedom to choose the ratio between the number of dynamic pressure layers and the number of velocity layers as required. Moreover, the model adopted numerical calculation framework OpenArray and PETSc (Portable, Extensible Toolkit for Scientific Computation) to achieve implicit parallelism and a concise code structure. Five classical numerical experiments were performed to validate the simulation capacity of this non-hydrostatic ocean model, including the standing wave in a closed basin, surface solitary wave, lock-exchange problem, periodic wave over a bar and tidally induced internal lee wave. The numerical simulation results are consistent with the analytical solutions and experimental data published in the literature. Benefitting from the advantages of OpenArray and PETSc, MERF v3.0 achieves an 87.8% parallel efficiency in the strong scaling test and an 84.1% parallel efficiency in the weak scaling test on 3584 processes, whereas the OpenArray limits the parallel efficiency of MERF v3.0 on more processes, which is beyond the scope of this article and will be further examined in our next study.

Challenges for an axion explanation of the muon g \u2212 2 measurement

The discrepancy between the muon g − 2 measurement and the Standard Model prediction points to new physics around or below the weak scale. It is tantalizing to consider the loop effects of a heavy axion (in the general sense, also known as an axion-like particle) coupling to leptons and photons as an explanation for this discrepancy. We provide an updated analysis of the necessary couplings, including two-loop contributions, and find that the new physics operators point to an axion decay constant on the order of 10s of GeV. This poses major problems for such an explanation, as the axion couplings to leptons and photons must be generated at low scales. We outline some possibilities for how such couplings can arise, and find that these scenarios predict new charged matter at or below the weak scale and new scalars can mix with the Higgs boson, raising numerous phenomenological challenges. These scenarios also all predict additional contributions to the muon g−2 itself, calling the initial application of the axion effective theory into question. We conclude that there is little reason to favor an axion explanation of the muon g – 2 measurement relative to other models postulating new weak-scale matter.

AIPerf: Automated machine learning as an AI-HPC benchmark

The plethora of complex Artificial Intelligence (AI) algorithms and available High-Performance Computing (HPC) power stimulates the expeditious development of AI components with heterogeneous designs. Consequently, the need for cross-stack performance benchmarking of AI-HPC systems has rapidly emerged. In particular, the defacto HPC benchmark, LINPACK, cannot reflect the AI computing power and input/output performance without a representative workload. Current popular AI benchmarks, such as MLPerf, have a fixed problem size and therefore limited scalability. To address these issues, we propose an end-to-end benchmark suite utilizing automated machinelearning, which not only represents real AI scenarios, but also is auto-adaptively scalable to various scales ofmachines. We implement the algorithms in a highly parallel and flexible way to ensure the efficiency and optimizationpotential on diverse systems with customizable configurations. We utilize Operations Per Second (OPS), which ismeasured in an analytical and systematic approach, as a major metric to quantify the AI performance. We performevaluations on various systems to ensure the benchmark's stability and scalability, from 4 nodes with 32 NVIDIA Tesla T4 (56.1 Tera-OPS measured) up to 512 nodes with 4096 Huawei Ascend 910 (194.53 Peta-OPS measured), and the results show near-linear weak scalability. With a flexible workload and single metric, AIPerf can easily scaleon and rank AI-HPC, providing a powerful benchmark suite for the coming supercomputing era.

Verification and Scalability of Mixed-Element USM3D for Benchmark Three-Dimensional Flows

The unstructured-grid mixed-element cell-centered finite-volume flow solver USM3D is enhanced for massively parallel computations. The paper describes modifications to the flow solver and parallelization strategy. The new parallel implementation is verified through comparison of residuals and force coefficients computed on a given grid using varying number of partitions. The accuracy of the parallel USM3D is verified through grid convergence studies and comparison with available reference solutions for two benchmark turbulent flows, namely, a subsonic separated flow around a three-dimensional hemisphere-cylinder configuration and a supersonic flow through a square duct. The strong and weak scalability assessments are conducted using two different nonlinear iteration methods.

An experimental comparison of a space-time multigrid method with PFASST for a reaction-diffusion problem

We consider two parallel-in-time approaches applied to a (reaction) diffusion problem, possibly non-linear. In particular, we consider PFASST (Parallel Full Approximation Scheme in Space and Time) and space-time multigrid strategies. For both approaches, we start from an integral formulation of the continuous time dependent problem. Then, a collocation form for PFASST and a discontinuous Galerkin discretization in time for the space-time multigrid are employed, resulting in the same discrete solution at the time nodes. Strong and weak scaling of both multilevel strategies are compared for varying orders of the temporal discretization. Moreover, we investigate the respective convergence behavior for non-linear problems and highlight quantitative differences in execution times. For the linear problem, we observe that the two methods show similar scaling behavior with PFASST being more favorable for high order methods or when few parallel resources are available. For the non-linear problem, PFASST is more flexible in terms of solution strategy, while space-time multigrid requires a full non-linear solve.

Baryogenesis from the weak scale to the grand unification scale

We review the current status of baryogenesis with emphasis on electroweak baryogenesis and leptogenesis. The first detailed studies were carried out for SU(5) GUT models where CP-violating decays of leptoquarks generate a baryon asymmetry. These GUT models were excluded by the discovery of B+L violating sphaleron processes at high temperatures. Yet a new possibility emerged, electroweak baryogenesis. Here sphaleron processes generate a baryon asymmetry during a strongly first-order phase transition. This mechanism has been studied in many extensions of the Standard Model. However, constraints from the LHC and from low-energy precision experiments exclude most of the known models, leaving composite Higgs models of electroweak symmetry breaking as an interesting possibility. Sphaleron processes are also the basis of leptogenesis, where CP-violating decays of heavy right-handed neutrinos generate a lepton asymmetry which is partially converted to a baryon asymmetry. This mechanism is closely related to the one of GUT baryogenesis, and simple estimates based on GUT models can explain the order of magnitude of the observed baryon-to-photon ratio. In the one-flavour approximation an upper bound on the light neutrino masses has been derived which is consistent with the cosmological upper bound on the sum of neutrino masses. For quasi-degenerate right-handed neutrinos the leptogenesis temperature can be lowered from the GUT scale down to the weak scale, and CP-violating oscillations of GeV sterile neutinos can also lead to successfull leptogenesis. Significant progress has been made in developing a full field theoretical description of thermal leptogenesis, which demonstrated that interactions with gauge bosons of the thermal plasma play a crucial role. Finally, we discuss recent ideas how the seesaw mechanism and B-L breaking at the GUT scale can be probed by gravitational waves.

Multichannel Analysis of Surface Waves Accelerated (MASWAccelerated): Software for efficient surface wave inversion using MPI and GPUs

Multichannel Analysis of Surface Waves (MASW) is a technique frequently used in geotechnical engineering and engineering geophysics to infer 1D layered models of seismic shear wave velocities in the top tens to hundreds of meters of the subsurface. We aim to accelerate MASW calculations by capitalizing on modern computer hardware available in the workstations of most engineers: multiple cores and graphics processing units (GPUs). We propose new parallel and GPU accelerated algorithms for computing 1D MASW inversion, and provide software implementations in C using Message Passing Interface (MPI) and CUDA. These algorithms take advantage of sparsity that arises in the problem, and the work balance between processes considers typical data trends. We compare our methods to an existing open source Matlab MASW tool. Our serial C implementation achieves a 2x speedup over the Matlab software, and we continue to see improvements by parallelizing the problem with MPI. We see nearly perfect strong and weak scaling for uniform data, and improve strong scaling for realistic data by repartitioning the problem to process mapping. By utilizing GPUs available on most modern workstations, we observe an additional 1.3x speedup over the serial C implementation on the first use of the method. We typically repeatedly evaluate theoretical dispersion curves as part of an optimization procedure, and on the GPU the kernel can be cached for faster reuse on later runs. We observe a 3.2x speedup on the cached GPU runs compared to the serial C runs. This work is the first open-source parallel or GPU-accelerated software tool for MASW imaging, and should enable geotechnical engineers to fully utilize all computer hardware at their disposal.

Distribution of supersymmetry μ parameter and Peccei-Quinn scale fa from the landscape

A scan of soft supersymmetry (SUSY) breaking parameters within the string theory landscape with the minimal supersymmetric standard model assumed as the low energy effective field theory---using a power-law draw to large soft terms coupled with an anthropic selection of a derived weak scale to be within a factor of 4 of our measured value---predicts a peak probability of ${m}_{h}\ensuremath{\simeq}125\text{ }\text{ }\mathrm{GeV}$ with sparticle masses typically beyond the reach of LHC Run 2. Such multiverse simulations usually assume a fixed value of the SUSY conserving superpotential $\ensuremath{\mu}$ parameter to be within the assumed anthropic range, $\ensuremath{\mu}\ensuremath{\lesssim}350\text{ }\text{ }\mathrm{GeV}$. However, depending on the assumed solution to the SUSY $\ensuremath{\mu}$ problem, the expected $\ensuremath{\mu}$ term distribution can actually be derived. In this paper, we examine two solutions to the SUSY $\ensuremath{\mu}$ problem. The first is the gravity-safe-Peccei-Quinn model based on an assumed ${\mathbb{Z}}_{24}^{R}$ discrete $R$-symmetry which allows a gravity-safe accidental, approximate Peccei-Quinn global symmetry to emerge which also solves the strong $CP$ problem. The second case is the Giudice-Masiero solution wherein the $\ensuremath{\mu}$ term effectively acts as a soft term and has a linear draw to large values. For the first case, we also present the expected landscape distribution for the Peccei-Quinn scale ${f}_{a}$; in this case, weak scale anthropics limits its range to the cosmological sweet zone of around ${f}_{a}\ensuremath{\sim}{10}^{11}\text{ }\text{ }\mathrm{GeV}$.

Combining EQ-5D-5L items into a level summary score: demonstrating feasibility using non-parametric item response theory using an international dataset

BackgroundThe EQ-5D-5L is a well-established health questionnaire that estimates health utilities by applying preference-based weights. Limited work has been done to examine alternative scoring approaches when utility weights are unavailable or inapplicable. We examined whether the Mokken scaling approach can elucidate 1) if the level summary score is appropriate for the EQ-5D-5L and 2) an interpretation of such a score.MethodsThe R package “mokken” was used to assess monotonicity (scaling coefficients H, automated item selection procedure) and manifest invariant item ordering (MIIO: paired item response functions [IRF], HT). We used a rich dataset (the Multiple Instrument Comparison, MIC) which includes EQ-5D-5L data from six Western countries.ResultsWhile all EQ-5D-5L items demonstrated monotonicity, the anxiety/depression (AD) item had weak scalability (Hi = 0.377). Without AD, scalability improved from Hs = 0.559 to Hs = 0.714. MIIO revealed that the 5 items can be ordered, and the ordering is moderately accurate in the MIC data (HT = 0.463). Excluding AD, HT improves to 0.743. Results were largely consistent across disease and country subgroups.DiscussionThe 5 items of the EQ-5D-5L form a moderate to strong Mokken scale, enabling persons to be ordered using the level summary score. Item ordering suggests that the lower range of the score represents mainly problems with pain and anxiety/depression, the mid-range indicates additional problems with mobility and usual activities, and middle to higher range of scores reveals additional limitations with self-care. Scalability and item ordering are even stronger when the anxiety/depression item is not included in the scale.

PeV gravitino, weak-scale Higgsino, and GeV axino in the Kachru-Kalosh-Linde-Trivedi setup

We realize high scale supersymmetry in the mirage mediation. The Higgs sector is extended with the Peccei-Quinn symmetry, and the higgsino mass term is generated by the Kim-Nilles mechanism. In particular, the Peccei-Quinn symmetry breaking scale naturally lies on the mirage messenger scale due to the mixed modulus-anomaly mediation with the gauge coupling unification. Consequently, the higgsino mass term is of order the weak scale while the gravitino mass is of PeV order. This hierarchy naturally leads to the correct electroweak symmetry breaking. The higgsinos are thus in the range accessible at future lepton colliders, while other sparticles are well-above the current LHC reach and consistent with the observed Higgs boson. The axino is dominantly produced from the modulus decay and accounts for the correct dark matter abundance.

Anomalous muon magnetic moment, supersymmetry, naturalness, LHC search limits and the landscape

The recent measurement of the muon anomalous magnetic moment aμ≡(g−2)μ/2 by the Fermilab Muon g−2 experiment sharpens an earlier discrepancy between theory and the BNL E821 experiment. We examine the predicted Δaμ≡aμ(exp)−aμ(th) in the context of supersymmetry with low electroweak naturalness (restricting to models which give a plausible explanation for the magnitude of the weak scale). A global analysis including LHC Higgs mass and sparticle search limits points to interpretation within the normal scalar mass hierarchy (NSMH) SUSY model wherein first/second generation matter scalars are much lighter than third generation scalars. We present a benchmark model for a viable NSMH point which is natural, obeys LHC Higgs and sparticle mass constraints and explains the muon magnetic anomaly. Aside from NSMH models, then we find the (g−2)μ anomaly cannot be explained within the context of natural SUSY, where a variety of data point to decoupled first/second generation scalars. The situation is worse within the string landscape where first/second generation matter scalars are pulled to values in the 10−50 TeV range. An alternative interpretation for SUSY models with decoupled scalar masses is that perhaps the recent lattice evaluation of the hadronic vacuum polarization could be confirmed which leads to a Standard Model theory-experiment agreement in which case there is no anomaly.