Current Supercomputers Research Articles

Born-Oppenheimer Molecular Dynamics with a Linear Combination of Atomic Orbitals and Hybrid Functionals for Condensed Matter Simulations Made Possible. Theory and Performance for the Microcanonical and Canonical Ensembles.

The implementation of an original Born-Oppenheimer molecular dynamics module is presented, which is able to perform simulations of large and complex condensed phase systems for sufficiently long time scales at the level of density functional theory with hybrid functionals, in the microcanonical (NVE) and canonical (NVT) ensembles. The algorithm is fully integrated in the Crystal code, a program for quantum mechanical simulations of materials, whose peculiarity stems from the use of atom-centered basis functions within a linear combination of atomic orbitals to describe the wave function. The corresponding efficiency in the evaluation of the exact Fock exchange series has led to the implementation of a rich variety of hybrid density functionals at a low computational cost. In addition, the molecular dynamics implementation benefits also from the effective MPI parallelization of the code, suited to exploit high-performance computing resources available on current generation supercomputer architectures. Furthermore, the information contained in the trajectory of the dynamics is extracted through a series of postprocessing algorithms that provide the radial distribution function, the diffusion coefficient and the vibrational density of states. In this work, we present a detailed description of the theoretical framework and the algorithmic implementation, followed by a critical evaluation of the accuracy and parallel performance (e.g., strong and weak scaling) of this approach, when ice and liquid water simulations are performed in the microcanonical and canonical ensembles.

Modeling realistic synaptic inputs of CA1 hippocampal pyramidal neurons and interneurons via Adaptive Generalized Leaky Integrate-and-Fire models

Computational models of brain regions are crucial for understanding neuronal network dynamics and the emergence of cognitive functions. However, current supercomputing limitations hinder the implementation of large networks with millions of morphological and biophysical accurate neurons. Consequently, research has focused on simplified spiking neuron models, ranging from the computationally fast Leaky Integrate and Fire (LIF) linear models to more sophisticated non-linear implementations like Adaptive Exponential (AdEX) and Izhikevic models, through Generalized Leaky Integrate and Fire (GLIF) approaches. However, in almost all cases, these models are tuned (and can be validated) only under constant current injections and they may not, in general, also reproduce experimental findings under variable currents. This study introduces an Adaptive GLIF (A-GLIF) approach that addresses this limitation by incorporating a new set of update rules. The extended A-GLIF model successfully reproduces both constant and variable current inputs, and it was validated against the results obtained using a biophysical accurate model neuron. This enhancement provides researchers with a tool to optimize spiking neuron models using classic experimental traces under constant current injections, reliably predicting responses to synaptic inputs, which can be confidently used for large-scale network implementations.

HPC & AI: The challenges and issues related to exascale for numerical modelling and simulation

High-performance digital technology has entered a new era in recent years with the arrival of exascale. Anticipated by experts for more than ten years, exascale is supposed to respond to increasingly varied uses that go far beyond traditional numerical simulation. Data processing and AI are shaking up the HPC landscape and have implications at all levels, from hardware to software, at the scale of applications and at the scale of supercomputers, which are losing their central role in the data-processing chain. In this talk, we will present the current and future supercomputing landscape and we will illustrate with a few examples from different scientific domains (climate, health, energy, …) the issues related to exascale.

Computação quântica: aplicações atuais e potenciais na agricultura digital

Abstract Quantum computers use the properties of quantum physics to perform information storage and processing operations. The operation of these computers involves concepts such as entanglement and superposition, which endow them with a great processing power that even surpasses that of the most powerful current supercomputers, while consuming significantly lower amounts of energy. The different studies analyzed in this review article suggest that quantum computing will have a deep impact in areas such as finance, logistics, transportation, space and automotive technology, materials science, energy, pharmaceutical and healthcare industry, cybersecurity, and agriculture. In digital agriculture, several applications that could be executed more efficiently in quantum computers for data processing and understanding of biological processes were identified and exemplified. These applications are grouped here into the following four areas: bioinformatics, remote sensing, climate modeling, and smart farming. This article also explores the strategic importance of mastering quantum computing, highlights some advantages in relation to classical computing, and presents a mapping of the services already available, enabling institutions to undertake strategic planning for the incorporation of quantum computing into their development processes. Finally, the challenges for the implementation of quantum computing are highlighted, along with some ongoing initiatives aimed at furthering research at the forefront of knowledge in this area applied to digital agriculture.

Tracing task‐based runtime systems: Feedbacks from theStarPUcase

SummaryGiven the complexity of current supercomputers and applications, being able to trace application executions to understand their behavior is not a luxury. As constraints, tracing systems have to be as little intrusive as possible in the application code and performances, and be precise enough in the collected data. In this article, we present how works the tracing system used by the task‐based runtime systemStarPU. We study the different sources of performance overhead coming from the tracing system and how to reduce these overheads. Then, we evaluate the accuracy of distributed traces with different clock synchronization techniques. Finally, we summarize our experiments and conclusions with the lessons we learned to efficiently trace applications, and the list of characteristics each tracing system should feature to be competitive. The reported experiments and implementation details comprise a feedback of integrating into a task‐based runtime system state‐of‐the‐art techniques to efficiently and precisely trace application executions. We highlight the points every application developer or end‐user should be aware of to seamlessly integrate a tracing system or just trace application executions.

Numerical simulations of stochastic inflation using importance sampling

We show how importance sampling can be used to reconstruct the statistics of rare cosmological fluctuations in stochastic inflation. We have developed a publicly available package, PyFPT,[https://github.com/Jacks0nJ/PyFPT.] that solves the first-passage time problem of generic one-dimensional Langevin processes. In the stochastic-δ N formalism, these are related to the curvature perturbation at the end of inflation. We apply this method to quadratic inflation, where the existence of semi-analytical results allows us to benchmark our approach. We find excellent agreement within the estimated statistical error, both in the drift- and diffusion-dominated regimes. The computation takes at most a few hours on a single CPU, and can reach probability values corresponding to less than one Hubble patch per observable universe at the end of inflation. With direct sampling, this would take more than the age of the universe to simulate even with the best current supercomputers. As an application, we study how the presence of large-field boundaries might affect the tail of the probability distribution. We also find that non-perturbative deviations from Gaussianity are not always of the simple exponential type.

The ICON-A model for direct QBO simulations on GPUs (version icon-cscs:baf28a514)

Abstract. Classical numerical models for the global atmosphere, as used for numerical weather forecasting or climate research, have been developed for conventional central processing unit (CPU) architectures. This hinders the employment of such models on current top-performing supercomputers, which achieve their computing power with hybrid architectures, mostly using graphics processing units (GPUs). Thus also scientific applications of such models are restricted to the lesser computer power of CPUs. Here we present the development of a GPU-enabled version of the ICON atmosphere model (ICON-A), motivated by a research project on the quasi-biennial oscillation (QBO), a global-scale wind oscillation in the equatorial stratosphere that depends on a broad spectrum of atmospheric waves, which originates from tropical deep convection. Resolving the relevant scales, from a few kilometers to the size of the globe, is a formidable computational problem, which can only be realized now on top-performing supercomputers. This motivated porting ICON-A, in the specific configuration needed for the research project, in a first step to the GPU architecture of the Piz Daint computer at the Swiss National Supercomputing Centre and in a second step to the JUWELS Booster computer at the Forschungszentrum Jülich. On Piz Daint, the ported code achieves a single-node GPU vs. CPU speedup factor of 6.4 and allows for global experiments at a horizontal resolution of 5 km on 1024 computing nodes with 1 GPU per node with a turnover of 48 simulated days per day. On JUWELS Booster, the more modern hardware in combination with an upgraded code base allows for simulations at the same resolution on 128 computing nodes with 4 GPUs per node and a turnover of 133 simulated days per day. Additionally, the code still remains functional on CPUs, as is demonstrated by additional experiments on the Levante compute system at the German Climate Computing Center. While the application shows good weak scaling over the tested 16-fold increase in grid size and node count, making also higher resolved global simulations possible, the strong scaling on GPUs is relatively poor, which limits the options to increase turnover with more nodes. Initial experiments demonstrate that the ICON-A model can simulate downward-propagating QBO jets, which are driven by wave–mean flow interaction.

Advanced efficient iterative methods to the Helmholtz equation

Parallel computing has recently gained widespread acceptance as a means of handling very large computational data. Since iterative methods are appealing for large systems of equations, and they are the prime candidates for implementations on parallel architectures, We presented based on exploration, through virtual technology having 30 cores, in literature solutions of Helmholtz equation is available up to 12 cores by Jacobi method, here we increased the number of cores and virtual machine having 30 cores first time used to find the solution of Helmholtz equation, our findings are encouraging and found that parallel computing by OpenMP implementations is effective on current supercomputing as well as virtual machine platforms and that is an auspicious programming model to use for applications to be run on emerging and future platforms with accelerated nodes.

Extending an asynchronous runtime system for high throughput applications: A case study

Current supercomputers are mostly composed of vast numbers of nodes enhanced with accelerators (usually in the form of GPUs). However, having these heterogeneous designs in the forefront have exposed the software toolchains and application designers to the underlying complexities of an ever evolving hardware substrate. The need for a more dynamic view from the system software (i.e., compilers and runtimes) has become more apparent in these environments. Due to this, adaptive, fine grain runtime systems have seen a rise in popularity in the past decades. With low overhead and small tasks, these runtimes help to hide long latency operations by exploiting the massive concurrency presented in different application workflows. Such features allow the reduction of idle time (a result from ever deeper and complex memory hierarchies and memory types) with the execution of unrelated work across the machine. Of these runtimes, the Asynchronous Many Task (AMT) Runtimes are excellent exemplars as they can efficiently map onto hardware substrates and exhibit a high degree of latency hiding. Because of their latency tolerant characteristics, applications such as Graph Analytics and Big Data applications (which are latency sensitive) can use these runtimes very efficiently. Thanks to these characteristics, we present how a careful design can help to exploit the properties of an AMT when running high latency applications such as the ones encountered in the Big Data domain. In addition, when combined with introspection / adaptive capabilities, the runtime can further exploit optimization opportunities during its execution based on the ever changing state of the underlying hardware substrate. As a vehicle for this exploration, we use the Performance Open Community Runtime (P-OCR) to test all these concepts with Big Data workloads.

OpenMP application experiences: Porting to accelerated nodes

As recent enhancements to the OpenMP specification become available in its implementations, there is a need to share the results of experimentation in order to better understand the OpenMP implementation’s behavior in practice, to identify pitfalls, and to learn how the implementations can be effectively deployed in scientific codes. We report on experiences gained and practices adopted when using OpenMP to port a variety of ECP applications, mini-apps and libraries based on different computational motifs to accelerator-based leadership-class high-performance supercomputer systems at the United States Department of Energy. Additionally, we identify important challenges and open problems related to the deployment of OpenMP. Through our report of experiences, we find that OpenMP implementations are successful on current supercomputing platforms and that OpenMP is a promising programming model to use for applications to be run on emerging and future platforms with accelerated nodes.

A Review of Supercomputer Performance Monitoring Systems

High Performance Computing is now one of the emerging fields in computer science and its applications. Top HPC facilities, supercomputers, offer great opportunities in modeling diverse processes thus allowing to create more and greater products without full-scale experiments. Current supercomputers and applications for them are very complex and thus are hard to use efficiently. Performance monitoring systems are the tools that help to understand the efficiency of supercomputing applications and overall supercomputer functioning. These systems collect data on what happens on a supercomputer (performance data, performance metrics) and present them in a way allowing to make conclusions about performance issues in programs running on the supercomputer. In this paper we give an overview of existing performance monitoring systems designed for or used on supercomputers. We give a comparison of performance monitoring systems found in literature, describe problems emerging in monitoring large scale HPC systems, and outline our vision on future direction of HPC monitoring systems development.

2D MHD simulation of spontaneous magnetic fields generated during interaction of 1315.2-nm laser radiation with copper slabs at 1016 W/cm2

Multidimensional modeling of phenomena and processes occurring during the expansion of the laser-produced plasma for different irradiation conditions related to both the laser beam parameters and the target constructions is a very complex issue, especially when modeling requires consideration of kinetic processes associated with the development of various types of microscopic instability. Multidimensional PIC codes create such a possibility, but their use is limited to modeling phenomena even in a very narrow timescale due to the limited computational capabilities of current supercomputers. For this reason, the paper attempts to interpret the results of the spontaneous magnetic field (SMF) measurements obtained during the PALS (Prague Asterix Laser System) experiment [Pisarczyk et al., AIP Adv. 10, 115201 (2020); Pisarczyk et al., Phys. Plasmas 22, 102706 (2015)] based on the 2D magneto-hydrodynamic (MHD) model [Jach et al., Computer Modeling of Dynamic Interaction of Bodies by Free Point Method (PWN, Warsaw, 2011)]. The MHD equations were used with included arbitrary (i) current of hot electrons treating it as an additional external current and (ii) ion-sound instability responsible for the increase in anomalous resistance in areas with high temperature and low-density plasma. The spatial distribution of magnetic fields and current density obtained from 2D modeling are in acceptable agreement with the experimental results [Pisarczyk et al., Plasma Phys. Controlled Fusion 62, 115020 (2020); Zaraś-Szydłowska et al., AIP Adv. 10, 115201 (2020); Pisarczyk et al., Phys. Plasmas 22, 102706 (2015)]. The inclusion of temporal changes in anomalous resistance in modeling allowed us to explain the persistence of high SMF amplitude at the level of several megagauss after the laser pulse ended due to the effect of magnetic field freezing.

Predictive Mixing for Density Functional Theory (and Other Fixed-Point Problems)

Density functional theory calculations use a significant fraction of current supercomputing time. The resources required scale with the problem size, the internal workings of the code, and the number of iterations to convergence, with the latter being controlled by what is called "mixing". This paper describes a new approach to handling trust regions within these and other fixed-point problems. Rather than adjusting the trust region based upon improvement, the prior steps are used to estimate what the parameters and trust regions should be, effectively estimating the optimal Polyak step from the prior history. Detailed results are shown for eight structures using both the "good" and "bad" multisecant versions as well as the Anderson method and a hybrid approach, all with the same predictive method. Additional comparisons are made for 36 cases with a fixed algorithm greed. The predictive method works well independent of which method is used for the candidate step, and it is capable of adapting to different problem types particularly when coupled with the hybrid approach.

Parallel Execution on HeterogeneousMultiprocessors from Algorithm Models Based on Petri Nets

Current supercomputers are composed by nodes containing a combination of general purpose computing units (CPUs) and specific mathematical coprocessors. In this way, GPGPUs or Xeon Phi cards are attached to the nodes to improve its performance. Both types of processors, CPUs and coprocessors, have many differences, like their architecture, the clock rate of the processors and the operation of the related memory. These are the main factors that conform an heterogeneous multiprocessor. A parallel program that wants to achieve the sum of the performance of both types of processors, must consider not only the complexity of the parallel algorithm, but also the differences in the architecture of processors, increasing its complexity. As a contribution on this problem, this paper presents a model of parallel execution based on Petri Nets, called PN-PEM, that is used not only to model a parallel algorithm, but also to execute it directly on a computer with heterogeneous multiprocessors. An asynchronous execution of tasks and a dynamic scheduler are the main characteristics that allow execute a parallel program on this type of parallel computer. Tests done on a multicore computer with two Xeon Phi cards reach the aggregate performance of both type of processors, confirming the quality of the model used.

Analysis of Neuromorphic Computing Systems and its Applications in Machine Learning

The domain of engineering has always taken inspiration from the biological world. Understanding the functionalities of the human brain is one of the key areas of interest over time and has caused many advancements in the field of computing systems. The computational capability per unit power per unit volume of the human brain exceeds the current best supercomputers. Mimicking the physics of computations used by the nervous system and the brain can bring a paradigm shift to the computing systems. The concept of bridging computing and neural systems can be termed as neuromorphic computing and it is bringing revolutionary changes in the computing hardware. Neuromorphic computing systems have seen swift progress in the past decades. Many organizations have introduced a variety of designs, implementation methodologies and prototype chips. This paper discusses the parameters that are considered in the advanced neuromorphic computing systems and the tradeoffs between them. There have been attempts made to make computer models of neurons. Advancements in the hardware implementation are fuelling the applications in the field of machine learning. This paper presents the applications of these modern computing systems in Machine Learning.

STARFORGE: Towards a comprehensive numerical model of star cluster formation and feedback

ABSTRACT We present STARFORGE (STAR FORmation in Gaseous Environments): a new numerical framework for 3D radiation magnetohydrodynamic (MHD) simulations of star formation that simultaneously follow the formation, accretion, evolution, and dynamics of individual stars in massive giant molecular clouds (GMCs), while accounting for stellar feedback, including jets, radiative heating and momentum, stellar winds, and supernovae. We use the gizmo code with the MFM mesh-free Lagrangian MHD method, augmented with new algorithms for gravity, time-stepping, sink particle formation and accretion, stellar dynamics, and feedback coupling. We survey a wide range of numerical parameters/prescriptions for sink formation and accretion and find very small variations in star formation history and the IMF (except for intentionally unphysical variations). Modules for mass-injecting feedback (winds, SNe, and jets) inject new gas elements on the fly, eliminating the lack of resolution in diffuse feedback cavities otherwise inherent in Lagrangian methods. The treatment of radiation uses GIZMO’s radiative transfer solver to track five frequency bands (IR, optical, NUV, FUV, ionizing), coupling direct stellar emission and dust emission with gas heating and radiation pressure terms. We demonstrate accurate solutions for SNe, winds, and radiation in problems with known similarity solutions, and show that our jet module is robust to resolution and numerical details, and agrees well with previous AMR simulations. STARFORGE can scale up to massive (>105 M⊙) GMCs on current supercomputers while predicting the stellar (≳0.1 M⊙) range of the IMF, permitting simulations of both high- and low-mass cluster formation in a wide range of conditions.

MFIX-Exa: A path toward exascale CFD-DEM simulations

MFIX-Exa is a computational fluid dynamics–discrete element model (CFD-DEM) code designed to run efficiently on current and next-generation supercomputing architectures. MFIX-Exa combines the CFD-DEM expertise embodied in the MFIX code—which was developed at NETL and is used widely in academia and industry—with the modern software framework, AMReX, developed at LBNL. The fundamental physics models follow those of the original MFIX, but the combination of new algorithmic approaches and a new software infrastructure will enable MFIX-Exa to leverage future exascale machines to optimize the modeling and design of multiphase chemical reactors.

Quantum Calcium-Ion Affective Influences Measured by EEG

Background: Since circa 1980, a model of neocortical interactions, Statistical Mechanics of Neocortical Interactions (SMNI) has been successful in calculating many experimental phenomena, including fits to electroencephalographic (EEG) data in attention tasks, using an importance-sampling code Adaptive Simulated Annealing (ASA). The SMNI model is developed in the context of classical path-integrals, which affords intuitive insights as well as direct numerical benefits, e.g., using the effective Action as a a cost/objective function for parameter fits to data. Objective: Previous authors have fit affective EEG data to neural-network models. This project seeks to use models based on physics and biology to fit this same data. Previous work showed improvements in fits to EEG for attention states; this project extends these methods to affective states. Method: Path integrals are used in both classical and quantum contexts. Classical path integrals are used to define a cost/objective function to fit data, and quantum path integrals are used to derive a closed-form analytic expression for Ca-ion waves in the presence of a magnetic vector potential which is generated by highly synchronous neuronal firings which give rise to EEG. ASA is used to fit EEG data. Results: The mathematical-physics and computer parts of the study are successful, in that cost/objective functions used to fit EEG data using these models are consistent with previous work published by other authors. However, since the SMNI model includes these quantum effects, this is another reason to continue examining these issues. The results here are consistent, not better, than previous work using neural-network models, albeit only one parameter was used here, instead of multiple filters and kernels used previously on such data. Conclusion: Although these quantum effects are highly speculative, explicit calculations have shown them to be consistent with experimental data, at least to date. The current supercomputer project extends this model to affective/emotion data. Results from several authors using neural-network approaches at individual electrode sites show some predictive capabilities; the results given here are consistent with these other results. However, since the SMNI model includes these quantum effects, this is another reason to continue examining these issues.

Nascent exascale supercomputers offer promise, present challenges

Sometime next year, managers at the US Department of Energy’s (DOE) Argonne National Laboratory in Lemont, IL, will power up a calculating machine the size of 10 tennis courts and vault the country into a new age of computing. The $500-million mainframe, called Aurora, could become the world’s first “exascale” supercomputer, running an astounding 1018, or 1 quintillion, operations per second. Rows of cabinets hold incredible processing power for one of the world's best supercomputers, Summit, at Oak Ridge National Laboratory in TN. Exascale computing will surpass these existing computers by leaps and bounds. Image credit: Flickr/Oak Ridge National Laboratory, licensed under CC BY 2.0. Aurora is expected to have more than twice the peak performance of the current supercomputer record holder, a machine named Fugaku at the RIKEN Center for Computational Science in Kobe, Japan. Fugaku and its calculation kin serve a vital function in modern scientific advancement, performing simulations crucial for discoveries in a wide range of fields. But the transition to exascale will not be easy. “As these machines grow, they become harder and harder to exploit efficiently,” says Danny Perez, a physicist at Los Alamos National Laboratory in NM. “We have to change our computing paradigms, how we write our programs, and how we arrange computation and data management.” That’s because supercomputers are complex beasts, consisting of cabinets containing hundreds of thousands of processors. For these processors to operate as a single entity, a supercomputer needs to pass data back and forth between its various parts, running huge numbers of computations at the same time, all while minimizing power consumption. Writing programs for such parallel computing is not easy, and theorists will need to leverage new tools such as machine learning and artificial intelligence to make scientific breakthroughs. Given these challenges, researchers have …

WaLBerla: A block-structured high-performance framework for multiphysics simulations

Programming current supercomputers efficiently is a challenging task. Multiple levels of parallelism on the core, on the compute node, and between nodes need to be exploited to make full use of the system. Heterogeneous hardware architectures with accelerators further complicate the development process. waLBerla addresses these challenges by providing the user with highly efficient building blocks for developing simulations on block-structured grids. The block-structured domain partitioning is flexible enough to handle complex geometries, while the structured grid within each block allows for highly efficient implementations of stencil-based algorithms. We present several example applications realized with waLBerla, ranging from lattice Boltzmann methods to rigid particle simulations. Most importantly, these methods can be coupled together, enabling multiphysics simulations. The framework uses meta-programming techniques to generate highly efficient code for CPUs and GPUs from a symbolic method formulation.