Mira Supercomputer Research Articles

Improving batch schedulers with node stealing for failed jobs

SummaryAfter a machine failure, batch schedulers typically re‐schedule the job that failed with a high priority. This is fair for the failed job but still requires that job to re‐enter the submission queue and to wait for enough resources to become available. The waiting time can be very long when the job is large and the platform highly loaded, as is the case with typical HPC platforms. We propose another strategy: when a job fails, if no platform node is available, we steal one node from another job , and use it to continue the execution of despite the failure. In this work, we give a detailed assessment of this node stealing strategy using traces from the Mira supercomputer at Argonne National Laboratory. The main conclusion is that node stealing improves the utilization of the platform and dramatically reduces the flow of large jobs, at the price of slightly increasing the flow of small jobs.

Resilient Scheduling Heuristics for Rigid Parallel Jobs

This paper focuses on the resilient scheduling of parallel jobs on high-performance computing (HPC) platforms to minimize the overall completion time, or the makespan. We revisit the classical problem while assuming that jobs are subject to failures caused by transient or silent errors, and hence may need to be re-executed each time they fail to complete successfully. This work generalizes the classical framework where jobs are known offline and do not fail: in this framework, list scheduling that gives priority to the longest jobs is known to be a 3-approximation when imposing to use shelves, and a 2-approximation without this restriction. We show that when jobs can fail, using shelves can be arbitrarily bad, but unrestricted list scheduling remains a 2-approximation. The paper focuses on the design of several heuristics, some list-based and some shelf-based, along with different priority rules and backfilling strategies. We assess and compare their performance through an extensive set of simulations using both synthetic jobs and log traces from the Mira supercomputer.

HPC Opens a New Frontier in Fuel-Engine Research

An industry-led research team at the US Department of Energys Argonne National Laboratory recently conducted a computationally guided combustion system optimization on the IBM Blue Gene/Q Mira supercomputer. The team used a high-fidelity simulation approach to optimize the fuel spray and combustion bowl geometry of a heavy-duty diesel engine using a gasoline-like fuel. The accelerated simulation time allowed the team to evaluate an unprecedented number of design variations and improve the production design using a new fuel.

Numerical modeling of laser-driven experiments aiming to demonstrate magnetic field amplification via turbulent dynamo

The universe is permeated by magnetic fields, with strengths ranging from a femtogauss in the voids between the filaments of galaxy clusters to several teragauss in black holes and neutron stars. The standard model behind cosmological magnetic fields is the nonlinear amplification of seed fields via turbulent dynamo to the values observed. We have conceived experiments that aim to demonstrate and study the turbulent dynamo mechanism in the laboratory. Here, we describe the design of these experiments through simulation campaigns using FLASH, a highly capable radiation magnetohydrodynamics code that we have developed, and large-scale three-dimensional simulations on the Mira supercomputer at the Argonne National Laboratory. The simulation results indicate that the experimental platform may be capable of reaching a turbulent plasma state and determining the dynamo amplification. We validate and compare our numerical results with a small subset of experimental data using synthetic diagnostics.

Argonne Discovery Yields Self-Healing Diamond-Like Carbon

Large-scale reactive molecular dynamics simulations carried out on the US Department of Energy's IBM Blue Gene/Q Mira supercomputer at the Argonne Leadership Computing Facility, along with experiments conducted by researchers in Argonne's Energy Systems Division, enabled the design of a self-healing anti-wear coating that drastically reduces friction and related degradation in engines and moving machinery. Now, the computational work advanced for this purpose is being used to identify the friction-fighting potential of other catalysts.

Optimising the Termofluids CFD code for petascale simulations

ABSTRACTThis paper presents some recent efforts carried out on the expansion of the scalability of TermoFluids multi-physics Computational Fluid Dynamics (CFD) code, aiming to achieve petascale capacity for a single simulation. We describe different aspects that we have improved in our code in order to efficiently run it on 131,072 CPU-cores. This work has been developed using the BlueGene/Q Mira supercomputer of the Argonne Leadership Computing Facility, where we have obtained feedback at the targeted scale. In summary, this is a practical paper showing our experience at reaching the petascale paradigm for a single simulation with TermoFluids.

Delivering Science on Day One

While benchmarking software is useful for testing the performance limits and stability of Argonne National Laboratory's new Theta supercomputer, there's no substitute for running real applications to explore the system's potential. The Argonne Leadership Computing Facility's Theta Early Science Program, modeled after its highly successful code migration program for the Mira supercomputer, has one primary aim: to deliver science on day one. This article offers a closer look at the type of science problems that will get early access to Theta, a next-generation machine being rolled out this year.

Asynchronous Two-level Checkpointing Scheme for Large-scale Adjoints in the Spectral-Element Solver Nek5000

Adjoints are an important computational tool for large-scale sensitivity evaluation, uncertainty quantification, and derivative-based optimization. An essential component of their performance is the storage/recomputation balance in which efficient adjoint checkpointing strategies play a key role. We introduce a novel asynchronous two-level adjoint checkpointing scheme for numerical time discretizations targeted at large-scale numerical simulations. The checkpointing scheme combines bandwidth-limited disk checkpointing and space-limited binomial memory checkpointing. Based on assumptions about the target petascale systems, which we later demonstrate to be realistic on the IBM Blue Gene/Q system Mira, we create a model of the predicted performance of the adjoint computation and validate it using the highly scalable Navier-Stokes spectral-element solver Nek5000 on small to moderate subsystems of the Mira supercomputer.