Advanced Scientific Computing Research Research Articles

Perspectives on AI Architectures and Codesign for Earth System Predictability

Abstract Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new paradigm for Earth system predictability focused on enabling artificial intelligence (AI) across the field, laboratory, modeling, and analysis activities, called model experimentation (ModEx). BER’s ModEx is an iterative approach that enables process models to generate hypotheses. The developed hypotheses inform field and laboratory efforts to collect measurement and observation data, which are subsequently used to parameterize, drive, and test model (e.g., process based) predictions. A total of 17 technical sessions were held in this AI4ESP workshop series. This paper discusses the topic of the AI Architectures and Codesign session and associated outcomes. The AI Architectures and Codesign session included two invited talks, two plenary discussion panels, and three breakout rooms that covered specific topics, including 1) DOE high-performance computing (HPC) systems, 2) cloud HPC systems, and 3) edge computing and Internet of Things (IoT). We also provide forward-looking ideas and perspectives on potential research in this codesign area that can be achieved by synergies with the other 16 session topics. These ideas include topics such as 1) reimagining codesign, 2) data acquisition to distribution, 3) heterogeneous HPC solutions for integration of AI/ML and other data analytics like uncertainty quantification with Earth system modeling and simulation, and 4) AI-enabled sensor integration into Earth system measurements and observations. Such perspectives are a distinguishing aspect of this paper. Significance Statement This study aims to provide perspectives on AI architectures and codesign approaches for Earth system predictability. Such visionary perspectives are essential because AI-enabled model-data integration has shown promise in improving predictions associated with climate change, perturbations, and extreme events. Our forward-looking ideas guide what is next in codesign to enhance Earth system models, observations, and theory using state-of-the-art and futuristic computational infrastructure.

Learning Symbolic Expressions: Mixed-Integer Formulations, Cuts, and Heuristics

In this paper, we consider the problem of learning a regression function without assuming its functional form. This problem is referred to as symbolic regression. An expression tree is typically used to represent a solution function, which is determined by assigning operators and operands to the nodes. Cozad and Sahinidis propose a nonconvex mixed-integer nonlinear program (MINLP), in which binary variables are used to assign operators and nonlinear expressions are used to propagate data values through nonlinear operators, such as square, square root, and exponential. We extend this formulation by adding new cuts that improve the solution of this challenging MINLP. We also propose a heuristic that iteratively builds an expression tree by solving a restricted MINLP. We perform computational experiments and compare our approach with a mixed-integer program–based method and a neural network–based method from the literature. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: This work was supported by the Applied Mathematics activity within the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research [Grant DE-AC02-06CH11357]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0050 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0050 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

A Fast Temporal Decomposition Procedure for Long-Horizon Nonlinear Dynamic Programming

We propose a fast temporal decomposition procedure for solving long-horizon nonlinear dynamic programs. The core of the procedure is sequential quadratic programming (SQP) that utilizes a differentiable exact augmented Lagrangian as the merit function. Within each SQP iteration, we approximately solve the Newton system using an overlapping temporal decomposition strategy. We show that the approximate search direction is still a descent direction of the augmented Lagrangian provided the overlap size and penalty parameters are suitably chosen, which allows us to establish the global convergence. Moreover, we show that a unit step size is accepted locally for the approximate search direction and further establish a uniform, local linear convergence over stages. This local convergence rate matches the rate of the recent Schwarz scheme (Na et al. 2022). However, the Schwarz scheme has to solve nonlinear subproblems to optimality in each iteration, whereas we only perform a single Newton step instead. Numerical experiments validate our theories and demonstrate the superiority of our method. Funding: This work was supported by the National Science Foundation [Grant CNS-1545046] and the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research [Grant DE-AC02-06CH11347].

Synergies Between Operations Research and Quantum Information Science

This article highlights synergies between quantum information science (QIS) and operations research for QIS-curious operations researchers (and vice versa), and the author challenges the community to discover more. History: This “Challenge” paper was invited by the Editor in Chief and based on the topics raised by the author at his plenary address at the 2022 INFORMS Computing Society Conference in Tampa, Florida. Funding: This material is based on work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Accelerated Research in Quantum Computing, Fundamental Algorithmic Research for Quantum Computing. This article has been authored by an employee of National Technology & Engineering Solutions of Sandia, LLC, under Contract DE-NA0003525 with the U.S. Department of Energy (DOE).

Engineering the Quantum Scientific Computing Open User Testbed

The Quantum Scientific Computing Open User Testbed (QSCOUT) at Sandia National Laboratories is a trapped-ion qubit system designed to evaluate the potential of near-term quantum hardware in scientific computing applications for the US Department of Energy (DOE) and its Advanced Scientific Computing Research (ASCR) program. Similar to commercially available platforms, most of which are based on superconducting qubits, it offers quantum hardware that researchers can use to perform quantum algorithms, investigate noise properties unique to quantum systems, and test novel ideas that will be useful for larger and more powerful systems in the future. However, unlike most other quantum computing testbeds, QSCOUT uses trapped $^{171}$Yb$^{+}$ ions as the qubits, provides full connectivity between qubits, and allows both quantum circuit and low-level pulse control access to study new modes of programming and optimization. The purpose of this manuscript is to provide users and the general community with details of the QSCOUT hardware and its interface, enabling them to take maximum advantage of its capabilities.

Advancing Fusion with Machine Learning Research Needs Workshop Report

Machine learning and artificial intelligence (ML/AI) methods have been used successfully in recent years to solve problems in many areas, including image recognition, unsupervised and supervised classification, game-playing, system identification and prediction, and autonomous vehicle control. Data-driven machine learning methods have also been applied to fusion energy research for over 2 decades, including significant advances in the areas of disruption prediction, surrogate model generation, and experimental planning. The advent of powerful and dedicated computers specialized for large-scale parallel computation, as well as advances in statistical inference algorithms, have greatly enhanced the capabilities of these computational approaches to extract scientific knowledge and bridge gaps between theoretical models and practical implementations. Large-scale commercial success of various ML/AI applications in recent years, including robotics, industrial processes, online image recognition, financial system prediction, and autonomous vehicles, have further demonstrated the potential for data-driven methods to produce dramatic transformations in many fields. These advances, along with the urgency of need to bridge key gaps in knowledge for design and operation of reactors such as ITER, have driven planned expansion of efforts in ML/AI within the US government and around the world. The Department of Energy (DOE) Office of Science programs in Fusion Energy Sciences (FES) and Advanced Scientific Computing Research (ASCR) have organized several activities to identify best strategies and approaches for applying ML/AI methods to fusion energy research. This paper describes the results of a joint FES/ASCR DOE-sponsored Research Needs Workshop on Advancing Fusion with Machine Learning, held April 30–May 2, 2019, in Gaithersburg, MD (full report available at https://science.osti.gov/-/media/fes/pdf/workshop-reports/FES_ASCR_Machine_Learning_Report.pdf). The workshop drew on broad representation from both FES and ASCR scientific communities, and identified seven Priority Research Opportunities (PRO’s) with high potential for advancing fusion energy. In addition to the PRO topics themselves, the workshop identified research guidelines to maximize the effectiveness of ML/AI methods in fusion energy science, which include focusing on uncertainty quantification, methods for quantifying regions of validity of models and algorithms, and applying highly integrated teams of ML/AI mathematicians, computer scientists, and fusion energy scientists with domain expertise in the relevant areas.

Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources

In January 2019, the US Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions (PRDs) for in situ data management (ISDM). A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery. This article describes six PRDs identified by the workshop, which highlight the components and capabilities needed for ISDM to be successful for a wide variety of applications—making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

Molecular Structure and Ion Transport Near Graphite-Electrolyte Interfaces in Lithium-Ion Batteries

The performance of lithium-ion secondary batteries is strongly tied to ionic transport through electrode-electrolyte interfaces. Ion solvation structures near the interface as well as the chemistry of the electrode surface, in particular the edge termination of graphite electrodes, can impact Li ion transport into and out of the electrode. Understanding these effects is critical to optimizing battery performance. We have performed molecular dynamics simulations of organic liquid electrolytes in contact with graphite electrodes, with and without electric field, to understand the relationship between ion solvation structure and transport near the interface. Prototypical electrolytes consisting of ethylene carbonate (EC) or ethyl methyl carbonate (EMC) organic solvent containing dissolved LiPF6 salt were studied, in contact with armchair-edge oriented graphite electrodes. Various surface terminations of the graphite, including H and OH, were compared. We find that Li+ maintains a well-structured solvation shell even when an electric field is present. The bulkier PF6 – anion does not present a strong solvation structure and exhibits much higher mobility in an electric field. We further observe an energy barrier for Li+ insertion into graphite, particularly when the surface is hydroxyl terminated. Lithium extraction, however, is facile even against an applied electric field. The energetics of Li+ insertion and extraction were analyzed using density functional theory to sample the free energy surfaces. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. Support for this work was provided through Scientific Discovery through Advanced Computing (SciDAC) program funded by U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research and Basic Energy Sciences.

Modest but uneven R&D increases proposed for FY 2013

Uncertainty created by the Budget Control Act of 2011 clouds the outlook for federal science and technologyfunding. Some Department of Energyphysics programs face significant cuts.

Data management and analysis for the Earth System Grid

The international climate community is expected to generate hundreds of petabytes of simulation data within the next five to seven years. This data must be accessed and analyzed by thousands of analysts worldwide in order to provide accurate and timely estimates of the likely impact of climate change on physical, biological, and human systems. Climate change is thus not only a scientific challenge of the first order but also a major technological challenge. In order to address this technological challenge, the Earth System Grid Center for Enabling Technologies (ESG-CET) has been established within the U.S. Department of Energy's Scientific Discovery through Advanced Computing (SciDAC)-2 program, with support from the offices of Advanced Scientific Computing Research and Biological and Environmental Research. ESG-CET's mission is to provide climate researchers worldwide with access to the data, information, models, analysis tools, and computational capabilities required to make sense of enormous climate simulation datasets. Its specific goals are to (1) make data more useful to climate researchers by developing Grid technology that enhances data usability; (2) meet specific distributed database, data access, and data movement needs of national and international climate projects; (3) provide a universal and secure web-based data access portal for broad multi-model data collections; and (4) provide a wide-range of Grid-enabled climate data analysis tools and diagnostic methods to international climate centers and U.S. government agencies. Building on the successes of the previous Earth System Grid (ESG) project, which has enabled thousands of researchers to access tens of terabytes of data from a small number of ESG sites, ESG-CET is working to integrate a far larger number of distributed data providers, high-bandwidth wide-area networks, and remote computers in a highly collaborative problem-solving environment.

Complexity in Scalable Computing

The rich history of scalable computing research owes much to a rapid rise in computing platform scale in terms of size and speed. As platforms evolve, so must algorithms and the software expressions of those algorithms. Unbridled growth in scale inevitably leads to complexity. This special issue grapples with two facets of this complexity: scalable execution and scalable development. The former results from efficient programming of novel hardware with increasing numbers of processing units (e.g., cores, processors, threads or processes). The latter results from efficient development of robust, flexible software with increasing numbers of programming units (e.g., procedures, classes, components or developers). The progression in the above two parenthetical lists goes from the lowest levels of abstraction (hardware) to the highest (people). This issue's theme encompasses this entire spectrum. The lead author of each article resides in the Scalable Computing Research and Development Department at Sandia National Laboratories in Livermore, CA. Their co-authors hail from other parts of Sandia, other national laboratories and academia. Their research sponsors include several programs within the Department of Energy's Office of Advanced Scientific Computing Research and its National Nuclear Security Administration, along with Sandia's Laboratory Directed Research and Development program and the Office ofmore » Naval Research. The breadth of interests of these authors and their customers reflects in the breadth of applications this issue covers. This article demonstrates how to obtain scalable execution on the increasingly dominant high-performance computing platform: a Linux cluster with multicore chips. The authors describe how deep memory hierarchies necessitate reducing communication overhead by using threads to exploit shared register and cache memory. On a matrix-matrix multiplication problem, they achieve up to 96% parallel efficiency with a three-part strategy: intra-node multithreading, non-blocking inter-node message passing, and a dedicated communications thread to facilitate concurrent communications and computations. On a quantum chemistry problem, they spawn multiple computation threads and communication threads on each node and use one-sided communications between nodes to minimize wait times. They reduce software complexity by evolving a multi-threaded factory pattern in C++ from a working, message-passing program in C.« less

Integrated multiscale modeling of molecular computing devices

Molecular electronics, in which single organic molecules are designed to perform the functions of transistors, diodes, switches and other circuit elements used in current siliconbased microelecronics, is drawing wide interest as a potential replacement technology for conventional silicon-based lithographically etched microelectronic devices. In addition to their nanoscopic scale, the additional advantage of molecular electronics devices compared to silicon-based lithographically etched devices is the promise of being able to produce them cheaply on an industrial scale using wet chemistry methods (i.e., self-assembly from solution). The design of molecular electronics devices, and the processes to make them on an industrial scale, will require a thorough theoretical understanding of the molecular and higher level processes involved. Hence, the development of modeling techniques for molecular electronics devices is a high priority from both a basic science point of view (to understand the experimental studies in this field) and from an applied nanotechnology (manufacturing) point of view. Modeling molecular electronics devices requires computational methods at all length scales – electronic structure methods for calculating electron transport through organic molecules bonded to inorganic surfaces, molecular simulation methods for determining the structure of self-assembled films of organic molecules on inorganic surfaces, mesoscale methods to understand and predict the formation of mesoscale patterns on surfaces (including interconnect architecture), and macroscopic scale methods (including finite element methods) for simulating the behavior of molecular electronic circuit elements in a larger integrated device. Here we describe a large Department of Energy project involving six universities and one national laboratory aimed at developing integrated multiscale methods for modeling molecular electronics devices. The project is funded equally by the Office of Basic Energy Sciences and the Office of Advanced Scientific Computing Research, both within the Office of Science of the Department of Energy.

Report of the Fusion Simulation Project Steering Committee

The Fusion Simulation Project (FSP) is envisioned as a 15 year, $20M/year multi-institutional project to develop a comprehensive simulation capability for magnetic fusion experiments with a focus on the International Thermonuclear Experimental Reactor (ITER). The FSP would be able to contribute to design decisions, experimental planning and performance optimization for ITER, substantially increasing ITER’s likelihood of success and its value to the US Fusion Program. The FSP would be jointly supported by the DOE Office of Fusion Energy Sciences and the DOE Office of Advanced Scientific Computing Research. The potential for developing this simulation capability rests on the exponential growth of computer power over the last 50 years, the progress in physics understanding developed by the international fusion program and the continued progress in computational mathematics that enables the use of the new “ultra-scale” computers to solve difficult mathematical problems. The initial concept for the FSP was developed by the Fusion Energy Sciences Advisory Committee Integrated Simulation and Optimization of Fusion Systems Subcommittee (J. Dahlburg and J. Corones, et al., J. Fusion Energy, 20(4), 135–196.). The DOE asked the FSP Steering Committee to develop a project vision, a governance concept and a roadmap for the FSP. The Committee recommends that the FSP consist of three elements: a production component, a research and integration component, and a software infrastructure component. The key challenge is developing components that bridge the enormous distance and time scales involved with the disparate physics elements of tokamak performance. The committee recommended that this challenge be met through “Focused Integration Initiatives” that would first seek to integrate different physics packages with disparate distance and time scales. An example is the integration of Radio Frequency (RF) Current Drive and Magnetohydrodynamics (MHD) components to produce an integrated capability to simulate the use of RF current drive to suppress MHD instabilities. This report also defines the requirements for a governance structure. The FSP Steering Committee judged that the project begin with a conceptual design phase lasting one or two years and be followed by a staged ramp-up over a few years to the full funding level.