High-performance Scientific Computing Research Articles

Quadratic Optimization Models for Balancing Preferential Access and Fairness: Formulations and Optimality Conditions

Typically, within facility location problems, fairness is defined in terms of accessibility of users. However, for facilities perceived as undesirable by communities hosting them, fairness between the usage of facilities becomes especially important. Limited research exists on this notion of fairness. To close this gap, we develop a series of optimization models for the allocation of populations of users to facilities such that access for users is balanced with a fair utilization of facilities. The optimality conditions of the underlying nonconvex quadratic models state the precise balance between accessibility and fairness. We define new classes of fairness and a metric to quantify the extent to which fairness is achieved in both optimal and suboptimal allocations. We show that a continuous relaxation of our central model is sufficient to achieve a perfect extent of fairness, while a special case reduces to the classical notion of proportional fairness. Our work is motivated by pervasive ecological challenges faced by the waste management community as policymakers seek to reduce the number of recycling centers in the last few years. As a computational case study, applying our models on data for the state of Bavaria in Germany, we find that even after the closure of a moderate number of recycling centers, large degrees of access can be ensured, provided that the closures are conducted optimally. Fairness, however, is impacted more, with facilities in rural regions shouldering larger loads of visiting populations than those in urban regions. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: Computer resources and support provided by the Erlangen Regional Computing Center are gratefully acknowledged. B. Singh was partially financially supported by the Bavarian State Ministry for Science and Art (Bayerisches Staatsministerium für Wissenschaft und Kunst) under the Competence Network for Scientific High Performance Computing in Bavaria. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0308 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0308 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Refining DIIS algorithms for Si and GaAs solar cells: incorporation of weight regularization, conjugate gradient, and reverse automatic differentiation techniques.

Pivotal enhancements in electronic minimization algorithms, which are vital for the advancement of computational materials science, are introduced in this research. Our research is dedicated to refining the DIIS algorithm specifically for electronic structure calculations of silicon (Si) and gallium arsenide (GaAs) solar cells, aiming to enhance their efficiency and stability. We have enriched DIIS by integrating a weight regularization factor, significantly bolstering its convergence stability. This modification enhances iteration robustness and curtails the average iteration duration, thus streamlining the convergence process. Furthermore, we have incorporated the conjugate gradient (CG) algorithm to proficiently resolve symmetric positive definite residual matrices. This inclusion substantially accelerates the solution-finding process within the DIIS framework. A novel aspect of our research is the application of reverse automatic differentiation (AD), deployed in two distinct methodologies: the construction of the Jacobian matrix and direct chain rule application for gradient computation. These approaches involve sophisticated mathematical techniques that enhance computational precision and efficiency specifically for Si and GaAs solar cell materials in determining the optimal weights for residual combinations during DIIS iterations. The integration of these advanced methods into the DIIS algorithm not only augments its convergence stability but also ensures a substantial reduction in total computational time. Our findings demonstrate that the enhanced DIIS, CG-enhanced DIIS, and AD-integrated DIIS methods collectively lead to a more efficient electronic minimization process. This balance of stability and efficiency is crucial in high-performance computational materials science, particularly for complex systems analysis. The findings of this research represent a notable advancement in computational strategies for Si and GaAs solar cell materials, providing enhanced methodologies and insights that significantly improve the efficiency and stability of electronic structure calculations in these critical components of renewable energy technologies.

A Review on High Performance Computing and Computational Science

(HPC) is an approach that combines computing power to solve complex High-performance computing problems in science, engineering and business, and provides far greater productivity than a typical desktop or workstation to solve the problem of computing encourage Hardware, software programs, algorithms, languages, architectures etc. Build solutions on a large scale . Computer Science and Engineering (CSE) is a discipline that develops and applies computer modeling and simulation to solve complex physical problems arising from engineering research, often in conjunction with high performance computing, and deals with structural and physics descriptions

Comparing the Performance of Julia on CPUs versus GPUs and Julia-MPI versus Fortran-MPI: a case study with MPAS-Ocean (Version 7.1)

Abstract. Some programming languages are easy to develop at the cost of slow execution, while others are fast at runtime but much more difficult to write. Julia is a programming language that aims to be the best of both worlds – a development and production language at the same time. To test Julia's utility in scientific high-performance computing (HPC), we built an unstructured-mesh shallow water model in Julia and compared it against an established Fortran-MPI ocean model, the Model for Prediction Across Scales–Ocean (MPAS-Ocean), as well as a Python shallow water code. Three versions of the Julia shallow water code were created: for single-core CPU, graphics processing unit (GPU), and Message Passing Interface (MPI) CPU clusters. Comparing identical simulations revealed that our first version of the Julia model was 13 times faster than Python using NumPy, where both used an unthreaded single-core CPU. Further Julia optimizations, including static typing and removing implicit memory allocations, provided an additional 10–20× speed-up of the single-core CPU Julia model. The GPU-accelerated Julia code was almost identical in terms of performance to the MPI parallelized code on 64 processes, an unexpected result for such different architectures. Parallelized Julia-MPI performance was identical to Fortran-MPI MPAS-Ocean for low processor counts and ranges from 2× faster to 2× slower for higher processor counts. Our experience is that Julia development is fast and convenient for prototyping but that Julia requires further investment and expertise to be competitive with compiled codes. We provide advice on Julia code optimization for HPC systems.

Assessing Saiph, a task-based DSL for high-performance computational fluid dynamics

Scientific applications face the challenge of efficiently exploiting increasingly complex parallel and distributed systems. Developing hand-tuned codes is a time-consuming, tedious and hardly reusable task. Reaching high performance appears detrimental to productivity and portability and unreasonable to expect from scientists. Domain-Specific Languages (DSLs) are collaborative environments aiming to overcome such difficulties by decoupling the problem description from the algorithmic implementation. However, developing a competitive tool in High-Performance Computing (HPC) is challenging: DSLs for HPC environments have two additional critical requirements, performance and scalability. Moreover, documented and successful cases are few, making it difficult to popularise DSLs as problem-solving environments for scientific HPC code development. In this context, Saiph is a task-based DSL easing the simulation of physical phenomena from Computational Fluid Dynamics (CFD), developed to meet HPC productivity and performance requirements. This work reports the tuning and evaluation of Saiph using the Taylor–Green Vortex (TGV) problem as a case study. We assess Saiph’s productivity, numerical methods, and high-performance strategies to illustrate its use and demonstrate its competitiveness, viability and benefits for CFD software developments in HPC environments. Hence, we contribute to the popularisation of HPC DSLs as suitable problem-solving environments able to unify modern computational and scientific knowledge.

Sampling rare conformational transitions with a quantum computer

Structural rearrangements play a central role in the organization and function of complex biomolecular systems. In principle, Molecular Dynamics (MD) simulations enable us to investigate these thermally activated processes with an atomic level of resolution. In practice, an exponentially large fraction of computational resources must be invested to simulate thermal fluctuations in metastable states. Path sampling methods focus the computational power on sampling the rare transitions between states. One of their outstanding limitations is to efficiently generate paths that visit significantly different regions of the conformational space. To overcome this issue, we introduce a new algorithm for MD simulations that integrates machine learning and quantum computing. First, using functional integral methods, we derive a rigorous low-resolution spatially coarse-grained representation of the system’s dynamics, based on a small set of molecular configurations explored with machine learning. Then, we use a quantum annealer to sample the transition paths of this low-resolution theory. We provide a proof-of-concept application by simulating a benchmark conformational transition with all-atom resolution on the D-Wave quantum computer. By exploiting the unique features of quantum annealing, we generate uncorrelated trajectories at every iteration, thus addressing one of the challenges of path sampling. Once larger quantum machines will be available, the interplay between quantum and classical resources may emerge as a new paradigm of high-performance scientific computing. In this work, we provide a platform to implement this integrated scheme in the field of molecular simulations.

Innovation of Tin Oxide Ceramic Manufacturing Process Based on WSN and Remote Visualization Technology.

A wireless sensor network (WSN) collects information from small sensors, processes the information effectively through a built-in computing system, forms a dynamic network topology in a self-organizing method, and sends it to the base station through wireless communication. At the same time, the scale of data generated by high-performance scientific computing has rapidly increased from GB and TB to EB. The traditional scientific vision postprocessing model that initially saved the data and then performed the vision processing became a very time-consuming and troublesome process due to the limitation of disk I/O efficiency, which seriously affected the development of technology, and the accuracy was relatively low. The development of high-performance computer parallel processing technology has promoted parallel visualization, and the expansion of Internet bandwidth has accelerated remote visualization. SnO2, or tin oxide, is a kind of N-type semiconductor material with a rutile structure. The crystal structure is stable, the corrosion resistance is good, the melting point is high, the resistance after doping is low, and the sinterability is good. Materials that are sensitive to optics, electrodes, and gases have a wide range of uses. This paper proposes a new data fusion algorithm based on BP neural network and studies the key technologies for the design and implementation of a web-based multiuser remote interactive vision system. In this paper, the effects of ZnO, MnO2, Nb2O5, CuO, and Li2CO3 doping on the conductivity and bulk density of SnO2 semiconductor ceramics were studied by means of ceramic preparation method, using intelligent resistance meter, bulk density tester, X-ray diffractometer, and scanning electron microscope, thus promoting the innovation of tin oxide ceramic preparation process.

Ginkgo : A Modern Linear Operator Algebra Framework for High Performance Computing

In this article, we present Ginkgo , a modern C++ math library for scientific high performance computing. While classical linear algebra libraries act on matrix and vector objects, Ginkgo ’s design principle abstracts all functionality as “linear operators,” motivating the notation of a “linear operator algebra library.” Ginkgo ’s current focus is oriented toward providing sparse linear algebra functionality for high performance graphics processing unit (GPU) architectures, but given the library design, this focus can be easily extended to accommodate other algorithms and hardware architectures. We introduce this sophisticated software architecture that separates core algorithms from architecture-specific backends and provide details on extensibility and sustainability measures. We also demonstrate Ginkgo ’s usability by providing examples on how to use its functionality inside the MFEM and deal.ii finite element ecosystems. Finally, we offer a practical demonstration of Ginkgo ’s high performance on state-of-the-art GPU architectures.

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. In this paper, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for O(N) task execution and O(NlnN) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

A TensorFlow simulation framework for scientific computing of fluid flows on tensor processing units

A computational fluid dynamics (CFD) simulation framework for fluid-flow prediction is developed on the Tensor Processing Unit (TPU) platform. The TPU architecture is featured with accelerated dense matrix multiplication, large high bandwidth memory, and a fast inter-chip interconnect, making it attractive for high-performance scientific computing. The CFD framework solves the variable-density Navier-Stokes equation using a low-Mach approximation, and the governing equations are discretized by a finite-difference method on a collocated structured mesh. It uses the graph-based TensorFlow as the programming paradigm. The accuracy and performance of this framework is studied both numerically and analytically, specifically focusing on effects of TPU-native single precision floating point arithmetic. The algorithm and implementation are validated with canonical 2D and 3D Taylor-Green vortex simulations. To demonstrate the capability for simulating turbulent flows, simulations are conducted for two configurations, namely decaying homogeneous isotropic turbulence and a turbulent planar jet. Both simulations show good statistical agreement with reference solutions. The performance analysis shows a linear weak scaling and a superlinear strong scaling up to a full TPU v3 pod with 2048 cores.

Accelerating Scientific Computing in the Post-Moore’s Era

Novel uses of graphical processing units for accelerated computation revolutionized the field of high-performance scientific computing by providing specialized workflows tailored to algorithmic requirements. As the era of Moore’s law draws to a close, many new non–von Neumann processors are emerging as potential computational accelerators, including those based on the principles of neuromorphic computing, tensor algebra, and quantum information. While development of these new processors is continuing to mature, the potential impact on accelerated computing is anticipated to be profound. We discuss how different processing models can advance computing in key scientific paradigms: machine learning and constraint satisfaction. Significantly, each of these new processor types utilizes a fundamentally different model of computation, and this raises questions about how to best use such processors in the design and implementation of applications. While many processors are being developed with a specific domain target, the ubiquity of spin-glass models and neural networks provides an avenue for multi-functional applications. This also hints at the infrastructure needed to integrate next-generation processing units into future high-performance computing systems.

High-performance sampling of generic determinantal point processes.

Determinantal point processes (DPPs) were introduced by Macchi (Macchi 1975 Adv. Appl. Probab. 7, 83-122) as a model for repulsive (fermionic) particle distributions. But their recent popularization is largely due to their usefulness for encouraging diversity in the final stage of a recommender system (Kulesza & Taskar 2012 Found. Trends Mach. Learn. 5, 123-286). The standard sampling scheme for finite DPPs is a spectral decomposition followed by an equivalent of a randomly diagonally pivoted Cholesky factorization of an orthogonal projection, which is only applicable to Hermitian kernels and has an expensive set-up cost. Researchers Launay et al. 2018 (http://arxiv.org/abs/1802.08429); Chen & Zhang 2018 NeurIPS (https://papers.nips.cc/paper/7805-fast-greedy-map-inference-for-determinantal-point-process-to-improve-recommendation-diversity.pdf) have begun to connect DPP sampling to LDLH factorizations as a means of avoiding the initial spectral decomposition, but existing approaches have only outperformed the spectral decomposition approach in special circumstances, where the number of kept modes is a small percentage of the ground set size. This article proves that trivial modifications of LU and LDLH factorizations yield efficient direct sampling schemes for non-Hermitian and Hermitian DPP kernels, respectively. Furthermore, it is experimentally shown that even dynamically scheduled, shared-memory parallelizations of high-performance dense and sparse-direct factorizations can be trivially modified to yield DPP sampling schemes with essentially identical performance. The software developed as part of this research, Catamari (hodgestar.com/catamari) is released under the Mozilla Public License v.2.0. It contains header-only, C++14 plus OpenMP 4.0 implementations of dense and sparse-direct, Hermitian and non-Hermitian DPP samplers. This article is part of a discussion meeting issue 'Numerical algorithms for high-performance computational science'.

Big telescope, big data: towards exascale with the Square Kilometre Array.

Unlike optical telescopes, radio interferometers do not image the sky directly but require specialized image formation algorithms. For the Square Kilometre Array (SKA), the computational requirements of this image formation are extremely demanding due to the huge data rates produced by the telescope. This processing will be performed by the SKA Science Data Processor facilities and a network of SKA Regional Centres, which must not only deal with SKA-scale data volumes but also with stringent science-driven image fidelity requirements.This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

The parallelism motifs of genomic data analysis.

Genomic datasets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share these data with the research community, but some of these genomic data analysis problems require large-scale computational platforms to meet both the memory and computational requirements. These applications differ from scientific simulations that dominate the workload on high-end parallel systems today and place different requirements on programming support, software libraries and parallel architectural design. For example, they involve irregular communication patterns such as asynchronous updates to shared data structures. We consider several problems in high-performance genomics analysis, including alignment, profiling, clustering and assembly for both single genomes and metagenomes. We identify some of the common computational patterns or ‘motifs’ that help inform parallelization strategies and compare our motifs to some of the established lists, arguing that at least two key patterns, sorting and hashing, are missing.This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

The physics of numerical analysis: a climate modelling case study.

The case is made for a much closer synergy between climate science, numerical analysis and computer science. This article is part of a discussion meeting issue 'Numerical algorithms for high-performance computational science'.

A survey of algorithms for transforming molecular dynamics data into metadata for in situ analytics based on machine learning methods.

This paper presents the survey of three algorithms to transform atomic-level molecular snapshots from molecular dynamics (MD) simulations into metadata representations that are suitable for in situ analytics based on machine learning methods. MD simulations studying the classical time evolution of a molecular system at atomic resolution are widely recognized in the fields of chemistry, material sciences, molecular biology and drug design; these simulations are one of the most common simulations on supercomputers. Next-generation supercomputers will have a dramatically higher performance than current systems, generating more data that needs to be analysed (e.g. in terms of number and length of MD trajectories). In the future, the coordination of data generation and analysis can no longer rely on manual, centralized analysis traditionally performed after the simulation is completed or on current data representations that have been defined for traditional visualization tools. Powerful data preparation phases (i.e. phases in which original row data is transformed to concise and still meaningful representations) will need to proceed data analysis phases. Here, we discuss three algorithms for transforming traditionally used molecular representations into concise and meaningful metadata representations. The transformations can be performed locally. The new metadata can be fed into machine learning methods for runtime in situ analysis of larger MD trajectories supported by high-performance computing. In this paper, we provide an overview of the three algorithms and their use for three different applications: protein-ligand docking in drug design; protein folding simulations; and protein engineering based on analytics of protein functions depending on proteins' three-dimensional structures. This article is part of a discussion meeting issue 'Numerical algorithms for high-performance computational science'.

The future of computing beyond Moore’s Law

Moore's Law is a techno-economic model that has enabled the information technology industry to double the performance and functionality of digital electronics roughly every 2 years within a fixed cost, power and area. Advances in silicon lithography have enabled this exponential miniaturization of electronics, but, as transistors reach atomic scale and fabrication costs continue to rise, the classical technological driver that has underpinned Moore's Law for 50 years is failing and is anticipated to flatten by 2025. This article provides an updated view of what a post-exascale system will look like and the challenges ahead, based on our most recent understanding of technology roadmaps. It also discusses the tapering of historical improvements, and how it affects options available to continue scaling of successors to the first exascale machine. Lastly, this article covers the many different opportunities and strategies available to continue computing performance improvements in the absence of historical technology drivers. This article is part of a discussion meeting issue 'Numerical algorithms for high-performance computational science'.

Exascale applications: skin in the game.

As noted in Wikipedia, skin in the game refers to having ‘incurred risk by being involved in achieving a goal’, where ‘skin is a synecdoche for the person involved, and game is the metaphor for actions on the field of play under discussion’. For exascale applications under development in the US Department of Energy Exascale Computing Project, nothing could be more apt, with the skin being exascale applications and the game being delivering comprehensive science-based computational applications that effectively exploit exascale high-performance computing technologies to provide breakthrough modelling and simulation and data science solutions. These solutions will yield high-confidence insights and answers to the most critical problems and challenges for the USA in scientific discovery, national security, energy assurance, economic competitiveness and advanced healthcare.This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

Rethinking arithmetic for deep neural networks.

We consider efficiency in the implementation of deep neural networks. Hardware accelerators are gaining interest as machine learning becomes one of the drivers of high-performance computing. In these accelerators, the directed graph describing a neural network can be implemented as a directed graph describing a Boolean circuit. We make this observation precise, leading naturally to an understanding of practical neural networks as discrete functions, and show that the so-called binarized neural networks are functionally complete. In general, our results suggest that it is valuable to consider Boolean circuits as neural networks, leading to the question of which circuit topologies are promising. We argue that continuity is central to generalization in learning, explore the interaction between data coding, network topology, and node functionality for continuity and pose some open questions for future research. As a first step to bridging the gap between continuous and Boolean views of neural network accelerators, we present some recent results from our work on LUTNet, a novel Field-Programmable Gate Array inference approach. Finally, we conclude with additional possible fruitful avenues for research bridging the continuous and discrete views of neural networks.This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

Machine learning and big scientific data.

This paper reviews some of the challenges posed by the huge growth of experimental data generated by the new generation of large-scale experiments at UK national facilities at the Rutherford Appleton Laboratory (RAL) site at Harwell near Oxford. Such ‘Big Scientific Data’ comes from the Diamond Light Source and Electron Microscopy Facilities, the ISIS Neutron and Muon Facility and the UK's Central Laser Facility. Increasingly, scientists are now required to use advanced machine learning and other AI technologies both to automate parts of the data pipeline and to help find new scientific discoveries in the analysis of their data. For commercially important applications, such as object recognition, natural language processing and automatic translation, deep learning has made dramatic breakthroughs. Google's DeepMind has now used the deep learning technology to develop their AlphaFold tool to make predictions for protein folding. Remarkably, it has been able to achieve some spectacular results for this specific scientific problem. Can deep learning be similarly transformative for other scientific problems? After a brief review of some initial applications of machine learning at the RAL, we focus on challenges and opportunities for AI in advancing materials science. Finally, we discuss the importance of developing some realistic machine learning benchmarks using Big Scientific Data coming from several different scientific domains. We conclude with some initial examples of our ‘scientific machine learning’ benchmark suite and of the research challenges these benchmarks will enable.This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.