High Performance Computing Centers Research Articles

Accelerating HPC With Quantum Computing: It Is a Software Challenge Too

With quantum computing (QC) maturing, high-performance computing (HPC) centers are already preparing to host early-phase production versions of such systems. Unlike their experimental predecessors in physics laboratories, with a very small and dedicated user community, this next generation of systems needs to serve a wider user community and must work in concert with existing HPC systems and software stacks. This article describes our vision for an integrated ecosystem that combines existing HPC and evolving quantum software stacks into a single system to enable a common and continuous user experience. This integration comes with several major challenges as quantum systems pose significantly different requirements including increased need for compilation at run time, long optimization times, statistical evaluations of results, and the need to work with few centralized resources. To overcome these challenges, new scheduling approaches on the HPC side and new programming approaches on the QC side are required.

Burner: Recipe Automatic Generation for HPC Container Based on Domain Knowledge Graph

As one of the emerging cloud computing technologies, containers are widely used in academia and industry. The cloud computing built by the container in the high performance computing (HPC) center can provide high-quality services to users at the edge. Singularity Definition File and Dockerfile (we refer to such files as recipes) have attracted wide attention due to their encapsulation of the application running environment in a container. However, creating a recipe requires extensive domain knowledge, which is error-prone and time-consuming. Accordingly, more than 34% of Dockerfiles in Github cannot successfully build container images. The crucial points about recipe creation include selecting the entities (base images and packages) and determining their relationships (correct installation order for transitive dependencies). Since the relationships between entities can be expressed accurately and efficiently by the knowledge graph, we introduce knowledge graph to generate high-quality recipes automatically. This paper proposes an automatic recipe generation system named Burner, enabling users with no professional computer background to generate the recipes. We first develop a toolset including a recipe parser and an entity-relationship miner. Our two-phase recipe parsing method can perform abstract syntax tree (AST) parsing more deeply on the recipe file to achieve entity extraction; the parsing success rate (PSR) of the two-phase parsing method is 10.1% higher than the one-phase parsing. Then, we build a knowledge base containing 2,832 entities and 62,614 entity relationships, meeting the needs of typical HPC applications. In the test of image build, the singularity image build success rate reaches 80%. Compared with the ItemCF recommendation method, our recommendation method TB-TFIDF achieves a performance improvement by up to 50.86%.

Performance trade-offs in reconfigurable networks for HPC

Designing efficient interconnects to support high-bandwidth and low-latency communication is critical toward realizing high performance computing (HPC) and data center (DC) systems in the exascale era. At extreme computing scales, providing the requisite bandwidth through overprovisioning becomes impractical. These challenges have motivated studies exploring reconfigurable network architectures that can adapt to traffic patterns at runtime using optical circuit switching. Despite the plethora of proposed architectures, surprisingly little is known about the relative performances and trade-offs among different reconfigurable network designs. We aim to bridge this gap by tackling two key issues in reconfigurable network design. First, we study how cost, power consumption, network performance, and scalability vary based on optical circuit switch (OCS) placement in the physical topology. Specifically, we consider two classes of reconfigurable architectures: one that places OCSs between top-of-rack (ToR) switches—ToR-reconfigurable networks (TRNs)—and one that places OCSs between pods of racks—pod-reconfigurable networks (PRNs). Second, we tackle the effects of reconfiguration frequency on network performance. Our results, based on network simulations driven by real HPC and DC workloads, show that while TRNs are optimized for low fan-out communication patterns, they are less suited for carrying high fan-out workloads. PRNs exhibit better overall trade-off, capable of performing comparably to a fully non-blocking fat tree for low fan-out workloads, and significantly outperform TRNs for high fan-out communication patterns.

A Study of Failure Recovery and Logging of High-Performance Parallel File Systems

Large-scale parallel file systems (PFSs) play an essential role in high-performance computing (HPC). However, despite their importance, their reliability is much less studied or understood compared with that of local storage systems or cloud storage systems. Recent failure incidents at real HPC centers have exposed the latent defects in PFS clusters as well as the urgent need for a systematic analysis. To address the challenge, we perform a study of the failure recovery and logging mechanisms of PFSs in this article. First, to trigger the failure recovery and logging operations of the target PFS, we introduce a black-box fault injection tool called PFault , which is transparent to PFSs and easy to deploy in practice. PFault emulates the failure state of individual storage nodes in the PFS based on a set of pre-defined fault models and enables examining the PFS behavior under fault systematically. Next, we apply PFault to study two widely used PFSs: Lustre and BeeGFS. Our analysis reveals the unique failure recovery and logging patterns of the target PFSs and identifies multiple cases where the PFSs are imperfect in terms of failure handling. For example, Lustre includes a recovery component called LFSCK to detect and fix PFS-level inconsistencies, but we find that LFSCK itself may hang or trigger kernel panics when scanning a corrupted Lustre. Even after the recovery attempt of LFSCK, the subsequent workloads applied to Lustre may still behave abnormally (e.g., hang or report I/O errors). Similar issues have also been observed in BeeGFS and its recovery component BeeGFS-FSCK. We analyze the root causes of the abnormal symptoms observed in depth, which has led to a new patch set to be merged into the coming Lustre release. In addition, we characterize the extensive logs generated in the experiments in detail and identify the unique patterns and limitations of PFSs in terms of failure logging. We hope this study and the resulting tool and dataset can facilitate follow-up research in the communities and help improve PFSs for reliable high-performance computing.

Current development of professional education and science based on traditions and innovations

The paper briefly determines the key trends of science and digital university based on the experience of fruitful activities and development of the Novosibirsk Science Center. The deep integration of science, education and production (M.A. Lavrentiev’s ‘triangle’) and the active participation of federal and regional authorities (the ‘triangle’ turns into the ‘’tetrahedron’ in accordance with specialists’ opinion) within short timeframes enables to mobilize intellectual, educational, personnel, technological and management resources for solving practical problems such as developing the Siberian Ring Photon Source, the Siberian National Center for High Performance Computing and Data Processing.

Preparing for the Future—Rethinking Proxy Applications

A considerable amount of research and engineering went into designing proxy applications, which represent common high-performance computing (HPC) workloads, to co-design and evaluate the current generation of supercomputers, e.g., RIKEN’s supercomputer Fugaku, ANL’s Aurora, or ORNL’s Frontier. This process was necessary to standardize the procurement while avoiding duplicated effort at each HPC center to develop their own benchmarks. Unfortunately, proxy applications force HPC centers and providers (vendors) into an undesirable state of rigidity, in contrast to the fast-moving trends of current technology and future heterogeneity. To accommodate an extremely heterogeneous future, we have to reconsider how to co-design supercomputers during the next decade, and avoid repeating past mistakes.

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. In this paper, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for O(N) task execution and O(NlnN) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

Proposal of a Dynamic Algorithm for the Maintenance and Vehicle Routing Problem with Time Windows

Context: In the context of business organizations, every process in which the product is immersed has a cost and time associated with it. The area of maintenance planning and scheduling is no exception; however, it is an aspect in which few companies specialize, tending to be outsourced. In this sense, the application of combinatorial models is a tool with a high potential to improve the overall performance of the organization through the understanding of the integral maintenance process. Method: A two-phase (maintenance and routing) dynamic algorithm is proposed which considers a set of clients distributed in a maintenance network (distance), where each of the technicians start from the same central node (depot), which, in turn, is the endpoint of each assigned route. The objective is to minimize the total cost associated with the development of preventive and corrective maintenance of all machines to be evaluated. With this purpose, the formulation of the mathematical problem for each of the phases and its interrelation method is proposed. Then, performance measures are expressed to evaluate the achieved objectives. Results: The results satisfy a consistent alternative for the resolution of problems of the NP-Hard type, which generates a high level of complexity to the model. That is, it proposes a tool for solving problems of these characteristics in low computational response times and with appealing results. Conclusions: The combined maintenance and routing model using a dynamic algorithm addresses the maintenance and routing problem satisfactorily. The model shows good results with respect to the comparison optimization model in percentage gaps of performance measures lower than 5%. As for the computational time required, a reduction of up to 98% was achieved, which makes it an ideal alternative for highly complex scenarios. Finally, achieving a higher level of characterization, employing multi-objective decision criteria and a greater number of constraints to the problem, is proposed in future research. Acknowledgements: To the High-Performance Computing Center (CECAD - Centro de computación de Alto Desempeño) of Universidad Distrital Francisco José de Caldas for their support, as well as for providing us with a virtual machine to run the proposed mathematical model, which was an essential element in the results obtained.

API Development Increases Access to Shared Computing Resources at Boston University.

Within the last few decades, increases in computational resources have contributed enormously to the progress of science and engineering (S & E). To continue making rapid advancements, the S & E community must be able to access computing resources. One way to provide such resources is through High-Performance Computing (HPC) centers. Many academic research institutions offer their own HPC Centers but struggle to make the computing resources easily accessible and user-friendly. Here we present SHABU, a RESTful Web API framework that enables S & E communities to access resources from Boston University's Shared Computing Center (SCC). The SHABU requirements are derived from the use cases described in this work.

Open IACS: apertura, interoperabilità e riuso dati dai sistemi integrati di gestione e controllo per i pagamenti in agricoltura

Open LOD platform based on HPC capabilities for Integrated Administration and Control System of Common Agrarian Policy (Open IACS) is an EU funded project (4.5 MEur) fostering the administrative data reuse in the context of the Common Agricultural Policy (CAP) and granting open access to farmers, policy makers and third parties. Data from European paying agencies for agriculture will be harmonised, and a network of Linked Open Data will be developed. An interoperable system of EU High Performance Computing Centres will provide a platform for demonstrating different use case scenarios (e.g. applications for computing CAP performance indicators, climate change indicators and integration between environmental and agricultural data).

Electrochemical Aziridination of Tetrasubstituted Alkenes with Ammonia

Ammonia (NH3) is an ideal nitrogen source in terms of availability, reactivity, safety, atom economy, environmental compatibility, and ease of isolation. However, its utility for amine synthesis is...

Automatic Fetal Ultrasound Standard Plane Recognition Based on Deep Learning and IIoT

Intelligent ultrasound imaging based on deep learning is one of the important applications in the field of intelligent medical care. In this article, we propose an automatic fetal ultrasound standard plane recognition (FUSPR) model based on deep learning in the Industrial Internet of Things (IIoT) environment. We build a distributed ultrasound data processing and predicting platform by using the IIoT and high-performance computing (HPC) technology. The FUSPR model deployed in the HPC center consists of a convolutional neural network (CNN) component and a recurrent neural network (RNN) component, which learns the spatial and temporal features of the ultrasound video stream by using multitask learning, respectively. The CNN component identifies fetal key anatomical structures from each video frame and accurately recognizes the potential four fetal standard planes. The RNN component obtains the temporal information between adjacent frames, and it realizes precise localization and tracking of fetal organs across frames. In addition, we introduce two feature fusion strategies into the FUSPR model, i.e., CNN fusion and RNN fusion, to fit the spatial sequence and motion representation in the video stream, thereby effectively improving the accuracy and robustness of the model. Extensive experiments conducted on more than 1000 ultrasound videos show that the FUSPR model is superior to the competing baselines in terms of accuracy and performance.

Interactive Supercomputing With Jupyter

Rich user interfaces like Jupyter have the potential to make interacting with a supercomputer easier and more productive, consequently attracting new kinds of users and helping to expand the application of supercomputing to new science domains. For the scientist user, the ideal rich user interface delivers a familiar, responsive, introspective, modular, and customizable platform upon which to build, run, capture, document, re-run, and share analysis workflows. From the provider or system administrator perspective, such a platform would also be easy to configure, deploy securely, update, customize, and support. Jupyter checks most if not all of these boxes. But from the perspective of leadership computing organizations that provide supercomputing power to users, such a platform should also make the unique features of a supercomputer center more accessible to users and more composable with high-performance computing (HPC) workflows. Project Jupyter’s ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://jupyter.org/about</uri> ) core design philosophy of extensibility, abstraction, and agnostic deployment has allowed HPC centers like National Energy Research Scientific Computing Center to bring in advanced supercomputing capabilities that can extend the interactive notebook environment. This has enabled a rich scientific discovery platform, particularly for experimental facility data analysis and machine learning problems.

Transparent Integration of Opportunistic Resources into the WLCG Compute Infrastructure

The inclusion of opportunistic resources, for example from High Performance Computing (HPC) centers or cloud providers, is an important contribution to bridging the gap between existing resources and future needs by the LHC collaborations, especially for the HL-LHC era. However, the integration of these resources poses new challenges and often needs to happen in a highly dynamic manner. To enable an effective and lightweight integration of these resources, the tools COBalD and TARDIS are developed at KIT. In this contribution we report on the infrastructure we use to dynamically offer opportunistic resources to collaborations in the World Wide LHC Computing Grid (WLCG). The core components are COBalD/TARDIS, HTCondor, CVMFS and modern virtualization technology. The challenging task of managing the opportunistic resources is performed by COBalD/TARDIS. We showcase the challenges, employed solutions and experiences gained with the provisioning of opportunistic resources from several resource providers like university clusters, HPC centers and cloud setups in a multi VO environment. This work can serve as a blueprint for approaching the provisioning of resources from other resource providers.

Harnessing HPC resources for CMS jobs using a Virtual Private Network

The processing needs for the High Luminosity (HL) upgrade for the LHC require the CMS collaboration to harness the computational power available on non-CMS resources, such as High-Performance Computing centers (HPCs). These sites often limit the external network connectivity of their computational nodes. In this paper we describe a strategy in which all network connections of CMS jobs inside a facility are routed to a single point of external network connectivity using a Virtual Private Network (VPN) server by creating virtual network interfaces in the computational nodes. We show that when the computational nodes and the host running the VPN server have the namespaces capability enabled, the setup can run entirely on user space with no other root permissions required. The VPN server host may be a privileged node inside the facility configured for outside network access, or an external service that the nodes are allowed to contact. When namespaces are not enabled at the client side, then the setup falls back to using a SOCKS server instead of virtual network interfaces. We demonstrate the strategy by executing CMS Monte Carlo production requests on opportunistic non-CMS resources at the University of Notre Dame. For these jobs, cvmfs support is tested via fusermount (cvmfsexec), and the native fuse module.

Heterogeneous Reconstruction of Tracks and Primary Vertices With the CMS Pixel Tracker.

The High-Luminosity upgrade of the Large Hadron Collider (LHC) will see the accelerator reach an instantaneous luminosity of 7 × 1034 cm−2 s−1 with an average pileup of 200 proton-proton collisions. These conditions will pose an unprecedented challenge to the online and offline reconstruction software developed by the experiments. The computational complexity will exceed by far the expected increase in processing power for conventional CPUs, demanding an alternative approach. Industry and High-Performance Computing (HPC) centers are successfully using heterogeneous computing platforms to achieve higher throughput and better energy efficiency by matching each job to the most appropriate architecture. In this paper we will describe the results of a heterogeneous implementation of pixel tracks and vertices reconstruction chain on Graphics Processing Units (GPUs). The framework has been designed and developed to be integrated in the CMS reconstruction software, CMSSW. The speed up achieved by leveraging GPUs allows for more complex algorithms to be executed, obtaining better physics output and a higher throughput.

Toward lower-diameter large-scale HPC and data center networks with co-packaged optics

We investigate the advantages of using co-packaged optics for building low-diameter, large-scale high-performance computing (HPC) and data center networks. The increased escape bandwidth offered by co-packaged optics can enable high-radix switch implementations of more than 150 switch ports, which can be combined with data rates of up to 400 Gb/s per port. From the network architecture perspective, the key benefits of using co-packaged optics in future fat-tree networks include (a) the ability to implement large-scale topologies of &gt; --> 11 , 000 end points by eliminating the need for a third switching layer and (b) the ability to provide up to 4 × higher bisection bandwidth compared to existing solutions, reducing at the same time the number of required switch application-specific integrated circuits by &gt; --> 80 % . From the network operation perspective, both reduced energy consumption and lower packet delays can be achieved since fewer hops are required; i.e., packets need to traverse fewer serializer/deserializer lanes and fewer switch buffers, which reduces the probability of contending with other packets and improves the tolerance of network congestion. The performance of the proposed architecture is evaluated via discrete-event simulations for a wide range of representative HPC synthetic-traffic cases that include both hotspot and non-hotspot scenarios. The simulation results suggest that co-packaged optics form a promising solution to keep up with bandwidth scaling in future networks, while the reduced number of switching layers can lead to significant mean packet delay improvements that start from 30% and reach up to 74% for high-load conditions.

Unsupervised KPIs-Based Clustering of Jobs in HPC Data Centers.

Performance analysis is an essential task in high-performance computing (HPC) systems, and it is applied for different purposes, such as anomaly detection, optimal resource allocation, and budget planning. HPC monitoring tasks generate a huge number of key performance indicators (KPIs) to supervise the status of the jobs running in these systems. KPIs give data about CPU usage, memory usage, network (interface) traffic, or other sensors that monitor the hardware. Analyzing this data, it is possible to obtain insightful information about running jobs, such as their characteristics, performance, and failures. The main contribution in this paper was to identify which metric/s (KPIs) is/are the most appropriate to identify/classify different types of jobs according to their behavior in the HPC system. With this aim, we had applied different clustering techniques (partition and hierarchical clustering algorithms) using a real dataset from the Galician computation center (CESGA). We concluded that (i) those metrics (KPIs) related to the network (interface) traffic monitoring provided the best cohesion and separation to cluster HPC jobs, and (ii) hierarchical clustering algorithms were the most suitable for this task. Our approach was validated using a different real dataset from the same HPC center.

Sensitivity of Botswana Ex-Tropical Cyclone Dineo rainfall simulations to cloud microphysics scheme

Background: Numerical weather and climate models rely on the use of microphysics schemes to simulate clouds and produce precipitation at convective scales. It is important that we understand how different microphysics schemes perform when simulating high impact weather to inform operational forecasting. Methods: Simulations a heavy rainfall event from 17-20 February 2017 over Botswana were made with the Weather Research and Forecasting (WRF) model using four different microphysics schemes. The schemes used were the Weather Research and Forecasting Single Moment 6-class scheme (WSM6); Weather Research and Forecasting Single Moment 5-class scheme (WSM5); Stony Brook University scheme (SBU-YLIN); and Thompson scheme. WSM5 is considered as the least sophisticated of the four schemes, while Thompson is the most sophisticated. Simulations were initialized and forced by the Global Forecast System (GFS), and configured with a grid spacing of 9km over an outer domain and 3km for a nested inner domain without the convection parameterization. The simulations were produced using the University of Botswana and the Centre for High Performance Computing (CHPC) High Performance Computing (HPC) systems. Results: WSM5 and WSM6 simulations are mostly similar; the presence of graupel in WSM6 did not result in large differences in the rainfall simulations. SBU-YLIN simulated the least amount of rainfall, followed by Thompson. All the schemes captured the north-south rainfall gradient observed on 17 February, but with all simulations rainfall is simulated slightly south of where it was observed. All the schemes overestimated rainfall on 18 February over the central parts of Botswana, and underestimated rainfall on 19 February over most of Botswana. Conclusions: Simulations with different microphysics looked more similar to each other, than to observations. Future studies will test WRF configurations including a single nest over Botswana to determine the best configuration for operational forecasting by the Botswana Department of Meteorological Services.

A Review of a Distributed High performance Computing Implementation

ABSTRACT This article presents the case of a regional government and academia working together to realize High-Performance Computing Wales, a shared supercomputing infrastructure. The initiative aimed to support economic development through creation of a distributed supercomputing capability, a skills development programme, and an initial programme of collaborative research and innovation. From the perspective of Smart Specialization, the study examines whether the economic development ambitions were achieved, and adds to the limited literature reviewing this aspect of High-Performance Computing Centers. Using principles of public sector investment guidelines, the study presents for Smart Specialization theory that such facilities can provide a general purpose technology. Further, from the perspective of practitioners, it highlights important steps in the appraisal, monitoring, and evaluation of such initiatives.