High Performance Computing Facilities Research Articles

Status of DUNE Offline Computing

We summarize the status of Deep Underground Neutrino Experiment (DUNE) Offline Software and Computing program. We describe plans for the computing infrastructure needed to acquire, catalog, reconstruct, simulate and analyze the data from the DUNE experiment and its prototypes in pursuit of the experiment’s physics goals of precision measurements of neutrino oscillation parameters, detection of astrophysical neutrinos, measurement of neutrino interaction properties and searches for physics beyond the Standard Model. In contrast to traditional HEP computational problems, DUNE’s Liquid Argon Time Projection Chamber data consist of simple but very large (many GB) data objects which share many characteristics with astrophysical images. We have successfully reconstructed and simulated data from 4% prototype detector runs at CERN. The data volume from the full DUNE detector, when it starts commissioning late in this decade will present memory management challenges in conventional processing but significant opportunities to use advances in machine learning and pattern recognition as a frontier user of High Performance Computing facilities capable of massively parallel processing. Our goal is to develop infrastructure resources that are flexible and accessible enough to support creative software solutions as HEP computing evolves.

Scaling the Inference of Digital Pathology Deep Learning Models Using CPU-Based High-Performance Computing

Digital pathology whole-slide images (WSIs) are large-size gigapixel images and image analysis based on deep learning artificial intelligence (AI) technology often involves pixel-wise testing of a trained deep learning neural network (DLNN) on hundreds of WSI images, which is time consuming. We take advantage of High-Performance Computing (HPC) facilities to parallelize this procedure into multiple independent (and hence delightfully parallel) tasks. However, traditional software parallelization techniques and regular file formats can have significant scaling problems on HPC clusters. In this work, a useful computational strategy is designed to localize and extract relevant patches in WSI files and group them in HDF5 files well suited for parallel I/O. HPC's array job facilities are adapted for hierarchical scaling and parallelization of WSI pre-processing and testing of trained algorithms. Applying these techniques to testing a trained DLNN on the CAMELYON datasets with 399 WSIs reduced the theoretical processing time of 18 years on a single CPU or 30 days on a single GPU to less than 45 hours on an HPC cluster of 4,000 CPU cores. The efficiency-accuracy trade-off we demonstrated on this dataset further reinforced the importance of efficient computation techniques, without which the accuracy may be sacrificed. The framework developed here for testing DLNNs does not rely on any specific neural network architecture and HPC cluster setup and can be utilized for any large-scale image processing and big-data analysis.

Implementation of Joint Funded ICT Infrastructure Projects for Digital Acceleration and Youth Empowerment. The Case of NRF-Kibabii University

Kibabii University is the 23rd Public University in Kenya and is committed to high quality teaching, research, and extension services to create value to her stakeholders. The University considers Information and Communication Technology (ICT) as its flagship programmes and purposes to be a Computing Research and Innovation Centre (CRIC) /hub in East and Central Africa. The vision of the University is to be a global University for Research, Innovation and Technology. To achieve this, it requires improved ICT infrastructure to enhance digital acceleration and empowerment of the Youth. High Performance Computing facility was required to enable students and researchers perform simulation and modelling. Other facilities required were: mobile computing, digital forensics, CISCO professional laboratory, and video conferencing and data centre. To achieve this the University required funding beyond its normal capitation. To raise the additional funding the University responded to a call for Funding by National Research Fund. After approval and disbursement of the funds the implementation of the Project was initiated. An engineering design approach was adopted. This paper highlights some of technical issues that were faced during the Implementation of the projects. The project was successfully implemented and Commissioned by the joint team. It awaits fully exploitation for benefits of job creation and youth empowerment

Namibia's first high performance computer

High performance computing (HPC) refers to the practice of aggregating computing power of several computing nodes in a way that delivers much higher performance than one could achieve by a typical desktop computer in order to solve large problems in business, science, or engineering. The University of Namibia has so far received two HPC racks from the Centre for High Performance Computing in South Africa, of which one is operational. The primary use of the rack was foreseen to be human capacity development and awareness in HPC and to form part of Namibia’s readiness in participating in the Square Kilometre Array (SKA) and the African Very Long Baseline Interferometry Network (AVN) projects, but is now also being used for research in multi-wavelength astronomy and beyond. This is one of the first HPC services set up and operated by an entirely African team. We perform tests to benchmark the computational power and data transfer capabilities of the system and find that each node, on average, has a peak performance power of 82.4±1.1 GFLOPS. We also summarise all the projects that have enlisted the HPC facility.

A Sustainability Model for Continental Hybridization of High Performance Computing (HPC) in Africa

The potential for High-performance computing (HPC) opportunities in the areas of health, environment, energy, climate change, business and livelihood are immense in Africa. However, the lack of sufficient HPC facilities and capacity skills deficiencies continue to curtail its growth. This paper maps the status of HPC facilities in Africa. Thereafter based on a review of various business and funding models, it suggests a hybridization model as a way for building sustainable HPC infrastructures across the continent to provide support for teaching and learning, research and education. The main contribution of this paper is the anchoring of hybridization as a sustainable business model for improving the access and establishment of HPC infrastructure in Africa.

Nitrogen Reduction to Ammonia by a Phosphorus-Nitrogen PN 3 P-Mo(V) Nitride Complex: Significant Enhancement via Ligand Postmodification

Nitrogen Reduction to Ammonia by a Phosphorus-Nitrogen PN ³ P-Mo(V) Nitride Complex: Significant Enhancement via Ligand Postmodification

Computing efficiency of XBeach hydro- and wave dynamics on Graphics Processing Units (GPUs)

Numerical prediction of coastal inundation can be complex due to the multiple physical processes involved and typically requires two-dimensional numerical model extents, particularly in areas with complex along-shore morphology. Such model domains often incur relatively high computational expense. Recent extreme inundation studies for Wellington, New Zealand, were executed using the numerical tool, XBeach. Here, the two-dimensional physical dynamics associated with multiple small embayments and both reef and sandy beach substrates, require large, high spatial resolution numerical model extents, informed by a multi-source elevation surface. XBGPU, is a translation of key XBeach features into code that permits GPU-based acceleration. The present study presents a comparison between XBeach and XBGPU for the same numerical model configuration and extents. Three model resolutions were employed, ranging from a typical desktop CPU based XBeach model resolution, to the highest resolution model that will require High Performance Computing (HPC) scale resources. Two CPU HPC facilities were used and five GPUs to investigate the scalability of both XBeach and XBGPU. The latter ranged from desktop grade units to GPUs associated with professional computing facilities. XBeach scalability is investigated by mean of the speed-up ratio, the time saving ratio and the computational efficiency. These are in reference to the computational speed of a model running on one CPU core. The XBGPU speed-up ratio is presented as a function of the slowest GPU. Direct comparisons between XBeach and XBGPU were achieved by using computational capacity as a metric. The results indicate that even a desktop grade GPU can compete with the computational efficiency of HPC-scale CPU facilities. Small model resolutions presented inefficient scaling on both high-performance CPUs and GPUs while the high-resolution model presented near linear scalability for the high-performance GPUs. Nevertheless, HPCs can be efficient to solve large computational problems if enough CPUs are employed.

Dynamic non-local damage analysis using an octree pattern-based massively parallel explicit solver

This paper presents the development of a massively parallel explicit solver based on the central difference method (CDM) for the dynamic analysis of damage processes. In this context, the material degradation is incorporated using an integral-type non-local isotropic damage model. A fully automatic preprocessing is enabled by following the octree-based mesh generation paradigm. To handle neighbouring elements with different sizes and to avoid hanging nodes in the generated mesh, polyhedral elements are implemented within the framework of the scaled boundary finite element method (SBFEM). A pre-computation approach is advocated to develop a highly efficient solver, where the limited number of master cells in a balanced octree grid are exploited. The parallelisation is carried out using a mesh-partitioning technique and message-passing-interface (MPI) directives, which is highly complementary to the use in high-performance computing (HPC) facilities. In order to showcase the performance of the proposed solver, a problem of practical interest is selected, featuring the analysis of an XCT image-based model of concrete containing about 1 billion voxels. The simulated results have been obtained within minutes by exploiting the computational power of approximately 12k cores.

Successful Retreatment With Venetoclax in a Patient With Chronic Lymphocytic Leukemia.

Successful Retreatment With Venetoclax in a Patient With Chronic Lymphocytic Leukemia.

A Survey on Malleability Solutions for High-Performance Distributed Computing

Maintaining a high rate of productivity, in terms of completed jobs per unit of time, in High-Performance Computing (HPC) facilities is a cornerstone in the next generation of exascale supercomputers. Process malleability is presented as a straightforward mechanism to address that issue. Nowadays, the vast majority of HPC facilities are intended for distributed-memory applications based on the Message Passing (MP) paradigm. For this reason, many efforts are based on the Message Passing Interface (MPI), the de facto standard programming model. Malleability aims to rescale executions on-the-fly, in other words, reconfigure the number and layout of processes in running applications. Process malleability involves resources reallocation within the HPC system, handling processes of the application, and redistributing data among those processes to resume the execution. This manuscript compiles how different frameworks address process malleability, their main features, their integration in resource management systems, and how they may be used in user codes. This paper is a detailed state-of-the-art devised as an entry point for researchers who are interested in process malleability.

High-Performance Computing and ABMS for High-Resolution COVID-19 Spreading Simulation

This paper presents an approach for the modeling and the simulation of the spreading of COVID-19 based on agent-based modeling and simulation (ABMS). Our goal is not only to support large-scale simulations but also to increase the simulation resolution. Moreover, we do not assume an underlying network of contacts, and the person-to-person contacts responsible for the spreading are modeled as a function of the geographical distance among the individuals. In particular, we defined a commuting mechanism combining radiation-based and gravity-based models and we exploited the commuting properties at different resolution levels (municipalities and provinces). Finally, we exploited the high-performance computing (HPC) facilities to simulate millions of concurrent agents, each mapping the individual’s behavior. To do such simulations, we developed a spreading simulator and validated it through the simulation of the spreading in two of the most populated Italian regions: Lombardy and Emilia-Romagna. Our main achievement consists of the effective modeling of 10 million of concurrent agents, each one mapping an individual behavior with a high-resolution in terms of social contacts, mobility and contribution to the virus spreading. Moreover, we analyzed the forecasting ability of our framework to predict the number of infections being initialized with only a few days of real data. We validated our model with the statistical data coming from the serological analysis conducted in Lombardy, and our model makes a smaller error than other state of the art models with a final root mean squared error equal to 56,009 simulating the entire first pandemic wave in spring 2020. On the other hand, for the Emilia-Romagna region, we simulated the second pandemic wave during autumn 2020, and we reached a final RMSE equal to 10,730.11.

Improvement of Energy-Efficiency in High Performance Computing (HPC)

Power continues to be a challenging problem facing institutions which are deploying high performance computing (HPC) facility in developing countries including Tanzania. These institutions like Dar es Salaam Institute of Technology (DIT) can face high operational costs due to the high energy consumption caused by computational operations of HPC systems. Because of the exceedingly high energy cost, diminishing energy consumption has become a major concern in deploying HPC systems. This manuscript per the authors used empirical modeling (EM) to predict the trends of power and cost operation for the year 2018 and 2019 on running HPC system. It was noticed that in addition to optimizing the system configuration, the proposed hybrid algorithm can both minimize completion time while increasing throughput as a result of decreasing energy consumption and maximize resource utilization. The experimental results showed that Infiniband switch can save energy up to 10.7%, and the proposed hybrid algorithm can reduce time up to 25% as a result of energy reduction.

Benchmarking 2D against 3D FDTD codes for the assessment of the measurement performance of a low field side plasma position reflectometer applicable to IDTT

O-mode reflectometry, a technique to diagnose fusion plasmas, is foreseen as a source of real-time (RT) plasma position and shape measurements for control purposes in the coming generation of machines such as DEMO. It is, thus, of paramount importance to predict the behavior and capabilities of these new reflectometry systems using synthetic diagnostics. Finite-difference time-domain (FDTD) time-dependent codes allow for a comprehensive description of reflectometry but are computationally demanding, especially when it comes to three-dimensional (3D) simulations, which requires access to High Performance Computing (HPC) facilities, making the use of two-dimensional (2D) codes much more common. It is important to understand the compromises made when using a 2D model in order to decide if it is applicable or if a 3D approach is required. This work attempts to answer this question by comparing simulations of a potential plasma position reflectometer (PPR) at the Low Field-Side (LFS) on the Italian Divertor Tokamak Test facility (IDTT) carried out using two full-wave FDTD codes, REFMULF (2D) and REFMUL3 (3D). In particular, the simulations consider one of IDTT’s foreseen plasma scenarios, namely, a Single Null (SN) configuration, at the Start Of Flat-top (SOF) of the plasma current.

WeBrain: A web-based brainformatics platform of computational ecosystem for EEG big data analysis

The current evolution of ‘cloud neuroscience’ leads to more efforts with the large-scale EEG applications, by using EEG pipelines to handle the rapidly accumulating EEG data. However, there are a few specific cloud platforms that seek to address the cloud computational challenges of EEG big data analysis to benefit the EEG community. In response to the challenges, a WeBrain cloud platform (https://webrain.uestc.edu.cn/) is designed as a web-based brainformatics platform and computational ecosystem to enable large-scale EEG data storage, exploration and analysis using cloud high-performance computing (HPC) facilities. WeBrain connects researchers from different fields to EEG and multimodal tools that have become the norm in the field and the cloud processing power required to handle those large EEG datasets. This platform provides an easy-to-use system for novice users (even no computer programming skills) and provides satisfactory maintainability, sustainability and flexibility for IT administrators and tool developers. A range of resources are also available on https://webrain.uestc.edu.cn/, including documents, manuals, example datasets related to WeBrain, and collected links to open EEG datasets and tools. It is not necessary for users or administrators to install any software or system, and all that is needed is a modern web browser, which reduces the technical expertise required to use or manage WeBrain. The WeBrain platform is sponsored and driven by the China-Canada-Cuba international brain cooperation project (CCC-Axis, http://ccc-axis.org/), and we hope that WeBrain will be a promising cloud brainformatics platform for exploring brain information in large-scale EEG applications in the EEG community.

On the use of CFD to obtain head loss coefficients in hydraulic systems and its application to liquid metal MHD flows in nuclear fusion reactor blankets

When an incompressible fluid flows through a contraction in a conduit, the increase in the kinetic energy of the fluid is accompanied by a pressure drop. This pressure drop is not to be assimilated with head loss. If downstream the fluid encounters an expansion in the conduit, the energy conversion will take place in the opposite way. Therefore, when a geometrical singularity is analysed to assess its contribution to the pumping power requirements of the system, the whole mechanical energy transfer of the fluid in the singularity has to be taken into account, and not only the pressure variation. The first part of the present work establishes a method to obtain head loss coefficients in geometric singularities of hydrodynamic circuits using the results of computational fluid dynamics (CFD) calculations. These coefficients are of interest when modelling the whole system with a 1D system code, for instance. In the second part of the article, the method is applied to a more complex case, involving magnetohydrodynamic (MHD) phenomena. Thus, a prototypical channel singularity in a liquid metal circuit subject to a magnetic field is analysed. The layout is representative of a case that could be found in the liquid metal blankets to be used in nuclear fusion reactors. The influence of the MHD phenomena is studied and the differences with a purely hydrodynamic case are pointed out. The MHD analyses have been done in the Marconi High Performance Computing facility, using 48 cores, each case needing between one and two weeks to complete.

When will I get out of the Hospital? Modeling Length of Stay using Comorbidity Networks

ABSTRACT A reliable and accurate estimate of the expected hospital length of stay (LOS) of a patient is important to patients, medical providers, and insurance companies. Predicting hospital Length of Stay (LOS) is a complex and ill-structured problem, driven by many factors such as a patient’s individual characteristics, treatment plans, and disease-interactions. In this paper, we develop a novel model to predict the expected LOS at the time of admission by combining network science and deep learning. We propose a two-dimensional construct of latent comorbidities comprising historical and probable comorbidities that a patient does not currently manifest but could likely develop during the course of hospital stay. The probable comorbidities are derived from a network comprising relationships among diseases observed in 3.2 million patient records in hundreds of US hospitals. We employ this construct of latent comorbidities in deep learning models to predict patients’ LOS using almost 10 million other patient visits belonging to various disease categories. Implementing these models and analyses required a high-performance computing (Big Data) facility. The average mean absolute percent error of our models across all categories of diseases was 29.8%, which is the best in the current state-of-the-art. Our primary contribution is in developing a generalizable method to create a predictor construct for recognizing underlying relationships through network analyses, which can then be used in a deep learning model to predict an exogenous dependent variable.

FourPhonon: An extension module to ShengBTE for computing four-phonon scattering rates and thermal conductivity

FourPhonon is a computational package that can calculate four-phonon scattering rates in crystals. It is built within ShengBTE framework, which is a well-recognized lattice thermal conductivity solver based on Boltzmann transport equation. An adaptive energy broadening scheme is implemented for the calculation of four-phonon scattering rates. In analogy with thirdorder.py in ShengBTE, we also provide a separate python script, Fourthorder.py, to calculate fourth-order interatomic force constants. The extension module preserves all the nice features of the well-recognized lattice thermal conductivity solver ShengBTE, including good parallelism and straightforward workflow. In this paper, we discuss the general theory, program design, and example calculations on Si, BAs and LiCoO2. Program summaryProgram Title:FourPhononCPC Library link to program files:https://doi.org/10.17632/jjcwt5436h.1Developer's repository link:https://github.com/FourPhononLicensing provisions: GNU General Public License version 3Programming language: Fortran 90, MPINature of problem: Calculation of lattice thermal conductivity and related quantities, determination of both three-phonon and four-phonon scattering ratesSolution method: Four-phonon scattering rates at RTA level, adaptive broadening schemeAdditional comments including restrictions and unusual features: For a productive run, one needs to use High Performance Computing facilities and it takes several hours to several days to finish calculations

The Italian open data meteorological portal: MISTRAL

AbstractAt the national level, in Italy, observational and forecast data are collected by various public bodies and are often kept in various small, heterogeneous and non‐interoperable repositories, released under different licenses, thus limiting the usability for external users. In this context, MISTRAL (the Meteo Italian SupercompuTing PoRtAL) was launched as the first Italian meteorological open data portal, with the aim of promoting the reuse of meteorological data sets available at national level coverage. The MISTRAL portal provides (and archives) meteorological data from various observation networks, both public and private, and forecast data that are generated and post‐processed within the Consortium for Small‐scale Modeling‐Limited Area Model Italia (COSMO‐LAMI) agreement using high performance computing (HPC) facilities. Also incorporated is the Italy Flash Flood use case, implemented with the collaboration of European Centre for Medium‐Range Weather Forecasts (ECMWF), which exploits cutting edge advances in HPC‐based post‐processing of ensemble precipitation forecasts, for different model resolutions, and applies those to deliver novel blended‐resolution forecasts specifically for Italy. Finally, in addition to providing architectures for the acquisition and display of observational data, MISTRAL also delivers an interactive system for visualizing forecast data of different resolutions as superimposed multi‐layer maps.

Future directions of chemical theory and computation

Abstract Theoretical and computational chemistry aims to develop chemical theory and to apply numerical computation and simulation to reveal the mechanism behind complex chemical phenomena via quantum theory and statistical mechanics. Computation is the third pillar of scientific research together with theory and experiment. Computation enables scientists to test, discover, and build models/theories of the corresponding chemical phenomena. Theoretical and computational chemistry has been advanced to a new era due to the development of high-performance computational facilities and artificial intelligence approaches. The tendency to merge electronic structural theory with quantum chemical dynamics and statistical mechanics is of increasing interest because of the rapid development of on-the-fly dynamic simulations for complex systems plus low-scaling electronic structural theory. Another challenging issue lies in the transition from order to disorder, from thermodynamics to dynamics, and from equilibrium to non-equilibrium. Despite an increasingly rapid emergence of advances in computational power, detailed criteria for databases, effective data sharing strategies, and deep learning workflows have yet to be developed. Here, we outline some challenges and limitations of the current artificial intelligence approaches with an outlook on the potential future directions for chemistry in the big data era.

Probabilistic Tsunami Hazard Analysis: High Performance Computing for Massive Scale Inundation Simulations

Probabilistic Tsunami Hazard Analysis (PTHA) quantifies the probability of exceeding a specified inundation intensity at a given location within a given time interval. PTHA provides scientific guidance for tsunami risk analysis and risk management, including coastal planning and early warning. Explicit computation of site-specific PTHA, with an adequate discretization of source scenarios combined with high-resolution numerical inundation modelling, has been out of reach with existing models and computing capabilities, with tens to hundreds of thousands of moderately intensive numerical simulations being required for exhaustive uncertainty quantification. In recent years, more efficient GPU-based High-Performance Computing (HPC) facilities, together with efficient GPU-optimized shallow water type models for simulating tsunami inundation, have now made local long-term hazard assessment feasible. A workflow has been developed with three main stages: 1) Site-specific source selection and discretization, 2) Efficient numerical inundation simulation for each scenario using the GPU-based Tsunami-HySEA numerical tsunami propagation and inundation model using a system of nested topo-bathymetric grids, and 3) Hazard aggregation. We apply this site-specific PTHA workflow here to Catania, Sicily, for tsunamigenic earthquake sources in the Mediterranean. We illustrate the workflows of the PTHA as implemented for High-Performance Computing applications, including preliminary simulations carried out on intermediate scale GPU clusters. We show how the local hazard analysis conducted here produces a more fine-grained assessment than is possible with a regional assessment. However, the new local PTHA indicates somewhat lower probabilities of exceedance for higher maximum inundation heights than the available regional PTHA. The local hazard analysis takes into account small-scale tsunami inundation features and non-linearity which the regional-scale assessment does not incorporate. However, the deterministic inundation simulations neglect some uncertainties stemming from the simplified source treatment and tsunami modelling that are embedded in the regional stochastic approach to inundation height estimation. Further research is needed to quantify the uncertainty associated with numerical inundation modelling and to properly propagate it onto the hazard results, to fully exploit the potential of site-specific hazard assessment based on massive simulations.