High Performance Computing Resources Research Articles

Rotating neurons for all-analog implementation of cyclic reservoir computing

Hardware implementation in resource-efficient reservoir computing is of great interest for neuromorphic engineering. Recently, various devices have been explored to implement hardware-based reservoirs. However, most studies were mainly focused on the reservoir layer, whereas an end-to-end reservoir architecture has yet to be developed. Here, we propose a versatile method for implementing cyclic reservoirs using rotating elements integrated with signal-driven dynamic neurons, whose equivalence to standard cyclic reservoir algorithm is mathematically proven. Simulations show that the rotating neuron reservoir achieves record-low errors in a nonlinear system approximation benchmark. Furthermore, a hardware prototype was developed for near-sensor computing, chaotic time-series prediction and handwriting classification. By integrating a memristor array as a fully-connected output layer, the all-analog reservoir computing system achieves 94.0% accuracy, while simulation shows >1000× lower system-level power than prior works. Therefore, our work demonstrates an elegant rotation-based architecture that explores hardware physics as computational resources for high-performance reservoir computing.

An Optimal GeoAI Workflow for Pan-Arctic Permafrost Feature Detection from High-Resolution Satellite Imagery

High-spatial-resolution satellite imagery enables transformational opportunities to observe, map, and document the micro-topographic transitions occurring in Arctic polygonal tundra at multiple spatial and temporal frequencies. Knowledge discovery through artificial intelligence, big imagery, and high-performance computing (HPC) resources is just starting to be realized in Arctic permafrost science. We have developed a novel high-performance image-analysis framework—Mapping Application for Arctic Permafrost Land Environment (MAPLE)—that enables the integration of operational-scale GeoAI capabilities into Arctic permafrost modeling. Interoperability across heterogeneous HPC systems and optimal usage of computational resources are key design goals of MAPLE. We systematically compared the performances of four different MAPLE workflow designs on two HPC systems. Our experimental results on resource utilization, total time to completion, and overhead of the candidate designs suggest that the design of an optimal workflow largely depends on the HPC system architecture and underlying service-unit accounting model.

Data assimilation of volcanic aerosol observations using FALL3D+PDAF

Abstract. Modelling atmospheric dispersal of volcanic ash and aerosols is becoming increasingly valuable for assessing the potential impacts of explosive volcanic eruptions on buildings, air quality, and aviation. Management of volcanic risk and reduction of aviation impacts can strongly benefit from quantitative forecasting of volcanic ash. However, an accurate prediction of volcanic aerosol concentrations using numerical modelling relies on proper estimations of multiple model parameters which are prone to errors. Uncertainties in key parameters such as eruption column height and physical properties of particles or meteorological fields represent a major source of error affecting the forecast quality. The availability of near-real-time geostationary satellite observations with high spatial and temporal resolutions provides the opportunity to improve forecasts in an operational context by incorporating observations into numerical models. Specifically, ensemble-based filters aim at converting a prior ensemble of system states into an analysis ensemble by assimilating a set of noisy observations. Previous studies dealing with volcanic ash transport have demonstrated that a significant improvement of forecast skill can be achieved by this approach. In this work, we present a new implementation of an ensemble-based data assimilation (DA) method coupling the FALL3D dispersal model and the Parallel Data Assimilation Framework (PDAF). The FALL3D+PDAF system runs in parallel, supports online-coupled DA, and can be efficiently integrated into operational workflows by exploiting high-performance computing (HPC) resources. Two numerical experiments are considered: (i) a twin experiment using an incomplete dataset of synthetic observations of volcanic ash and (ii) an experiment based on the 2019 Raikoke eruption using real observations of SO2 mass loading. An ensemble-based Kalman filtering technique based on the local ensemble transform Kalman filter (LETKF) is used to assimilate satellite-retrieved data of column mass loading. We show that this procedure may lead to nonphysical solutions and, consequently, conclude that LETKF is not the best approach for the assimilation of volcanic aerosols. However, we find that a truncated state constructed from the LETKF solution approaches the real solution after a few assimilation cycles, yielding a dramatic improvement of forecast quality when compared to simulations without assimilation.

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. In this paper, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for O(N) task execution and O(NlnN) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

Teaching computational genomics and bioinformatics on a high performance computing cluster-a primer.

The burgeoning field of genomics as applied to personalized medicine, epidemiology, conservation, agriculture, forensics, drug development, and other fields comes with large computational and bioinformatics costs, which are often inaccessible to student trainees in classroom settings at universities. However, with increased availability of resources such as NSF XSEDE, Google Cloud, Amazon AWS, and other high-performance computing (HPC) clouds and clusters for educational purposes, a growing community of academicians are working on teaching the utility of HPC resources in genomics and big data analyses. Here, I describe the successful implementation of a semester-long (16 week) upper division undergraduate/graduate level course in Computational Genomics and Bioinformatics taught at San Diego State University in Spring 2022. Students were trained in the theory, algorithms and hands-on applications of genomic data quality control, assembly, annotation, multiple sequence alignment, variant calling, phylogenomic analyses, population genomics, genome-wide association studies, and differential gene expression analyses using RNAseq data on their own dedicated 6-CPU NSF XSEDE Jetstream virtual machines. All lesson plans, activities, examinations, tutorials, code, lectures, and notes are publicly available at https://github.com/arunsethuraman/biomi609spring2022.

The COVID-19 High-Performance Computing Consortium.

In March of 2020, recognizing the potential of High Performance Computing (HPC) to accelerate understanding and the pace of scientific discovery in the fight to stop COVID-19, the HPC community assembled the largest collection of worldwide HPC resources to enable COVID-19 researchers worldwide to advance their critical efforts. Amazingly, the COVID-19 HPC Consortium was formed within one week through the joint effort of the Office of Science and Technology Policy (OSTP), the U.S. Department of Energy (DOE), the National Science Foundation (NSF), and IBM to create a unique public-private partnership between government, industry, and academic leaders. This article is the Consortium's story-how the Consortium was created, its founding members, what it provides, how it works, and its accomplishments. We will reflect on the lessons learned from the creation and operation of the Consortium and describe how the features of the Consortium could be sustained as a National Strategic Computing Reserve to ensure the nation is prepared for future crises.

Immune Repertoire Analysis on High-Performance Computing Using VDJServer V1: A Method by the AIRR Community.

AIRR-seq data sets are usually large and require specialized analysis methods and software tools. A typical Illumina MiSeq sequencing run generates 20-30 million 2×300bp paired-end sequence reads, which roughly corresponds to 15GB of sequence data to be processed. Other platforms like NextSeq, which is useful in projects where the full V gene is not needed, create about 400million 2×150bp paired-end reads. Because of the size of the data sets, the analysis can be computationally expensive, particularly the early analysis steps like preprocessing and gene annotation that process the majority of the sequence data. A standard desktop PC may take 3-5days of constant processing for a single MiSeq run, so dedicated high-performance computational resources may be required.VDJServer provides free access to high-performance computing (HPC) at the Texas Advanced Computing Center (TACC) through a graphical user interface (Christley et al. Front Immunol 9:976, 2018). VDJServer is a cloud-based analysis portal for immune repertoire sequence data that provides access to a suite of tools for a complete analysis workflow, including modules for preprocessing and quality control of sequence reads, V(D)J gene assignment, repertoire characterization, and repertoire comparison. Furthermore, VDJServer has parallelized execution for tools such as IgBLAST, so more compute resources are utilized as the size of the input data grows. Analysis that takes days on a desktop PC might take only a few hours on VDJServer. VDJServer is a free, publicly available, and open-source licensed resource. Here, we describe the workflow for performing immune repertoire analysis on VDJServer's high-performance computing.

Inspiring the Next Generation of HPC Engineers with Reconfigurable, Multi-Tenant Resources for Teaching and Research

There is a tradition at our university for teaching and research in High Performance Computing (HPC) systems engineering. With exascale computing on the horizon and a shortage of HPC talent, there is a need for new specialists to secure the future of research computing. Whilst many institutions provide research computing training for users within their particular domain, few offer HPC engineering and infrastructure-related courses, making it difficult for students to acquire these skills. This paper outlines how and why we are training students in HPC systems engineering, including the technologies used in delivering this goal. We demonstrate the potential for a multi-tenant HPC system for education and research, using novel container and cloud-based architecture. This work is supported by our previously published work that uses the latest open-source technologies to create sustainable, fast and flexible turn-key HPC environments with secure access via an HPC portal. The proposed multi-tenant HPC resources can be deployed on a “bare metal” infrastructure or in the cloud. An evaluation of our activities over the last five years is given in terms of recruitment metrics, skills audit feedback from students, and research outputs enabled by the multi-tenant usage of the resource.

Machine learning-based analysis of COVID-19 pandemic impact on US research networks

This study explores how fallout from the changing public health policy around COVID-19 has changed how researchers access and process their science experiments. Using a combination of techniques from statistical analysis and machine learning, we conduct a retrospective analysis of historical network data for a period around the stay-at-home orders that took place in March 2020. Our analysis takes data from the entire ESnet infrastructure to explore DOE high-performance computing (HPC) resources at OLCF, ALCF, and NERSC, as well as User sites such as PNNL and JLAB. We look at detecting and quantifying changes in site activity using a combination of t-Distributed Stochastic Neighbor Embedding (t-SNE) and decision tree analysis. Our findings bring insights into the working patterns and impact on data volume movements, particularly during late-night hours and weekends.

Bridging Scales in Bioenergy and Catalysis: A Review of Mesoscale Modeling Applications, Methods, and Future Directions

Between the molecular and reactor scales, which are familiar to the chemical engineering community, lies an intermediate regime, here termed the “mesoscale,” where transport phenomena and reaction kinetics compete on similar time scales. Bioenergy and catalytic processes offer particularly important examples of mesoscale phenomena owing to their multiphase nature and the complex, highly variable porosity characteristic of biomass and many structured catalysts. In this review, we overview applications and methods central to mesoscale modeling as they apply to reaction engineering of biomass conversion and catalytic processing. A brief historical perspective is offered to put recent advances in context. Applications of mesoscale modeling are described, and several specific examples from biomass pyrolysis and catalytic upgrading of bioderived intermediates are highlighted. Methods including reduced order modeling, finite element and finite volume approaches, geometry construction and import, and visualization of simulation results are described; in each category, recent advances, current limitations, and areas for future development are presented. Owing to improved access to high-performance computational resources, advances in algorithm development, and sustained interest in reaction engineering to sustainably meet societal needs, we conclude that a significant upsurge in mesoscale modeling capabilities is on the horizon that will accelerate design, deployment, and optimization of new bioenergy and catalytic technologies.

OPERATIONAL-SCALE GEOAI FOR PAN-ARCTIC PERMAFROST FEATURE DETECTION FROM HIGH-RESOLUTION SATELLITE IMAGERY

Abstract. Regional extent and spatiotemporal dynamics of Arctic permafrost disturbances remain poorly quantified. High spatial resolution commercial satellite imagery enables transformational opportunities to observe, map, and document the micro-topographic transitions occurring in Arctic polygonal tundra at multiple spatial and temporal frequencies. The entire Arctic has been imaged at 0.5 m or finer resolution by commercial satellite sensors. The imagery is still largely underutilized, and value-added Arctic science products are rare. Knowledge discovery through artificial intelligence (AI), big imagery, high performance computing (HPC) resources is just starting to be realized in Arctic science. Large-scale deployment of petabyte-scale imagery resources requires sophisticated computational approaches to automated image interpretation coupled with efficient use of HPC resources. In addition to semantic complexities, multitude factors that are inherent to sub-meter resolution satellite imagery, such as file size, dimensions, spectral channels, overlaps, spatial references, and imaging conditions challenge the direct translation of AI-based approaches from computer vision applications. Memory limitations of Graphical Processing Units necessitates the partitioning of an input satellite imagery into manageable sub-arrays, followed by parallel predictions and post-processing to reconstruct the results corresponding to input image dimensions and spatial reference. We have developed a novel high performance image analysis framework –Mapping application for Arctic Permafrost Land Environment (MAPLE) that enables the integration of operational-scale GeoAI capabilities into Arctic science applications. We have designed the MAPLE workflow to become interoperable across HPC architectures while utilizing the optimal use of computing resources.

Improvements to Supercomputing Service Availability Based on Data Analysis

As the demand for high-performance computing (HPC) resources has increased in the field of computational science, an inevitable consideration is service availability in large cluster systems such as supercomputers. In particular, the factor that most affects availability in supercomputing services is the job scheduler utilized for allocating resources. Consequent to submitting user data through the job scheduler for data analysis, 25.6% of jobs failed because of program errors, scheduler errors, or I/O errors. Based on this analysis, we propose a K-hook method for scheduling to increase the success rate of job submissions and improve the availability of supercomputing services. By applying this method, the job-submission success rate was improved by 15% without negatively affecting users’ waiting time. We also achieved a mean time between interrupts (MTBI) of 24.3 days and maintained average system availability at 97%. As this research was verified on the Nurion supercomputer in a real service environment, the value of the research is expected to be found in significant service improvements.

Editorial: Computational Genomics and Molecular Medicine for Emerging COVID-19

The papers in this special section focus on computational genomics and molecular medicine for emerging COVID-19. In 2020, World Health Organization announced Coronavirus disease (COVID)-19 is a pandemic disease, which is devastated the socio-economic life around the world. The disease caused by the zoonotic single-strand RNA virus known as “SARS-CoV-2”. To overcome the pandemic, the diagnosis and therapeutics products needs to be developed in short term. Developing therapeutics for infectious diseases, especially viral diseases always a challenging task for the scientific community. However, the utility of high-performance computational resources, artificial intelligence, and machine-learning algorithms can make the process in an affordable way through the usage of genomics, proteomics, pharmacogenomics, and chemical data. Thus, the special section received potential research articles related to computational genomics, molecular medicine, and COVID-19 from reputed scientist around the world. Different articles were employed machine learning, molecular dynamics, computer aided drug design techniques, and emphasizing viral genomics, mutation, drug target, drug candidates, and patient data, were included in this special section.

DAPT: A package enabling distributed automated parameter testing.

Modern agent-based models (ABM) and other simulation models require evaluation and testing of many different parameters. Managing that testing for large scale parameter sweeps (grid searches), as well as storing simulation data, requires multiple, potentially customizable steps that may vary across simulations. Furthermore, parameter testing, processing, and analysis are slowed if simulation and processing jobs cannot be shared across teammates or computational resources. While high-performance computing (HPC) has become increasingly available, models can often be tested faster with the use of multiple computers and HPC resources. To address these issues, we created the Distributed Automated Parameter Testing (DAPT) Python package. By hosting parameters in an online (and often free) "database", multiple individuals can run parameter sets simultaneously in a distributed fashion, enabling ad hoc crowdsourcing of computational power. Combining this with a flexible, scriptable tool set, teams can evaluate models and assess their underlying hypotheses quickly. Here, we describe DAPT and provide an example demonstrating its use.

Development of an orchestration aid system for gridded crop growth simulations using Kubernetes

Spatial simulations of crop growth under climate change have been limited to researchers who have access to the resources for high performance computing. The objective of this study was to develop an orchestration aid system for concurrent gridded simulations of crop growth, which would support the design of climate change adaptation options on crop production without expertise in distributed computing. The orchestration aid system was designed to help a user build a set of virtualized cluster computers using a simple input file, which would require little expertise in distributed computing, rather than manual configuration. The orchestration aid system, which was referred to as GROWLERS-kube, was implemented to launch multiple sets of gridded simulations using pods or containers managed by Kubernetes. As a case study, GROWLER-kube was executed using 16 Raspberry Pi 4 computers to perform 120 sets of the gridded simulations under diverse crop management options, including varying planting date and cultivar, for the period from 2001 to 2010. The wall time or the elapsed time for the given sets of the gridded simulation differed by configuration of virtualized cluster computers, such as the number of pods used for server and client nodes, although the total number of physical nodes were identical. For example, the wall time difference between virtualized cluster computer sets was about 28.9% when 15 worker nodes were used. In particular, the acceleration of the gridded simulations was at maximum using a large number of the virtualized cluster computers with a small number of nodes. It was found that the spatial distribution of planting dates and cultivars was similar to that of a previous study based on field experiments mostly in regions where rice is usually grown. These results suggest that GROWLERS-kube would facilitate the spatial assessment of climate change impact on crop production without considerable effort and expertise in distributed computing, which would aid a researcher to focus on the design of adaptation strategies.

Bioimage analysis workflows: community resources to navigate through a complex ecosystem

Workflows are the keystone of bioimage analysis, and the NEUBIAS (Network of European BioImage AnalystS) community is trying to gather the actors of this field and organize the information around them. One of its most recent outputs is the opening of the F1000Research NEUBIAS gateway, whose main objective is to offer a channel of publication for bioimage analysis workflows and associated resources. In this paper we want to express some personal opinions and recommendations related to finding, handling and developing bioimage analysis workflows. The emergence of "big data” in bioimaging and resource-intensive analysis algorithms make local data storage and computing solutions a limiting factor. At the same time, the need for data sharing with collaborators and a general shift towards remote work, have created new challenges and avenues for the execution and sharing of bioimage analysis workflows. These challenges are to reproducibly run workflows in remote environments, in particular when their components come from different software packages, but also to document them and link their parameters and results by following the FAIR principles (Findable, Accessible, Interoperable, Reusable) to foster open and reproducible science. In this opinion paper, we focus on giving some directions to the reader to tackle these challenges and navigate through this complex ecosystem, in order to find and use workflows, and to compare workflows addressing the same problem. We also discuss tools to run workflows in the cloud and on High Performance Computing resources, and suggest ways to make these workflows FAIR.

Conceptual Design of Extreme Sea-Level Early Warning Systems Based on Uncertainty Quantification and Engineering Optimization Methods

Coastal hazards linked to extreme sea-level events are projected to have a direct impact (by flooding) on 630 million of people by year 2100. Numerous operational forecasts already provide coastal hazard assessments around the world. However, they are largely based on either deterministic tools (e.g., numerical ocean and atmospheric models) or ensemble approaches which are both highly demanding in terms of high-performance computing (HPC) resources. Through a robust learning process, we propose conceptual design of an innovative architecture for extreme sea-level early warning systems based on uncertainty quantification/reduction and optimization methods. This approach might be cost-effective in terms of real-time computational needs while maintaining reliability and trustworthiness of the hazard assessments. The proposed architecture relies on three main tools aligning numerical forecasts with observations: (1) surrogate models of extreme sea-levels using polynomial chaos expansion, Gaussian processes or machine learning, (2) fast data assimilation via Bayesian inference, and (3) optimal experimental design of the observational network. A surrogate model developed for meteotsunami events – i.e., atmospherically induced long ocean waves in a tsunami frequency band – has already been proven to greatly improve the reliability of extreme sea-level hazard assessments. Such an approach might be promising for several coastal hazards known to destructively impact the world coasts, like hurricanes or typhoons and seismic tsunamis.

Comparing workflow application designs for high resolution satellite image analysis

Very High Resolution satellite and aerial imagery are used to monitor and conduct large scale surveys of ecological systems. Convolutional Neural Networks have successfully been employed to analyze such imagery to detect large animals and salient features. As the datasets increase in volume and number of images, utilizing High Performance Computing resources becomes necessary. In this paper, we investigate three task-parallel, data-driven workflow designs to support imagery analysis pipelines with heterogeneous tasks on high performance computing platforms. We analyze the capabilities of each design when processing 3097 and 1575 images for two distinct use cases, for a total of 4,672 satellite and aerial images and 8.35 TB of data. We experimentally model the execution time of the tasks of the image processing pipelines. We perform experiments to characterize resource utilization, total time to completion and overheads of each design. Our analysis shows which design is best suited to scientific pipelines with similar characteristics.

Optimized pulse for stimulated Raman adiabatic passage on noisy experimental platform* *Project supported by the Natonal Natural Science Foundation of China (Grant No. 11904018). This research was supported by the high performance computing (HPC) resources at Beihang University and the college students’ innovation and entrepreneurship training program.

Stimulated Raman adiabatic passage (STIRAP) is an important technique to manipulate quantum states in quantum simulation and quantum computation. The transformation fidelity is limited in reality due to experimental imperfections. After systematically calculating the influence of dissipation caused by thermal fluctuations and instantaneous decay of the intermediate state, we find optimized control pulses of Rydberg atom in optical tweezer to increase the STIRAP fidelity via optimal control method. All constraints of currently available control lasers have been taken into account. The transition error can be further depressed when control lasers with shorter rise time and accordingly proper total evolution time are applied. Finally, the robustness of the control pulses with respect to random deviations between the theoretical pulse shape and the implemented ones is also enhanced by additional rounds of optimizations based on ensemble averaged fidelity.

Bioimage analysis workflows: community resources to navigate through a complex ecosystem.

Workflows are the keystone of bioimage analysis, and the NEUBIAS (Network of European BioImage AnalystS) community is trying to gather the actors of this field and organize the information around them. One of its most recent outputs is the opening of the F1000Research NEUBIAS gateway, whose main objective is to offer a channel of publication for bioimage analysis workflows and associated resources. In this paper we want to express some personal opinions and recommendations related to finding, handling and developing bioimage analysis workflows. The emergence of "big data" in bioimaging and resource-intensive analysis algorithms make local data storage and computing solutions a limiting factor. At the same time, the need for data sharing with collaborators and a general shift towards remote work, have created new challenges and avenues for the execution and sharing of bioimage analysis workflows. These challenges are to reproducibly run workflows in remote environments, in particular when their components come from different software packages, but also to document them and link their parameters and results by following the FAIR principles (Findable, Accessible, Interoperable, Reusable) to foster open and reproducible science. In this opinion paper, we focus on giving some directions to the reader to tackle these challenges and navigate through this complex ecosystem, in order to find and use workflows,and to compare workflows addressing the same problem. We also discuss tools to run workflows in the cloud and on High Performance Computing resources, and suggest ways to make these workflows FAIR.