High Performance Computing Cluster Research Articles

High-speed turbulent flows towards the exascale: STREAmS-2 porting and performance

Exascale High Performance Computing (HPC) represents a tremendous opportunity to push the boundaries of Computational Fluid Dynamics (CFD), but despite the consolidated trend towards the use of Graphics Processing Units (GPUs), programmability is still an issue. STREAmS-2 (Bernardini et al. Comput. Phys. Commun. 285 (2023) 108644) is a compressible solver for canonical wall-bounded turbulent flows capable of harvesting the potential of NVIDIA GPUs. Here we extend the already available CUDA Fortran backend with a novel HIP backend targeting AMD GPU architectures. The main implementation strategies are discussed along with a novel Python tool that can generate the HIP and CPU code versions allowing developers to focus their attention only on the CUDA Fortran backend. Single GPU performance is analysed focusing on NVIDIA A100 and AMD MI250x cards which are currently at the core of several HPC clusters. The gap between peak GPU performance and STREAmS-2 performance is found to be generally smaller for NVIDIA cards. Roofline analysis allows tracing this behavior to unexpectedly different computational intensities of the same kernel using the two cards. Additional single-GPU comparisons are performed to assess the impact of grid size, number of parallelized loops, thread masking and thread divergence. Parallel performance is measured on the two largest EuroHPC pre-exascale systems, LUMI (AMD GPUs) and Leonardo (NVIDIA GPUs). Strong scalability reveals more than 80% efficiency up to 16 nodes for Leonardo and up to 32 for LUMI. Weak scalability shows an impressive efficiency of over 95% up to the maximum number of nodes tested (256 for LUMI and 512 for Leonardo). This analysis shows that STREAmS-2 is the perfect candidate to fully exploit the power of current pre-exascale HPC systems in Europe, allowing users to simulate flows with over a trillion mesh points, thus reducing the gap between the Reynolds numbers achievable in high-fidelity simulations and those of real engineering applications.

Cb-WoFS: Migrating the Warn-on-Forecast System to the Cloud

Abstract This article provides a brief, technical narrative of the WoFS journey to a cloud-based high-performance computing (HPC) system, including some of the technological challenges encountered and solutions found. Also, discussed are a few new components that are in development for cloud-based Warn-on-Forecast System (cb-WoFS), such as the cloud infrastructure project for managing resources, as well as a new web application that manages cb-WoFS runs. An important initial step in our cloud journey is containerizing all of the compiled applications, such as WRF, GSI, and EnKF and their dependencies like netCDF and MPI. With these applications compiled within an Apptainer container, WoFS can run on any local or cloud-based HPC cluster that supports MPI. Furthermore, an additional software layer was developed that creates and manages cloud vendor resources. This layer, which is referred to as the WoFS framework, contains the workflow required to run cb-WoFS, as well as management for other aspects of cb-WoFS (including but not limited to creation of HPC pools in the end-to-end workflow, runtime notifications, and database management). This additional layer was developed to separate the WoFS business logic from vendor-specific API calls. The WoFS framework exposes features through its service library, which is then referenced by the newly developed cb-WoFS web application and other cloud applications. This makes WoFS a complete end-to-end cloud-based application, where an administrator can launch a model run, manage resources, and view output all within a single web app.

Enabling high-performance cloud computing for the Community Multiscale Air Quality Model (CMAQ) version 5.3.3: performance evaluation and benefits for the user community

Abstract. The Community Multiscale Air Quality Model (CMAQ) is a local- to hemispheric-scale numerical air quality modeling system developed by the U.S. Environmental Protection Agency (USEPA) and supported by the Community Modeling and Analysis System (CMAS) center. CMAQ is used for regulatory purposes by the USEPA program offices and state and local air agencies and is also widely used by the broader global research community to simulate and understand complex air quality processes and for computational environmental fate and transport and climate and health impact studies. Leveraging state-of-the-science cloud computing resources for high-performance computing (HPC) applications, CMAQ is now available as a fully tested, publicly available technology stack (HPC cluster and software stack) for two major cloud service providers (CSPs). Specifically, CMAQ configurations and supporting materials have been developed for use on their HPC clusters, including extensive online documentation, tutorials and guidelines to scale and optimize air quality simulations using their services. These resources allow modelers to rapidly bring together CMAQ, cloud-hosted datasets, and visualization and evaluation tools on ephemeral clusters that can be deployed quickly and reliably worldwide. Described here are considerations in CMAQ version 5.3.3 cloud use and the supported resources for each CSP, presented through a benchmark application suite that was developed as an example of a typical simulation for testing and verifying components of the modeling system. The outcomes of this effort are to provide findings from performing CMAQ simulations on the cloud using popular vendor-provided resources, to enable the user community to adapt this for their own needs, and to identify specific areas of potential optimization with respect to storage and compute architectures.

Ichor: A Python library for computational chemistry data management and machine learning force field development.

We present ichor, an open-source Python library that simplifies data management in computational chemistry and streamlines machine learning force field development. Ichor implements many easily extensible file management tools, in addition to a lazy file reading system, allowing efficient management of hundreds of thousands of computational chemistry files. Data from calculations can be readily stored into databases for easy sharing and post-processing. Raw data can be directly processed by ichor to create machine learning-ready datasets. In addition to powerful data-related capabilities, ichor provides interfaces to popular workload management software employed by High Performance Computing clusters, making for effortless submission of thousands of separate calculations with only a single line of Python code. Furthermore, a simple-to-use command line interface has been implemented through a series of menu systems to further increase accessibility and efficiency of common important ichor tasks. Finally, ichor implements general tools for visualization and analysis of datasets and tools for measuring machine-learning model quality both on test set data and in simulations. With the current functionalities, ichor can serve as an end-to-end data procurement, data management, and analysis solution for machine-learning force-field development.

Graphical User Interface for Biodiversity Digital Twins: Data Challenges

Digital Twins are a new concept in the field of biodiversity (de Koning et al. 2023), one aspect is the user interface for interacting with digital twins. In the BioDT (Biodiversity Digital Twin) (BioDT 2022) project we are creating a graphical web interface (Martinovič et al. 2024) using the R Shiny framework, which allows small-scale data analysis to be done directly on the server running the web interface, while making it possible to offload a large-scale analysis to a supercomputer such as LUMI (Large Unified Modern Infrastructure). Additionally, we foresee that multiple prototype Digital Twins (pDTs) will be available as part of one web application. This brings multiple challenges in optimization of data flows for the computation and interaction with users, especially since data used by the pDTs, developed in the BioDT, are usually stored across multiple data sources. In the BioDT web application, we want to enable users to access different pDTs concerned with questions such as ecosystem services, biodiversity dynamics, DNA-related biodiversity tasks, pollinators, invasive species, disease outbreaks, and more. While some of these pDTs compute results at a remote server and provide only the newest results to the application, others aim to allow users to execute their own pDT runs with their own data and settings. This leads to many different user scenarios and impacts authentication and authorization flows, as well as data flows. In addition to this, we want to make the whole system comply with the FAIR (Findable, Accessible, Interoperable, and Reusable) principles as much as possible. For this, we need to support traceability of data, models, and pDT executions. This means that we advocate for data to be hosted at data providers with APIs for machine-to-machine interaction and extensive metadata support; models to be as open as possible and versioned; and to have a workflow execution orchestrator that can keep track of the models' execution and their related inputs and outputs. In terms of data, we are using research infrastructures (RIs) such as GBIF (Global Biodiversity Information Facility) and eLTER (Integrated European Long-Term Ecosystem, critical zone and socio-ecological Research) for the input data streams and when it is not possible to use established RIs, we are using self-hosted services. This work is still in progress and some of the self-hosted datasets are still being used, but in the future, we are looking into the possibility of having a dynamic system that will support data formats that cannot be currently stored at the established RIs. Our colleagues at BioDT consortium are preparing a special data server (El-Gabbas et al. 2023) that can handle complex data processing and serve data through an API without the need to download data for multiple commonly used formats. Some of the BioDT pDTs leverage the computational strength of High-Performance Computing (HPC) clusters and in such cases, classical cloud workflow orchestrators are not an option due to the specific security policies of such centers. To solve this, we turned to the LEXIS Platform, which can execute predefined workflows on a combination of cloud and HPC resources, and track the executions and related execution metadata. We are looking into exporting the descriptions of the workflow executions to Research Object Crates (RO-Crates) and then uploading this information to a remote server, where users could check their execution settings later. A current main development focus is the question of how to tackle the challenge of multiple authentication systems. This is specifically of concern in the case of sensitive data, which need to remain secure and available only to selected people. Due to interactions with several systems, we are encountering authentication issues of multiple different identities that should be recognized as one. The simplest solution here is to use one platform for data storage, workflow execution and web application. However, in the future we hope to find a more general solution that would not require data transfer to a single platform, since this could lower the usage of the BioDT platform due to legal restriction on some data. Another challenge in optimization of the dataflows is to avoid downloading the same data repeatedly and to be able to provide users with relevant data in the web application as fast as possible, since long waiting time would result in people not using the web application. For these issues, we are considering a smart-caching mechanism, however such functionality is not yet defined.

MetaBakery: a Singularity implementation of bioBakery tools as a skeleton application for efficient HPC deconvolution of microbiome metagenomic sequencing data to machine learning ready information.

In this study, we present MetaBakery (http://metabakery.fe.uni-lj.si), an integrated application designed as a framework for synergistically executing the bioBakery workflow and associated utilities. MetaBakery streamlines the processing of any number of paired or unpaired fastq files, or a mixture of both, with optional compression (gzip, zip, bzip2, xz, or mixed) within a single run. MetaBakery uses programs such as KneadData (https://github.com/bioBakery/kneaddata), MetaPhlAn, HUMAnN and StrainPhlAn as well as integrated utilities and extends the original functionality of bioBakery. In particular, it includes MelonnPan for the prediction of metabolites and Mothur for calculation of microbial alpha diversity. Written in Python 3 and C++ the whole pipeline was encapsulated as Singularity container for efficient execution on various computing infrastructures, including large High-Performance Computing clusters. MetaBakery facilitates crash recovery, efficient re-execution upon parameter changes, and processing of large data sets through subset handling and is offered in three editions with bioBakery ingredients versions 4, 3 and 2 as versatile, transparent and well documented within the MetaBakery Users' Manual (http://metabakery.fe.uni-lj.si/metabakery_manual.pdf). It provides automatic handling of command line parameters, file formats and comprehensive hierarchical storage of output to simplify navigation and debugging. MetaBakery filters out potential human contamination and excludes samples with low read counts. It calculates estimates of alpha diversity and represents a comprehensive and augmented re-implementation of the bioBakery workflow. The robustness and flexibility of the system enables efficient exploration of changing parameters and input datasets, increasing its utility for microbiome analysis. Furthermore, we have shown that the MetaBakery tool can be used in modern biostatistical and machine learning approaches including large-scale microbiome studies.

Design of an iterative method for environmental-sustainable development: Integrating bioinspired computing techniques

The need for sustainable development has grown in response to global environmental, social, and economic challenges. Conventional computational methods frequently struggle to address the complex nature of the Sustainable Development Goals (SDGs), lacking the ability to balance global search with local optimization and failing to prioritize goals related to sustainability. To address these restrictions, this work introduces the Integrated Bioinspired Computing Model for Sustainable Development (IBCMSD). By combining Genetic Algorithms (GAs), Artificial Neural Networks (ANNs), and Ant Colony Optimization (ACO), a cohesive hybrid model is developed that improves exploration and exploitation, balance for increased efficiency, and solution quality. It is implemented on High-Performance Computing (HPC) clusters to ensure scalability and resilience when dealing with complicated optimization challenges. Furthermore, using a multidisciplinary co-design method completes the model with multiple views, increasing its relevance and applicability in real-world circumstances. IBCMSD makes a significant contribution to computational sustainability by leveraging bioinspired computing, potentially enabling informed decision-making and SDG accomplishment across multiple domains.

HPC for SAR-MSS Data Fusion Experiments

Satellite image fusion techniques are compute intensive models mainly due to the availability of voluminous temporal data from multiple sensors. The need and support of High Performance Computing clusters in modelling various satellite data fusion techniques is unavoidable as the 5V’s such as volume, velocity, variety, value and veracity of space based Big Data has to be scientifically integrated over a HPC environment. Any such integration is meaningful, only when a suitable domain specific parallel processing algorithm is developed that can extract the much required critical information in near real time or real time mode. In this regard, this article mainly focuses on defining a HPC system environment and corresponding parallel algorithm which together supports the image fusion experiments of various space borne satellite data sets. Emphasis has been given in fusing different frequency, polarization Synthetic Aperture Radar’s, intensity data with Multi Spectral optical satellite data. The well proved and most commonly followed IHS and PAC image fusion techniques are studied and its parallel version of algorithm that can be enabled over the proposed HPC environment are discussed. In the current world scenario, setting up the proposed HPC system and enablement of proposed parallel models are unavoidable, mainly to address the real time requirements of defense and disaster management operations.

Real-time Performance Monitoring of HPC Clusters: Techniques and Challenges

In the era of exascale computing, High-Performance Computing (HPC) clusters have become essential for addressing complex computational challenges across various domains. The increasing scale and complexity of HPC systems pose significant challenges in ensuring optimal performance and resource utilization. Real-time performance monitoring has emerged as a critical requirement to address these challenges effectively. The paper offers a comprehensive analysis of real-time performance monitoring techniques and challenges within HPC clusters focusing on three key monitoring approaches: Prometheus for agent-based monitoring, the ELK (Elasticsearch, Logstash, Kibana) stack for log-based monitoring, and machine learning/artificial intelligence techniques for proactive analysis. The implementation of Prometheus and Node Exporter for real-time metrics collection, the utilization of the ELK stack for log aggregation and analysis, and the integration of ML/AI for anomaly detection and predictive analytics are explored. Challenges such as scalability, heterogeneity, and dynamic workloads are addressed in the context of each monitoring approach. The findings demonstrate that each technique offers unique advantages and trade-offs, providing valuable insights for improving HPC performance monitoring practices. Key Words: High-Performance Computing, real-time performance monitoring, Prometheus, ELK stack, Machine Learning, Artificial Intelligence, HPC clusters.

Alrecon: computed tomography reconstruction web application based on Solara

Synchrotron X-ray computed tomography is a non-destructive 3D imaging technique that offers the possibility to study the internal microstructure of samples with high spatial and temporal resolution. Given its unmatched image quality and acquisition speed, and the possibility to preserve the specimens, there is an increasing demand for this technique, from scientific users from innumerable disciplines. Computed tomography reconstruction is the computational process by which experimental radiographs are converted to a meaningful 3-dimensional image after the scan. The procedure involves pre-processing steps for image background and artifact correction on raw data, a reconstruction step approximating the inverse Radon-transform, and writing of the reconstructed volume image to disk. Several open-source Python packages exist to help scientists in the process of tomography reconstruction, by offering efficient implementations of reconstruction algorithms exploiting central or graphics processing unit (CPU and GPU, respectively), and by automating significant portions of the data processing pipeline. A further increase in productivity is attained by scheduling and parallelizing demanding reconstructions on high performance computing (HPC) clusters. Nevertheless, visual inspection and interactive selection of optimal reconstruction parameters remain crucial steps that are often performed in close interaction with the end-user of the data. As a result, the reconstruction task involves more than one software. Graphical user interfaces are provided to the user for fast inspection and optimization of reconstructions, while HPC resources are often accessed through scripts and command line interface. We propose Alrecon, a pure Python web application for tomographic reconstruction built using Solara. Alrecon offers users an intuitive and reactive environment for exploring data and customizing reconstruction pipelines. By leveraging upon popular 3D image visualization tools, and by providing a user-friendly interface for reconstruction scheduling on HPC resources, Alrecon guarantees productivity and efficient use of resources for any type of beamline user.

Containerized deep learning in agriculture: Orchestrating GoogleNet with Kubernetes on high performance computing

SummarySmart Farming has become a cornerstone of modern agriculture, offering data‐driven insights and automation that optimize resource utilization and increase crop yields. The use of cutting‐edge technologies in agriculture has given rise to Smart Farming, which has transformed traditional farming practices into efficient, data‐driven operations. This paper explores the synergy between high‐performance computing (HPC) systems, Kubernetes orchestration, GoogleNet architecture, and containerization to redefine the future of farming. At the heart of this transformation lies the GoogleNet architecture, a deep learning powerhouse recognized for its efficiency and accuracy in image recognition tasks. The orchestration capabilities of Kubernetes, a versatile tool for managing containerized workloads efficiently on HPC clusters. Hence, in this work, we investigate the intricacies of deploying GoogleNet‐based deep learning models within containerized environments orchestrated by Kubernetes on HPC infrastructure. It explores resource optimization, scalability, security, and adaptability, all tailored to the unique demands of the agricultural domain to evaluate the effectiveness of the given technique it is compared with the existing techniques namely Hermes, Horus, CYBELE, and RZ‐SHAN. The attained ranges of proposed method of various measures of accuracy, precision, recall, and F1‐score are 98.65%, 97.45%, 97.87%, and 98.12% for the Pilot Wheat Ear dataset. Also, the processing time for the proposed approach is 181.50 and 120.2 m for the Pilot Wheat Ear Dataset and the Pilot soya bean farming dataset. The latency of the proposed approach attains a lower value of 1.5 and 1.1 s pilot soya bean farming dataset and Pilot Wheat Ear dataset. The experimental outcome demonstrates the efficiency of the proposed approaches to improve Smart Farming agriculture.

Swift : a modern highly parallel gravity and smoothed particle hydrodynamics solver for astrophysical and cosmological applications

ABSTRACT Numerical simulations have become one of the key tools used by theorists in all the fields of astrophysics and cosmology. The development of modern tools that target the largest existing computing systems and exploit state-of-the-art numerical methods and algorithms is thus crucial. In this paper, we introduce the fully open-source highly-parallel, versatile, and modular coupled hydrodynamics, gravity, cosmology, and galaxy-formation code Swift. The software package exploits hybrid shared- and distributed-memory task-based parallelism, asynchronous communications, and domain-decomposition algorithms based on balancing the workload, rather than the data, to efficiently exploit modern high-performance computing cluster architectures. Gravity is solved for using a fast-multipole-method, optionally coupled to a particle mesh solver in Fourier space to handle periodic volumes. For gas evolution, multiple modern flavours of Smoothed Particle Hydrodynamics are implemented. Swift also evolves neutrinos using a state-of-the-art particle-based method. Two complementary networks of sub-grid models for galaxy formation as well as extensions to simulate planetary physics are also released as part of the code. An extensive set of output options, including snapshots, light-cones, power spectra, and a coupling to structure finders are also included. We describe the overall code architecture, summarize the consistency and accuracy tests that were performed, and demonstrate the excellent weak-scaling performance of the code using a representative cosmological hydrodynamical problem with ≈300 billion particles. The code is released to the community alongside extensive documentation for both users and developers, a large selection of example test problems, and a suite of tools to aid in the analysis of large simulations run with Swift.

Sustainability of Data Center Digital Twins with Reinforcement Learning

The rapid growth of machine learning (ML) has led to an increased demand for computational power, resulting in larger data centers (DCs) and higher energy consumption. To address this issue and reduce carbon emissions, intelligent design and control of DC components such as IT servers, cabinets, HVAC cooling, flexible load shifting, and battery energy storage are essential. However, the complexity of designing and controlling them in tandem presents a significant challenge. While some individual components like CFD-based design and Reinforcement Learning (RL) based HVAC control have been researched, there's a gap in the holistic design and optimization covering all elements simultaneously. To tackle this, we've developed DCRL-Green, a multi-agent RL environment that empowers the ML community to design data centers and research, develop, and refine RL controllers for carbon footprint reduction in DCs. It is a flexible, modular, scalable, and configurable platform that can handle large High Performance Computing (HPC) clusters. Furthermore, in its default setup, DCRL-Green provides a benchmark for evaluating single as well as multi-agent RL algorithms. It easily allows users to subclass the default implementations and design their own control approaches, encouraging community development for sustainable data centers. Open Source Link: https://github.com/HewlettPackard/dc-rl

Alrecon: computed tomography reconstruction web application based on Solara.

Synchrotron X-ray computed tomography is a non-destructive 3D imaging technique that offers the possibility to study the internal microstructure of samples with high spatial and temporal resolution. Given its unmatched image quality and acquisition speed, and the possibility to preserve the specimens, there is an increasing demand for this technique, from scientific users from innumerable disciplines. Computed tomography reconstruction is the computational process by which experimental radiographs are converted to a meaningful 3-dimensional image after the scan. The procedure involves pre-processing steps for image background and artifact correction on raw data, a reconstruction step approximating the inverse Radon-transform, and writing of the reconstructed volume image to disk. Several open-source Python packages exist to help scientists in the process of tomography reconstruction, by offering efficient implementations of reconstruction algorithms exploiting central or graphics processing unit (CPU and GPU, respectively), and by automating significant portions of the data processing pipeline. A further increase in productivity is attained by scheduling and parallelizing demanding reconstructions on high performance computing (HPC) clusters. Nevertheless, visual inspection and interactive selection of optimal reconstruction parameters remain crucial steps that are often performed in close interaction with the end-user of the data. As a result, the reconstruction task involves more than one software. Graphical user interfaces are provided to the user for fast inspection and optimization of reconstructions, while HPC resources are often accessed through scripts and command line interface. We propose Alrecon, a pure Python web application for tomographic reconstruction built using Solara. Alrecon offers users an intuitive and reactive environment for exploring data and customizing reconstruction pipelines. By leveraging upon popular 3D image visualization tools, and by providing a user-friendly interface for reconstruction scheduling on HPC resources, Alrecon guarantees productivity and efficient use of resources for any type of beamline user.

A New Cloud-Native Tool for Pharmacogenetic Analysis.

The advancement of next-generation sequencing (NGS) technologies provides opportunities for large-scale Pharmacogenetic (PGx) studies and pre-emptive PGx testing to cover a wide range of genotypes present in diverse populations. However, NGS-based PGx testing is limited by the lack of comprehensive computational tools to support genetic data analysis and clinical decisions. Bioinformatics utilities specialized for human genomics and the latest cloud-based technologies were used to develop a bioinformatics pipeline for analyzing the genomic sequence data and reporting PGx genotypes. A database was created and integrated in the pipeline for filtering the actionable PGx variants and clinical interpretations. Strict quality verification procedures were conducted on variant calls with the whole genome sequencing (WGS) dataset of the 1000 Genomes Project (G1K). The accuracy of PGx allele identification was validated using the WGS dataset of the Pharmacogenetics Reference Materials from the Centers for Disease Control and Prevention (CDC). The newly created bioinformatics pipeline, Pgxtools, can analyze genomic sequence data, identify actionable variants in 13 PGx relevant genes, and generate reports annotated with specific interpretations and recommendations based on clinical practice guidelines. Verified with two independent methods, we have found that Pgxtools consistently identifies variants more accurately than the results in the G1K dataset on GRCh37 and GRCh38. Pgxtools provides an integrated workflow for large-scale genomic data analysis and PGx clinical decision support. Implemented with cloud-native technologies, it is highly portable in a wide variety of environments from a single laptop to High-Performance Computing (HPC) clusters and cloud platforms for different production scales and requirements.

Global Navigation Satellite System (GNSS) time series and velocities about a slowly convergent margin processed on high-performance computing (HPC) clusters: products and robustness evaluation

Abstract. The Global Navigation Satellite System (GNSS) is a well-known and fundamental tool for crustal monitoring projects and tectonic studies, thanks to its high coverage and the high quality of the data they provide. In particular, at slowly convergent margins, where deformation rates are of the order of a few millimetres per year, GNSS monitoring proves to be beneficial in detecting the diffuse deformation responsible for tectonic stress accrual. Its strength lies in the high precision achieved by GNSS permanent stations, especially when long-term data and stable structures are available at the stations. North-eastern Italy is a tectonically active region located in the northernmost sector of the Adria microplate, slowly converging with the Eurasia plate, characterized by low deformation rates and moderate seismicity. It greatly benefits from continuous and high-precision geodetic monitoring, since it has been equipped with a permanent GNSS network providing real-time data and daily observations over 2 decades. The Friuli Venezia Giulia Deformation Network (FReDNet) was established in the area in 2002 to monitor crustal deformation and contribute to the regional seismic hazard assessment. This paper describes GNSS time series spanning 2 decades of stations located in north-eastern Italy and surroundings as well as the outgoing velocity field. The documented dataset has been retrieved by processing the GNSS observations with the GAMIT/GLOBK software ver10.71, which allows calculation of high-precision coordinate time series, position and velocity for each GNSS station by taking advantage of the high-performance computing resources of the Italian High-Performance Computing Centre (CINECA) clusters. The GNSS observations (raw and standard RINEX – Receiver INdependent EXchange – formats) and the time series estimated with the same procedure are currently daily continued, collected and stored in the framework of a long-term monitoring project. Instead, velocity solutions are intended for annual updates. The time series and velocity field dataset documented here are available on Zenodo (https://doi.org/10.5281/zenodo.8055800, Tunini et al., 2024).

Modeling Microreactor Requirements for High-Performance Computing

The potential deployment of microreactors as a zero-emission source for critical applications within integrated energy systems such as microgrids has been gaining interest in recent years owing to the microreactors’ dispatchable nature, modular design, small site footprint, and carbon-free generation. A particularly high-value but challenging application with rapidly growing demand is in the deployment of high-performance computing (HPC) clusters within microgrids. In this work, a model of a HPC cluster in an energy-diverse microgrid is developed to determine the requirements of a technology-agnostic microreactor deployed for such a challenging application. The minute-resolution simulations revealed that the cluster’s electrical load fluctuation of up to 4.1 MW/min required a fast and responsive load-following capability. When the load-following capability of the microreactor was perturbed, the required microgrid storage capacity associated with having a 0.1 MW/min dispatchable microreactor decreased by two orders of magnitude as compared with load-following solely by energy storage devices, indicating that load-following capability in microreactors is of great value in such applications. The analysis methods described in this work can be extended to other microgrids, other HPC clusters, or other types of challenging applications, and can help microgrid planners in determining the storage size, output capacity, and ramping capabilities of the storage devices required for a given microgrid configuration.

Design and implementation of a scalable high-performance computing (HPC) cluster for omics data analysis: achievements, challenges and recommendations in LMICs.

The advent of high-throughput technologies, including cutting-edge sequencing devices, has revolutionized biomedical data generation and processing. Nevertheless, big data applications require novel hardware and software for parallel computing and management to handle the ever-growing data size and analysis complexity. On-premise, high-performance computing (HPC) is increasingly used in biomedical research for big data stewardship. In this work, we present Tunisia's first high-performance computational infrastructure for omics research. We highlight measurements and recommendations that may help institutions in other low- and middle-income countries that are eager to implement local HPC in facilities for bioinformatics research and omics data analyses.

Rapid Targeted Assembly of the Proteome Reveals Evolutionary Variation of GC Content in Avian Lice.

Nucleotide base composition plays an influential role in the molecular mechanisms involved in gene function, phenotype, and amino acid composition. GC content (proportion of guanine and cytosine in DNA sequences) shows a high level of variation within and among species. Many studies measure GC content in a small number of genes, which may not be representative of genome-wide GC variation. One challenge when assembling extensive genomic data sets for these studies is the significant amount of resources (monetary and computational) associated with data processing, and many bioinformatic tools have not been optimized for resource efficiency. Using a high-performance computing (HPC) cluster, we manipulated resources provided to the targeted gene assembly program, automated target restricted assembly method (aTRAM), to determine an optimum way to run the program to maximize resource use. Using our optimum assembly approach, we assembled and measured GC content of all of the protein-coding genes of a diverse group of parasitic feather lice. Of the 499 426 genes assembled across 57 species, feather lice were GC-poor (mean GC = 42.96%) with a significant amount of variation within and between species (GC range = 19.57%-73.33%). We found a significant correlation between GC content and standard deviation per taxon for overall GC and GC3, which could indicate selection for G and C nucleotides in some species. Phylogenetic signal of GC content was detected in both GC and GC3. This research provides a large-scale investigation of GC content in parasitic lice laying the foundation for understanding the basis of variation in base composition across species.

DIFFERENCE BETWEEN OPENHPC AND HTCONDOR CLUSTER SYSTEMS: IN-DEPTH ANALYSIS

The rapidly developing field of high-performance computing (HPC) requires efficient and scalable solutions to manage extensive computing loads. Although they use different approaches and architectures, OpenHPC and HTCondor are well-known platforms that meet these needs. This article thoroughly analyzes OpenHPC and HTCondor to identify their fundamental differences and work paradigms. OpenHPC is a comprehensive modular structure designed to facilitate the deployment, management, and maintenance of HPC clusters, offering a rich set of pre-integrated HPC software components. Conversely, HTCondor specializes in efficiently planning and managing resource-intensive tasks, using a unique partner selection system for dynamic resource allocation based on job requirements and resource availability. By examining aspects such as system architecture, resource management efficiency, scalability, flexibility, and the user ecosystem, this analysis sheds light on the strengths and weaknesses of each structure. The research aims to provide stakeholders in high-performance computing with the knowledge necessary to make informed decisions regarding the selection and implementation of high-performance computing management systems, ultimately aimed at optimizing the use of computing resources and optimizing research and development workflows.