HPC Infrastructure Research Articles

Scaling up statistical model checking of cyber-physical systems via algorithm ensemble and parallel simulations over HPC infrastructures

Model-based formal verification of industry-relevant Cyber-Physical Systems (CPSs) is often a computationally prohibitive task. In most cases, the complexity of the models precludes any prospect of symbolic analysis, leaving numerical simulation as the only viable option. Unfortunately, exhaustive simulation of a CPS model over the entire set of plausible operational scenarios is rarely possible in practice, and alternative strategies such as Statistical Model Checking (SMC) must be used instead.In this article, we show that the number of model simulations (samples) required by SMC techniques to converge can be significantly reduced by considering multiple (an ensemble of) Adaptive Stopping Algorithms (SAs) at once, and that the simulations themselves (by far the most expensive step of the entire workload) can be efficiently sped up by exploiting massively parallel platforms.With three industry-scale CPS models, we experimentally show that the use of an ensemble of two state-of-the-art SAs (AA and EBGStop) may require dozens of millions fewer samples when compared to running a single algorithm, with reductions in sample size of up to 78%. Furthermore, we show that our implementation, by massively parallelizing system model simulations on a HPC infrastructure, yields speed-ups for the completion time of the verification tasks which are practically linear with respect to the number of computational nodes, thus achieving an efficiency of virtually 100%, even on very large platforms. This makes it possible to complete tasks of model-based SMC verification for complex CPSs in a matter of hours or days, whereas a naïve sequential execution would require from months to many years.

Analysis on Privacy Preserving Clustering Methods for Big Datasets Using Random Number Generators

Due to the growing requirement for powerful processing capabilities, safeguarding and processing sensitive data in big data systems has become an increasingly prevalent issue. Additional effort is required to secure sensitive data on a system with numerous connections, each of which has its own privacy policy, and to exchange cryptographic keys in accordance with the appropriate protocols for each of these parties. Data encrypted using homomorphic encryption algorithms can undergo the same kinds of arithmetic operations as data encrypted using plaintext. This allows for the outsourcing of computations to cloud services while also providing data privacy by processing in the encrypted domain. Also, key exchange sessions become easier for everyone involved because of this. This study presents new privacy-preserving clustering algorithms and homomorphic encryption systems that may together operate on a shared HPC infrastructure, like the cloud. Therefore, it is not necessary for the participants in this system to have a lot of processing power, as the system would distribute the most power-intensive jobs to any provider of cloud computing. Multiple clustering techniques can have their distance matrices computed in a way that does not compromise user privacy using our technology.

Analysis on Privacy Preserving Clustering Methods for Big Datasets Using Random Number Generators

Due to the growing requirement for powerful processing capabilities, safeguarding and processing sensitive data in big data systems has become an increasingly prevalent issue. Additional effort is required to secure sensitive data on a system with numerous connections, each of which has its own privacy policy, and to exchange cryptographic keys in accordance with the appropriate protocols for each of these parties. Data encrypted using homomorphic encryption algorithms can undergo the same kinds of arithmetic operations as data encrypted using plaintext. This allows for the outsourcing of computations to cloud services while also providing data privacy by processing in the encrypted domain. Also, key exchange sessions become easier for everyone involved because of this. This study presents new privacy-preserving clustering algorithms and homomorphic encryption systems that may together operate on a shared HPC infrastructure, like the cloud. Therefore, it is not necessary for the participants in this system to have a lot of processing power, as the system would distribute the most power-intensive jobs to any provider of cloud computing. Multiple clustering techniques can have their distance matrices computed in a way that does not compromise user privacy using our technology.

Integrating FTS in the Fenix HPC Infrastructure

As compute requirements in experimental high-energy physics are expected to significantly increase, there is a need for leveraging high-performance computing (HPC) resources. However, HPC systems are currently organised and operated in a way that this is not easily possible. Here we will focus on a specific e-infrastructure that incorporates HPC resources, namely Fenix, which is based on a consortium of 6 leading European supercomputing centres. Fenix was initiated through the Human Brain Project (HBP) but also provides resources to other research communities in Europe. The Fenix sites are integrated into a common AAI and provide a so-called Archival Data Repository that can be accessed through a Swift API. In this paper, we report on our efforts to realise a data transfer service that allow to exchange data with the Fenix e-infrastructure. This has been enabled by implementing support of Swift in FTS3 and related software components. We will, in particular, discuss how FTS3 has been integrated into the Fenix AAI, which largely follows the architectural principles of the European Open Science Cloud (EOSC). Furthermore, we show how end-users can use this service through a WebFTS service that has been integrated into the science gateway of the HBP, which is also known as the HBP Collaboratory. Finally, we discuss how transfer commands can be automatically distributed over several FTS3 instances to optimise transfer between different Fenix sites.

A perspective on large-scale simulation as an enabler for novel biorobotics applications.

Our understanding of the complex mechanisms that power biological intelligence has been greatly enhanced through the explosive growth of large-scale neuroscience and robotics simulation tools that are used by the research community to perform previously infeasible experiments, such as the simulation of the neocortex's circuitry. Nevertheless, simulation falls far from being directly applicable to biorobots due to the large discrepancy between the simulated and the real world. A possible solution for this problem is the further enhancement of existing simulation tools for robotics, AI and neuroscience with multi-physics capabilities. Previously infeasible or difficult to simulate scenarios, such as robots swimming on the water surface, interacting with soft materials, walking on granular materials etc., would be rendered possible within a multi-physics simulation environment designed for robotics. In combination with multi-physics simulation, large-scale simulation tools that integrate multiple simulation modules in a closed-loop manner help address fundamental questions around the organization of neural circuits and the interplay between the brain, body and environment. We analyze existing designs for large-scale simulation running on cloud and HPC infrastructure as well as their shortcomings. Based on this analysis we propose a next-gen modular architecture design based on multi-physics engines, that we believe would greatly benefit biorobotics and AI.

Maximizing the potential of numerical weather prediction models: lessons learned from combining high-performance computing and cloud computing

Abstract. To promote cloud and HPC computing, GRAPEVINE* project objectives include using these tools along with open data sources to provide a reusable IT service. In this service a predictive model based on Machine learning (ML) techniques is created with the aim of preventing and controlling grape vine diseases in the wine cultivation sector. Aside from the predictive ML, meteorological forecasts are crucial input to train the ML models and on a second step to be used as input for the operational prediction of grapevine diseases. To this end, the Weather and Research Forecasting model (WRF) has been deployed in CESGA's HPC infrastructure to produce medium-range and sub-seasonal forecasts for the targeted pilot areas (Greece and Spain). The data assimilation component of WRF – WRFDA – has been also introduced for improving the initial conditions of the WRF model by assimilating observations from weather stations and satellite precipitation products (Integrated Multi-satellitE Retrieval for GPM – IMERG). This methodology for assimilation was developed during STARGATE* project, allowing the testing of the methodology in the operational service of GRAPEVINE. The operational production of the forecasts is achieved by the cloudify orchestrator on a Kubernetes cluster. The connections between the Kubernetes cluster and the HPC infrastructure, where the model resides, is achieved with the croupier plugin of cloudify. Blueprints that encapsule the workflows of the meteorological model and its dependencies were created. The instances of the blueprints (deployments) were created automatically to produce operationally weather forecasts and they were made available to the ML models via a THREDDS server. Valuable lessons were learned with regards the automation of the process and the coupling with the HPC in terms of reservations and operational production.

Serverless High-Performance Computing over Cloud

Abstract HPC clouds may provide fast access to fully configurable and dynamically scalable virtualized HPC clusters to address the complex and challenging computation and storage-intensive requirements. The complex environmental, software, and hardware requirements and dependencies on such systems make it challenging to carry out our large-scale simulations, prediction systems, and other data and compute-intensive workloads over the cloud. The article aims to present an architecture that enables HPC workloads to be serverless over the cloud (Shoc), one of the most critical cloud capabilities for HPC workloads. On one hand, Shoc utilizes the abstraction power of container technologies like Singularity and Docker, combined with the scheduling and resource management capabilities of Kubernetes. On the other hand, Shoc allows running any CPU-intensive and data-intensive workloads in the cloud without needing to manage HPC infrastructure, complex software, and hardware environment deployments.

A Collective Intelligence Platform to Support Older Cancer Survivors: Towards the Definition of LifeChamps System and Big Data Reference Architecture.

Within the most recent years, most of the cancer patients are older age, which implies the necessity to a better understanding of aging and cancer connection. This work presents the LifeChamps solution built on top of cutting-edge Big Data architecture and HPC infrastructure concepts. An innovative architecture was envisioned supported by the Big Data Value Reference Model and answering the system requirements from high to low level and from logical to physical perspective, following the "4+1 architectural model".

Deploying and Optimizing Embodied Simulations of Large-Scale Spiking Neural Networks on HPC Infrastructure.

Simulating the brain-body-environment trinity in closed loop is an attractive proposal to investigate how perception, motor activity and interactions with the environment shape brain activity, and vice versa. The relevance of this embodied approach, however, hinges entirely on the modeled complexity of the various simulated phenomena. In this article, we introduce a software framework that is capable of simulating large-scale, biologically realistic networks of spiking neurons embodied in a biomechanically accurate musculoskeletal system that interacts with a physically realistic virtual environment. We deploy this framework on the high performance computing resources of the EBRAINS research infrastructure and we investigate the scaling performance by distributing computation across an increasing number of interconnected compute nodes. Our architecture is based on requested compute nodes as well as persistent virtual machines; this provides a high-performance simulation environment that is accessible to multi-domain users without expert knowledge, with a view to enable users to instantiate and control simulations at custom scale via a web-based graphical user interface. Our simulation environment, entirely open source, is based on the Neurorobotics Platform developed in the context of the Human Brain Project, and the NEST simulator. We characterize the capabilities of our parallelized architecture for large-scale embodied brain simulations through two benchmark experiments, by investigating the effects of scaling compute resources on performance defined in terms of experiment runtime, brain instantiation and simulation time. The first benchmark is based on a large-scale balanced network, while the second one is a multi-region embodied brain simulation consisting of more than a million neurons and a billion synapses. Both benchmarks clearly show how scaling compute resources improves the aforementioned performance metrics in a near-linear fashion. The second benchmark in particular is indicative of both the potential and limitations of a highly distributed simulation in terms of a trade-off between computation speed and resource cost. Our simulation architecture is being prepared to be accessible for everyone as an EBRAINS service, thereby offering a community-wide tool with a unique workflow that should provide momentum to the investigation of closed-loop embodiment within the computational neuroscience community.

Analyzing the Performance of the S3 Object Storage API for HPC Workloads

The line between HPC and Cloud is getting blurry: Performance is still the main driver in HPC, while cloud storage systems are assumed to offer low latency, high throughput, high availability, and scalability. The Simple Storage Service S3 has emerged as the de facto storage API for object storage in the Cloud. This paper seeks to check if the S3 API is already a viable alternative for HPC access patterns in terms of performance or if further performance advancements are necessary. For this purpose: (a) We extend two common HPC I/O benchmarks—the IO500 and MD-Workbench—to quantify the performance of the S3 API. We perform the analysis on the Mistral supercomputer by launching the enhanced benchmarks against different S3 implementations: on-premises (Swift, MinIO) and in the Cloud (Google, IBM…). We find that these implementations do not yet meet the demanding performance and scalability expectations of HPC workloads. (b) We aim to identify the cause for the performance loss by systematically replacing parts of a popular S3 client library with lightweight replacements of lower stack components. The created S3Embedded library is highly scalable and leverages the shared cluster file systems of HPC infrastructure to accommodate arbitrary S3 client applications. Another introduced library, S3remote, uses TCP/IP for communication instead of HTTP; it provides a single local S3 gateway on each node. By broadening the scope of the IO500, this research enables the community to track the performance growth of S3 and encourage sharing best practices for performance optimization. The analysis also proves that there can be a performance convergence—at the storage level—between Cloud and HPC over time by using a high-performance S3 library like S3Embedded.

FOSquare: A Novel Optical HPC Interconnect Network Architecture Based on Fast Optical Switches with Distributed Optical Flow Control

Interconnecting networks adopting Fast Optical Switches (FOS) can achieve high bandwidth, low latency, and low power consumption. We propose and demonstrate a novel interconnecting topology based on FOS (FOSquare) with distributed fast flow control which is suitable for HPC infrastructures. We also present an Optimized Mapping (OPM) algorithm that maps the most communication-related processes inside a rack. We numerically investigate and compare the network performance of FOSquare with Leaf-Spine under real traffic traces collected by running multiple applications (CG, MG, MILC, and MINI_MD) in an HPC infrastructure. The numerical results show that the FOSquare can reduce >10% latency with respect to Leaf-Spine under the scenario of 16 available cores.

MILP, Pseudo-Boolean, and OMT Solvers for Optimal Fault-Tolerant Placements of Relay Nodes in Mission Critical Wireless Networks*

In critical infrastructures like airports, much care has to be devoted in protecting radio communication networks from external electromagnetic interference. Protection of such mission-critical radio communication networks is usually tackled by exploiting radiogoniometers: at least three suitably deployed radiogoniometers, and a gateway gathering information from them, permit to monitor and localise sources of electromagnetic emissions that are not supposed to be present in the monitored area. Typically, radiogoniometers are connected to the gateway through relay nodes. As a result, some degree of fault-tolerance for the network of relay nodes is essential in order to offer a reliable monitoring. On the other hand, deployment of relay nodes is typically quite expensive. As a result, we have two conflicting requirements: minimise costs while guaranteeing a given fault-tolerance. In this paper, we address the problem of computing a deployment for relay nodes that minimises the overall cost while at the same time guaranteeing proper working of the network even when some of the relay nodes (up to a given maximum number) become faulty (fault-tolerance). We show that, by means of a computation-intensive pre-processing on a HPC infrastructure, the above optimisation problem can be encoded as a 0/1 Linear Program, becoming suitable to be approached with standard Artificial Intelligence reasoners like MILP, PB-SAT, and SMT/OMT solvers. Our problem formulation enables us to present experimental results comparing the performance of these three solving technologies on a real case study of a relay node network deployment in areas of the Leonardo da Vinci Airport in Rome, Italy.

Desynchronization of simulation and optimization algorithms in HPC Environment

Need for scalability of an algorithm is essential, when one wants to utilize HPC infrastructure in an efficient and reasonable way. In such infrastructures, synchronization affects the efficiency of the parallel algorithms. However, one can consider introducing certain means of desynchronization in order to increase scalability. Allowing for omitting or delaying certain messages, can be easily accepted in the case of metaheuristics. Furthermore, some simulations can also follow this pattern and handle bigger environments. The paper presents a short survey of desynchronization idea, pointing out already obtained results or sketching out the future work focused on scaling the parallel and distributed computing or simulation algorithms leveraging desynchronization.

LAITOR4HPC: A text mining pipeline based on HPC for building interaction networks

BackgroundThe amount of published full-text articles has increased dramatically. Text mining tools configure an essential approach to building biological networks, updating databases and providing annotation for new pathways. PESCADOR is an online web server based on LAITOR and NLProt text mining tools, which retrieves protein-protein co-occurrences in a tabular-based format, adding a network schema. Here we present an HPC-oriented version of PESCADOR’s native text mining tool, renamed to LAITOR4HPC, aiming to access an unlimited abstract amount in a short time to enrich available networks, build new ones and possibly highlight whether fields of research have been exhaustively studied.ResultsBy taking advantage of parallel computing HPC infrastructure, the full collection of MEDLINE abstracts available until June 2017 was analyzed in a shorter period (6 days) when compared to the original online implementation (with an estimated 2 years to run the same data). Additionally, three case studies were presented to illustrate LAITOR4HPC usage possibilities. The first case study targeted soybean and was used to retrieve an overview of published co-occurrences in a single organism, retrieving 15,788 proteins in 7894 co-occurrences. In the second case study, a target gene family was searched in many organisms, by analyzing 15 species under biotic stress. Most co-occurrences regarded Arabidopsis thaliana and Zea mays. The third case study concerned the construction and enrichment of an available pathway. Choosing A. thaliana for further analysis, the defensin pathway was enriched, showing additional signaling and regulation molecules, and how they respond to each other in the modulation of this complex plant defense response.ConclusionsLAITOR4HPC can be used for an efficient text mining based construction of biological networks derived from big data sources, such as MEDLINE abstracts. Time consumption and data input limitations will depend on the available resources at the HPC facility. LAITOR4HPC enables enough flexibility for different approaches and data amounts targeted to an organism, a subject, or a specific pathway. Additionally, it can deliver comprehensive results where interactions are classified into four types, according to their reliability.

Conformational Dynamics and Energetics of Melittin and its Diastereomer Interacting with POPC and POPG Lipid Bilayers: A Molecular Dynamics Study

Often amphipathic membrane-active peptides assume helical secondary structure when partitioning into lipid membranes. Characterizing this membrane-induced folding process at single molecule level is fundamental to understanding peptide lipid-membrane interactions. To this end, we use all-atom, long time scale (∼1µs) molecular dynamics (MD) simulations to investigate and compare the conformational dynamics and energetics of the well-studied peptide melittin (MWT) and one of its less-studied diastereomer, with four D-amino acids, (MD4) interacting respectively with a zwitterionic POPC and an anionic POPG lipid bilayer. Unlike in previous MD studies, we start from a completely folded conformation of the peptides and follow their unfolding through free MD simulations. We find that (i) both MWT and MD4 unfold in solution and remain mostly disordered; (ii) MWT binds to both POPC and POPG, remaining mostly folded (with more than 70% helical content); and (iii) MD4 binds only to POPG and it is mostly disordered (with less than 40% helical content). Furthermore, by calculating the potential of mean force, as a function of peptide-lipid bilayer separation, we have determined the relative dissociation energy of MWT and MD4 from POPG, which can be related to the membrane induced folding energy of melittin. To our knowledge, this is the first MD study of the MD4 melittin analog, and the obtained results are in good agreement with previous experimental studies. Work supported in part by a grant from the MU Research Board. The computation for this work was performed on the high HPC infrastructure provided by RCSS and in part by the NSF under grant number CNS-1429294 at the University of Missouri, Columbia MO.

Deployment and Analysis of a Hybrid Shared/Distributed-Memory Parallel Visualization Tool for 3-D Oil Reservoir Grid on OpenStack Cloud Computing

The main goal of oil reservoir management is to provide more efficient, price-effective and environmentally more secure oil production. Oil production management includes an accurate characterization of the reservoir and strategies that involve interactions between reservoir data and human assessment. Hence, it is important to graphically visualize and handle massive data sets of oil and gas pressure / saturation levels to help decision makers in statistical analysis, history matching and recovery of hydrocarbons of the reservoir. In this article, we experimentally study the parallelization of intensive computation for a 3-D (three dimensional) oil reservoir data visualization tool. For this tool, we develop and implement a transformation and lighting model to visualize and react with the grid. Herein, we propose a hybrid (shared memory and distributed memory) parallelization technique to adapt with the data processing scalability. We tested these implementations on OpenStack Cloud Virtual Cluster. Our results indicate that although the virtual platform adds overhead for running parallel implementations, utilizing knowledge of the VM location on the compute host and network traffic among VMs to deploy the virtual environment can achieve significant performance enhancements. Hybrid parallel implementation using large data size can achieve $70\times $ speedup over serial execution without owning a costly HPC infrastructure as the conventional parallel processing deployment model.

LincoSim: a Web Based HPC-Cloud Platform for Automatic Virtual Towing Tank Analysis

Thanks to evolving web technologies, computational platforms, automation tools and open-source software business model, today, it is possible to develop powerful automatic and virtualized web services for complex physical problems in engineering and design. In particular, in this work, we are presenting a new web based HPC-cloud platform for automatic virtual towing tank analysis. It is well known that the design project of a new hull requires a continuous integration of shape hypothesis and hydrodynamics verifications using analytical tools, 3D computational methods, experimental facilities and sea keeping trial tests. The complexity and the cost of such design tools increase considerably moving from analytical tools to sea keeping trials. In order to perform a meaningful trade-off between costs and high quality data acquiring, during the last decade the usage of 3D computational models has grown pushed also by well-known technological factors. Nevertheless, in the past, there were several limiting factors on the wide diffusion of 3D computational models to perform virtual towing tank data acquiring. On one hand software licensing and hardware infrastructure costs, on the other hand the need of very specific technological skills limited the usage of such virtualized tools only to research centers and/or to large industrial companies. In this work we propose an innovative high-level approach which is embodied in the so-called LincoSim [17] web application in which a hypothetical designer user can carry out the simulation only starting from its own geometry and a set of meaningful physical parameters. LincoSim automatically manages and hides to the user all the necessary details of Computational Fluid Dynamics (CFD) modelling and of HPC infrastructure usage allowing them to access, visualize and analyze the outputs from the same single access point made up from the web browser. In addition to the web interface, the platform includes a back-end server which implements a Cloud logic and can be connected to multiple HPC machines for computing. LincoSim is currently set up with finite volume Open-FOAM CFD engine. A preliminary validation campaign has been performed to assess the robustness and the reliability of the tool and is proposed as a novel approach for the development of Computer Aided Engineering (CAE) applications.

System Level Optimization for High-Speed SerDes: Background and the Road Towards Machine Learning Assisted Design Frameworks

This decade has witnessed wide use of data-driven systems, from multimedia to scientific computing, and in each case quality data movement infrastructure is required, many with SerDes as a cornerstone. On the one hand, HPC and machine learning cloud infrastructure carry exabytes of data in a year through the backplanes of data centers. On the other hand, the growing need for edge computing in the IoT places a tight envelope on the energy per bits. In this survey, we give a system level overview of the common design challenges in implementing SerDes solutions under different scenarios and propose simulation methods benefiting from advanced machine learning techniques. Preliminary results with the proposed simulation platform are demonstrated and analyzed through machine learning based design methodologies.

Distributed ant colony optimization based on actor model

The parallelization of metaheuristics and care for the efficient use of the available infrastructure is very popular in the case of population-based algorithms (e.g., evolutionary ones), as many of them have structures intrinsically easy for parallelization. However, swarm computing algorithms (ACO in particular) must use certain global knowledge in order to be properly implemented (e.g., a pheromone matrix in the case of ACO algorithms). Thus, the parallelization of ACO is known to be difficult to realize. In this paper, we propose an actor-based approach for constructing an efficient and robust ACO implementation that leverages the HPC infrastructure. The presented results show the ability to be scaled for up to 30 nodes, and the relevant results support the claim that the implemented algorithm is equal to the original Ant System algorithm. Improving it further and increasing its scalability with the planned asynchrony in the pheromone matrix updates is envisioned as a direct future work.

Тенденции развития вычислительных узлов современных суперкомпьютеров

This work includes analysis of the computation nodes of modern supercomputers from two perspectives; first the hardware components focus and secondly discuss the infrastructure. Identified trends leads to basic options of computation node design. The paper classifies the modern architectures of universal processing cores and specialized hardware accelerators cores; studies the recent trends in memory hierarchy design and intra-node interconnect; the paper includes ways of using the non-volatile memory in modern memory hierarchy. Furthermore, the paper analyses the recent trends in HPC infrastructure, in particular in modern liquid cooling approaches and monitoring. The basic variants of HPC computing nodes design are based on energy efficient universal processor and set of energy-efficient specialized hardware accelerators cores, according to the observed trends. The paper focuses on recent technologies that are currently at various stages of production or at the functional prototype stage. The study also discusses state-of-the-art computational challenges and algorithms-to-architecture mapping issues. Lastly, the paper discusses the current technological problems and main areas to maintain the progress in HPC area.