High Performance Computing Service Research Articles

HPC Services to Characterize Genetic Mutations through Cloud Computing Federations

One child in every 200 births may be affected by one of the approximately 6,000 monogenic diseases discovered so far. Establishing the pathogenicity of the mutations detected with NGS (Next-Generation Sequencing) techniques, within the sequence of the associated genes, will allow Precision Medicine concept to be developed. However, sometimes the clinical significance of the mutations detected in a genome may be uncertain (VUS, Variant of Uncertain Significance) which prevents the development of health measures devoted to personalize individuals' treatments. A VUS pathogenicity can be inferred thanks to evidences obtained from specific types of NGS studies. Therefore the union of supercomputing through HPC (High-Performance Computing) tools and the cloud computing paradigm (HPCC), within a Data Center federation environment offers a solution to develop and provide services to infer the pathogenicity of a set of VUS detected in a genome, while guaranteeing both the security of the information generated during the whole workflow and its availability.

Emergent Order in the Kagome Ising Magnet Dy$_{3}$Mg$_{2}$Sb$_{3}$O$_{14}$

Work at Cambridge was supported through the Winton Programme for the Physics of Sustainability. The work of J.A.M.P., X.B. and M.M. and facilities at Georgia Tech were supported by the College of Sciences through M.M. start-up funds. J.A.M.P. gratefully acknowledges Churchill College, Cambridge for the provision of a Junior Research Fellowship. H.S.O. acknowledges a Teaching Scholarship (Overseas) from the Ministry of Education, Singapore. J.O.H. is grateful to the Engineering and Physical Sciences Research Council (EPSRC) for funding. C.C. was supported by EPSRC Grant No. EP/G049394/1, and the EPSRC NetworkPlus on ‘Emergence and Physics far from Equilibrium’. Experiments at the ISIS Pulsed Neutron and Muon Source were supported by a beamtime allocation from the Science and Technology Facilities Council. This work utilized facilities at the NIST Center for Neutron Research. Monte Carlo simulations were performed using the Darwin Supercomputer of the University of Cambridge High Performance Computing Service (http://www.hpc.cam.ac.uk/) and the ARCHER UK National Supercomputing Service (http://www.archer.ac.uk/, for which access was provided by an ARCHER Instant Access scheme).

Exposing HPC services in the Cloud: the CloudLightning Approach

Nowadays we are noticing important changes in the way High Performance Computing (HPC) providers are dealing with the demand. The growing requirements of modern data- and compute-intensive applications ask for new models for their development, deployment and execution. New approaches related with Big Data, peta- and exa-scale computing are going to dramatically change the design, development and exploitation of highly demanding applications, such as the HPC ones. Due to the increased complexity of these applications and their outstanding requirements which cannot be supported by the classical centralized cloud models, novel approaches, inspired by autonomic computing, are investigated as an alternative. In this paper, we offer an overview of such an approach, undertaken by the CloudLightning initiative. In this context, a novel cloud delivery model that offers the capabilities to describe and deliver dynamic and tailored services is being considered. This new delivery model, based on a self-organizing and self-managing approach, will allow provisioning and delivery of coalitions of heterogeneous cloud resources, built on top of the resources hosted by a cloud service provider.

Automating NEURON Simulation Deployment in Cloud Resources.

Simulations in neuroscience are performed on local servers or High Performance Computing (HPC) facilities. Recently, cloud computing has emerged as a potential computational platform for neuroscience simulation. In this paper we compare and contrast HPC and cloud resources for scientific computation, then report how we deployed NEURON, a widely used simulator of neuronal activity, in three clouds: Chameleon Cloud, a hybrid private academic cloud for cloud technology research based on the OpenStack software; Rackspace, a public commercial cloud, also based on OpenStack; and Amazon Elastic Cloud Computing, based on Amazon's proprietary software. We describe the manual procedures and how to automate cloud operations. We describe extending our simulation automation software called NeuroManager (Stockton and Santamaria, Frontiers in Neuroinformatics, 2015), so that the user is capable of recruiting private cloud, public cloud, HPC, and local servers simultaneously with a simple common interface. We conclude by performing several studies in which we examine speedup, efficiency, total session time, and cost for sets of simulations of a published NEURON model.

Heavy-tailed distribution of the SSH Brute-force attack duration in a multi-user environment

Quite a number of cyber-attacks to be place against supercomputers that provide highperformance computing (HPC) services to public researcher. Particularly, although the secure shell protocol (SSH) brute-force attack is one of the traditional attack methods, it is still being used. Because stealth attacks that feign regular access may occur, they are even harder to detect. In this paper, we introduce methods to detect SSH brute-force attacks by analyzing the server’s unsuccessful access logs and the firewall’s drop events in a multi-user environment. Then, we analyze the durations of the SSH brute-force attacks that are detected by applying these methods. The results of an analysis of about 10 thousands attack source IP addresses show that the behaviors of abnormal users using SSH brute-force attacks are based on human dynamic characteristics of a typical heavy-tailed distribution.

Brute-force Attacks Analysis against SSH in HPC Multi-user Service Environment

Background/Objectives: The brute-force attack is one of popular cyber security threats in the secure shell (SSH) service environment. The SANS Institute has warned about SSH brute-force attacks against remote services. Methods/Statistical Analysis: We describe two brute-force attack detection methods are applied in the High Performance Computing (HPC) service environment which has been operated by KISTI in KOREA. The first way parses failed authentication logs of systems. The second way analyze dropped events of network firewalls. Findings: We analyze SSH brute-force attacks that are detected applying these methods in our service environment. The analysis results show that SSH brute-force attacks are classified ‘1:N’ or ‘N:1’ types of attack between source and destination IP address. And a duration of attacks that is generally the time it takes to execute attacks keeps enough long times. Improvements/Applications: Two detection methods which are deployed in our HPC multi-user service environment are complementary to each other. These methods will be also able to apply for other service environment.

DESIGN OF RESEARCH-SPECIFIC CLOUD SYSTEM FOR HIGH-PERFORMANCE COMPUTING OVER CLOUD SERVICE

Traditional high-performance computing (HPC) services for scientific applications offer high performance and advantages such as large-scale task execution, concurrency, load balancing, and system utilization. However, they fail to provide any scalability due to their inflexible architectures. Therefore, to provide scalability as well as cloud advantages, “HPC over Cloud services” has become a major area of interest in the field of HPC research. To provide a HPC over Cloud service, any of three representative base architectures can be adopted. These are bare metal (BM)-based, container-based, and virtual machine (VM)-based. The results of a study which compared these three base architectures indicated that the container- and BM-based architectures offer almost the same levels of performance. In this study, we set out to design a research-specific cloud system for HPC over Cloud service. As part of our efforts, we build a test-bed system which provides a real container-based HPC over Cloud service for evaluation. The results of our experiments indicate that the proposed system architecture for a container-based HPC over Cloud service can be successfully adopted for practical applications.

Energy consumption model over parallel programs implemented on multicore architectures

In High Performance Computing, energy consump-tion is becoming an important aspect to consider. Due to the high costs that represent energy production in all countries it holds an important role and it seek to find ways to save energy. It is reflected in some efforts to reduce the energy requirements of hardware components and applications. Some options have been appearing in order to scale down energy use and, con-sequently, scale up energy efficiency. One of these strategies is the multithread programming paradigm, whose purpose is to produce parallel programs able to use the full amount of computing resources available in a microprocessor. That energy saving strategy focuses on efficient use of multicore processors that are found in various computing devices, like mobile devices. Actually, as a growing trend, multicore processors are found as part of various specific purpose computers since 2003, from High Performance Computing servers to mobile devices. However, it is not clear how multiprogramming affects energy efficiency. This paper presents an analysis of different types of multicore-based architectures used in computing, and then a valid model is presented. Based on Amdahl’s Law, a model that considers different scenarios of energy use in multicore architectures it is proposed. Some interesting results were found from experiments with the developed algorithm, that it was execute of a parallel and sequential way. A lower limit of energy consumption was found in a type of multicore architecture and this behavior was observed experimentally.

Self-service for software development projects and HPC activities

This contribution describes how CERN has implemented several essential tools for agile software development processes, ranging from version control (Git) to issue tracking (Jira) and documentation (Wikis). Running such services in a large organisation like CERN requires many administrative actions both by users and service providers, such as creating software projects, managing access rights, users and groups, and performing tool-specific customisation. Dealing with these requests manually would be a time-consuming task. Another area of our CERN computing services that has required dedicated manual support has been clusters for specific user communities with special needs. Our aim is to move all our services to a layered approach, with server infrastructure running on the internal cloud computing infrastructure at CERN. This contribution illustrates how we plan to optimise the management of our of services by means of an end-user facing platform acting as a portal into all the related services for software projects, inspired by popular portals for open-source developments such as Sourceforge, GitHub and others. Furthermore, the contribution will discuss recent activities with tests and evaluations of High Performance Computing (HPC) applications on different hardware and software stacks, and plans to offer a dynamically scalable HPC service at CERN, based on affordable hardware.

Business models of high performance computing centres in higher education in Europe

High performance computing (HPC) service centres are a vital part of the academic infrastructure of higher education organisations. However, despite their importance for research and the necessary high capital expenditures, business research on HPC service centres is mostly missing. From a business perspective, it is important to find an answer to the question of “on what business models do HPC centres run”. By means of an interview series at university HPC centres, this study describes how business models of HPC centres in higher education work. Interviews and visits to these HPC sites have led to the finding that four generic types of business models exist: library, shareholder, cost centre, and industry collaboration model; each of which run on different revenue streams like subsidies, selling shares, pay for use, and revenue from industrial partners, respectively. In practice, most university HPC centres combine these generic types and run on a hybrid model. The assessment of the advantages and disadvantages of the business models can help university HPC centres’ directors, school administration and governance to design a reasonable business model.

Virtual Fire: A web-based GIS platform for forest fire control

Α web-based Geographic Information Systems (GIS) platform – named Virtual Fire – for forest fire control has been developed to easily, validly and promptly share and utilize information and tools among firefighting forces. This state-of-the-art system enables fire management professionals to take advantage of GIS capabilities without needing to locally install complex software components. Fire management professionals can locate fire service vehicles and other resources online and in real-time. Fire patrol aircrafts and vehicles may use tracking devices to send their coordinates directly to the platform. Cameras can augment these data by transmitting images of high-risk areas into the graphical interface of the system. Furthermore, the system provides the geographical representation of fire ignition probability and identifies high-risk areas at different local regions daily, based on a high performance computing (HPC) pilot application that runs on Windows HPC Server. Real-time data from remote automatic weather stations and weather maps based on a weather forecasting system provide vital weather data needed for fire prevention and early warning. By using these methods and a variety of fire management information and tools, the end-users are given the ability to design an operational plan to encompass the forest fire, choosing the best ways to put the fire out within the proper recourses and time.

A unified framework for the deployment, exposure and access of HPC applications as services in clouds

Clouds have provided on-demand, scalable and affordable High Performance Computing (HPC) resources to discipline (e.g., Biology, Medicine, Chemistry) scientists. However, the steep learning curve of preparing a HPC cloud and deploying HPC applications has hindered many scientists to achieve innovative discoveries for which HPC resources must be relied on. With the world moving to web-based tools, scientists are also seeking more web-based technologies to support their research. Unfortunately, the discipline problems of high-performance computational research are both unique and complex, which make the development of web-based tools for this research difficult. This paper presents our work on developing a unified cloud framework that allows discipline users to easily deploy and expose HPC applications in public clouds as services. To provide a proof of concept, we have implemented the cloud framework prototype by integrating three components: (i) Amazon EC2 public cloud for providing HPC infrastructure, (ii) a HPC service software library for accessing HPC resources, and (iii) the Galaxy web-based platform for exposing and accessing HPC application services. This new approach can reduce the time and money needed to deploy, expose and access discipline HPC applications in clouds.

Pion: A Problem Solving Environment for Parallel Multivariate Integration

ParInt is a package for parallel multivariate numerical integration. This paper describes the design and implementation of a problem solving environment based on ParInt and Web technology. We call it ParInt ONline (Pion). It facilitates both common end-users and experts to solve computationally intensive numerical integration in parallel. No parallel programming experience or any knowledge of Unix/Linux operating systems is needed for the users. When the user submits an integration problem to Pion via a Web browser, the problem solving environment will compile the integrand function and link it dynamically with the ParInt package so that the execution can be done in parallel on high performance computing servers. The system was designed to be a globally accessible integration platform that operates as a black box, taking user data and producing the results.