High-performance Computational Grid Research Articles

Performance-aware energy-efficient parallel job scheduling in HPC grid using nature-inspired hybrid meta-heuristics

High-Performance Computing (HPC) systems offer massive computation strength to execute large-scale applications. However, the availability of thousands of CPU cores in the HPC Systems has also triggered a significant increase in the associated energy consumption translating to higher energy expenses of system providers and carbon emissions in the environment. Therefore efficient job schedulers, which can trade-off between user-desired performance and conflicting energy-efficiency objectives simultaneously, are the need of the hour and must nowadays. Job scheduling in HPC systems is a known NP-Hard problem for which meta-heuristics may provide a near-to-optimal solution. Cuckoo search (CS) is a well-known robust swarm-intelligence based meta-heuristic, which has been applied extensively in many optimization problems due to the strong searching efficiency and requirement of very few tuning parameters. However, it suffers from the likelihood of trapping in the local minima and lack of solution diversity towards the end of the algorithm. These drawbacks could result in unacceptable results when the CS algorithm applies to the parallel job scheduling problem. To overcome these limitations and improve the searching efficiency of the traditional CS, we have proposed a multi-objective hybrid scheduling algorithm called MOHCSFA to optimally schedule the batch of parallel jobs in HPC Grid. The proposed MOHCSFA policy combines the solution search mechanisms of both Cuckoo Search (CS) and Firefly algorithm (FA) during each generation. Our proposed policy is further integrated with efficient resource allocation (ERA) heuristic to improve job scheduler performance by effectively using multi-site resource allocation. The experiments are conducted on the GridSim simulator and the benchmarking of the proposed algorithm is done using real data-sets extracted from two supercomputing workload logs. The simulation results showed that the proposed MOHCSFA policy outperforms many heuristics and meta-heuristic scheduling policies for different test cases for both performance and energy-efficiency objectives. Specifically, in the case of Unilu-Gaia workloads, the MOHCSFA obtained 5.87–24.05%, 3.46–28.50%, and 7.06–26.76% performance improvement for the makespan, energy consumption and avg. flowtime, respectively over other tested scheduling policies. The statistical tests validated the stability and robustness of the proposed policy over other scheduling policies.

High Performance Computing in Resource Poor Settings: An Approach based on Volunteer Computing

High Performance Computing (HPC) systems aim to solve complex computing problems (in a short amount of time) that are either too large for standard computers or would take too long. They are used to solve computational problems in many fields such as medical science (for drug discovery, breast cancer detection in images, etc.), climate science, physics, mathematical science, etc. Existing solutions such as HPC Supercomputer, HPC Cluster, HPC Cloud or HPC Grid are not adapted for resource poor settings (mainly for developing countries) because their fees are generally beyond the funding (particularly for academics) and the administrative complexity to access to HPC Grid creates a higher barrier. This paper presents an approach allowing to build a Volunteer Computing system for HPC in resource poor settings. This solution does not require any additional investment in hardware, but relies instead on voluntary machines already owned by the private users. The experiment has been made on the mathematical problem of solving the matrices multiplication using Volunteer Computing system. Given the success of this experiment, the enrollment of other volunteers has already started. The goal being to create a powerful Volunteer Computing system with the maximum number of computers.

Reducing the number of response time service level objective violations by a cloud‐HPC convergence scheduler

SummaryJob scheduling is an old topic in High‐Performance Computing (HPC), and it is more and more studied in data centers. Large data centers are often split into separate partitions for cloud computing and HPC; each partition normally has its specific scheduler. The possibility of migrating jobs from the HPC partition to the cloud one is a topic widely discussed in the literature. However, job migration from cloud to HPC is a much less explored topic. Nevertheless, such migration may be useful in many situations, in particular when the HPC platform has a low resource usage level, and the cloud usage level is high. A large number of jobs that could migrate from the cloud to the HPC partition may be observed in Google data center workloads. Job scheduling using overbooking strategy is seen as the main reason for the high resource usage level in clouds. However, overbooking can lead to a high rate of rescheduling and job dumping, which potentially causes response time violations. This work shows that HPC platforms can host and execute some cloud jobs with low interference in HPC jobs and a low number of response time violations. We introduce the definition of a cloud‐HPC convergence area and propose a job scheduling strategy for it, aiming at reducing the number of response time violations of cloud jobs without interfering with HPC jobs execution. Our proposal is formally defined and then evaluated in different execution scenarios, using the SimGrid simulation framework, with workload data from production HPC grid. The experimental results show that often, there is a large number of empty areas in the scheduling plan of HPC platforms, which makes it possible to allocate cloud jobs by backfilling. This is due to the sparse HPC job submission pattern and the low resource usage level in some HPC platforms. One performed simulation scenario considered a set of 11K parallel HPC jobs running on a 2560‐processor platform having an average resource usage level of 38.0%. The proposed convergence scheduler succeeded to inject around 267K cloud jobs in the HPC platform, with a response time violation rate under 0.00094% for such jobs, considering 80 processors in the convergence area and no effects on the HPC workload. This experiment considered cloud jobs based on job features of Google public cloud workloads, with a processing time slack factor of 1.25 (which is considered as high priority in the Google cloud SLA—Service Level Agreement). Usually, most cloud jobs show a slack factor higher than 1.25 (most cloud jobs are medium or low priority). The same simulation, repeated with a higher slack factor (4), showed no response time violations.

Orchestrating computational algebra components into a high-performance parallel system

This paper demonstrates that it is possible to obtain good, scalable parallel performance by coordinating multiple instances of unaltered sequential computational algebra systems in order to deliver a single parallel system. The paper presents the first substantial parallel performance results for SymGrid-Par, a system that orchestrates computational algebra components into a high-performance parallel application. We show that SymGrid-Par is capable of exploiting different parallel/multicore architectures without any change to the computational algebra component. Ultimately, our intention is to extend our system so that it is capable of orchestrating heterogeneous computations across a high-performance computational grid.

Calibration of a spatial simulation model with volunteered geographical information

For many scientific disciplines, the continued progression of information technology has increased the availability of data, computation and analytical methodologies including simulation and visualisation. Geographical information science is no exception. In this article, we investigate the possibilities for deployment of e-infrastructures to inform spatial planning, analysis and policy-making. We describe an existing architecture that feeds both static and dynamic simulation models from a variety of sources, including not only administrative datasets but also attitudes and behaviours which are harvested online from crowds. This infrastructure also supports visualisation and computationally intensive processing. The main aim of this article is to illustrate how spatial simulation models can be calibrated with crowd-sourced data. We introduce an example in which popular attitudes to congestion charging in a major UK city (Manchester) were collected, with promotional support from a high-profile media organisation (the BBC). These data are used to estimate the parameters of a transport simulation model, using a hungry estimation procedure which is deployed within a high-performance computational grid. We indicate how the resulting model might be used to evaluate the impact of alternative policy options for regulating the traffic in Manchester. Whilst the procedure is novel in itself, we argue that greater credibility could be added by the incorporation of open-source simulation models and by the use of social networking mechanisms to share policy evaluations much more widely.

ChemInform Abstract: Hydra: A Self Regenerating High Performance Computing Grid for Drug Discovery.

AbstractChemInform is a weekly Abstracting Service, delivering concise information at a glance that was extracted from about 200 leading journals. To access a ChemInform Abstract of an article which was published elsewhere, please select a “Full Text” option. The original article is trackable via the “References” option.

Communication issues within high performance computing grids

This paper presents several ideas pertaining to desirable properties associated with protocols for moving large amounts of data over a grid of high performance computers. The protocol enhancements are discussed in terms of their scalability and identifying sources of potential protocol redundancy. In the context of this paper it is scalability as it relates to the transport of increasingly larger amounts of data over grid computing networks. Specifically we address efficacy in moving large data sets between nodes on a grid interconnected over a Wide Area Network (WAN) in terms of increasing the basic packet size and better suited to transport protocols. In addition we introduce a means of better evaluating error detection models as functions of increased packet size.

Data management and transfer in high-performance computational grid environments

An emerging class of data-intensive applications involve the geographically dispersed extraction of complex scientific information from very large collections of measured or computed data. Such applications arise, for example, in experimental physics, where the data in question is generated by accelerators, and in simulation science, where the data is generated by supercomputers. So-called Data Grids provide essential infrastructure for such applications, much as the Internet provides essential services for applications such as e-mail and the Web. We describe here two services that we believe are fundamental to any Data Grid: reliable, high-speed transport and replica management. Our high-speed transport service, GridFTP, extends the popular FTP protocol with new features required for Data Grid applications, such as striping and partial file access. Our replica management service integrates a replica catalog with GridFTP transfers to provide for the creation, registration, location, and management of dataset replicas. We present the design of both services and also preliminary performance results. Our implementations exploit security and other services provided by the Globus Toolkit.