Computational Grid Resources Research Articles

A hybrid heuristic of Variable Neighbourhood Descent and Great Deluge algorithm for efficient task scheduling in Grid computing

To achieve the ultimate success of global collaborative resource sharing in Grid computing, an effective and efficient Grid resource management system is necessary and it is only possible if its core component, the scheduler, can perform scheduling in an efficient manner. Scheduling tasks to resources in Grid computing is a challenging task and known as a NP hard problem. In this paper, we propose a novel hybrid heuristic-based algorithm, which synergised the excellent diversification capability of Great Deluge (GD) algorithm with the powerful systematic multi-neighbourhood search strategy captured in Variable Neighbourhood Descent (VND) algorithm, to efficiently schedule independent tasks in Grid computing environment with an objective of minimising the makespan. Simulation experiments have been conducted to examine the impact of hybridising GD and VND. In addition, the performance of the proposed algorithm has been evaluated and compared with some other recent meta-heuristics in the literature. The experimental simulation results show that our proposed algorithm outperforms the other algorithms in the literature and the performance improvement achieved by this hybrid strategy is effective and efficient with respect to makespan and computational time as it can obtain good quality (makespan) of solutions while obviating the drawback of requiring high computational cost from the VND.

Hierarchically distributed peer-to-peer architecture for load balancing and effective dynamic group scheduling in grid computing

The local or remote grid computing resources have been normally assigned the task of executing user jobs in a distributed environment. An important factor affecting grid processing is the network bandwidth that differs from network to network. In order to overcome this problem, a few typical algorithms are suggested keeping in view of computing grid resources based on load balancing across the grid system. These algorithms help in improving the performance to some extent, yet none of them has been found experimentally suitable to overcome the major short falls experienced in the grid computing system. A hierarchically distributed peer-to-peer (HDP2P) architecture is proposed to overcome all major problems encountered due to virtuality and heterogeneity. Results suggested that HDP2P architecture was of great use in balancing the load and allocating to the resources to the groups of jobs keeping MIPS accountability using grouping-based dynamic job scheduling algorithm (GBDJS). The proposed algorithm is compared with first come first served (FCFS) job scheduling algorithm and ant colony optimisation (ACO). There is a considerable improvement achieved in performance by using this novel architecture.

Hierarchically distributed Peer-to-Peer architecture for load balancing and effective dynamic group scheduling in Grid computing

The local or remote grid computing resources have been normally assigned the task of executing user jobs in a distributed environment. An important factor affecting grid processing is the network bandwidth that differs from network to network. In order to overcome this problem, a few typical algorithms are suggested keeping in view of computing grid resources based on load balancing across the grid system. These algorithms help in improving the performance to some extent, yet none of them has been found experimentally suitable to overcome the major short falls experienced in the grid computing system. A hierarchically distributed peer-to-peer (HDP2P) architecture is proposed to overcome all major problems encountered due to virtuality and heterogeneity. Results suggested that HDP2P architecture was of great use in balancing the load and allocating to the resources to the groups of jobs keeping MIPS accountability using grouping-based dynamic job scheduling algorithm (GBDJS). The proposed algorithm is compared with first come first served (FCFS) job scheduling algorithm and ant colony optimisation (ACO). There is a considerable improvement achieved in performance by using this novel architecture.

A Parallelized Dynamic Task Scheduling for Batch of Task in a computational grid

ABSTRACTThe emerging computational grid (CG) system is used for effective utilization of geographically dispersed computing resources in order to fulfill the compute-intensive requirement of the users. Mapping and scheduling of a Batch of Task (BoT) on CG resources have been proved to be NP-complete. In this paper, we propose a Parallelized Dynamic Task Scheduling (PDTS) approach for BoT to minimize job completion time and improving resource utilization by enhancing task level parallelism. The algorithm works in two phases; the first phase divides the task into subtasks to exploit parallelism while in the second phase when the task is available in the memory for execution, it allocates it to the most suitable and available resources dynamically. A simulation study has been carried out to evaluate PDTS by comparing with other heuristic approaches with similar objectives by using the GridSim toolkit. The obtained empirical results show that PDTS outperforms than other considered scheduling strategies in all the cases under study.

New Algorithms for Data Mining on Grid Computing

Data Mining on the grid is a new challenge which consists of processing a large quantity of data by knowledge discovery methods. New methods which are able to take into consideration all the specificities of data Mining problems as well as those of grid computing (data grid) are needed for a better exploitation of the treatment of available resources in grid computing. In this paper, we have proposed a method for the extraction of association rules, adapted for parallel and distributed environment such as grids. The goal here is to minimize the execution time by reducing the cost of parallelization and communication. A parallel version of the Partition algorithm for finding association rules has been proposed. The introduction of an intelligent distribution of the base on the grid using clustering to minimize the search space in each machine and consequently reduce the processing time has been considered. To this end, another parallel clustering algorithm for treating numerical and categorical data in a distributed environment has been proposed.

DYNAMIC JOB SCHEDULING IN GRID COMPUTING

Grid computing is growing rapidly in the distributed heterogeneous systems for utilizing and sharing large-scale resources to solve complex scientific problems. Scheduling is the most recent topic used to achieve high performance in grid environments. It aims to find a suitable allocation of resources for each job. A typical problem which arises during this task is the decision of scheduling. It is about an effective utilization of processor to minimize tardiness time of a job, when it is being scheduled. Scheduling jobs to resources in grid computing is complicated due to the distributed and heterogeneous nature of the resources. The efficient scheduling of independent jobs in a heterogeneous computing environment is an important problem in domains such as grid computing. In general, finding optimal schedule for such an environment using the traditional sequential method is an NP-hard problem whereas heuristic approaches will provide near optimal solutions for complex problems. The Ant colony algorithm, which is one of the heuristic algorithms, suits well for the grid scheduling environment using stigmeric communication.

Enabling Grid Computing resources within the KM3NeT computing model

KM3NeT is a future European deep-sea research infrastructure hosting a new generation neutrino detectors that – located at the bottom of the Mediterranean Sea – will open a new window on the universe and answer fundamental questions both in particle physics and astrophysics. International collaborative scientific experiments, like KM3NeT, are generating datasets which are increasing exponentially in both complexity and volume, making their analysis, archival, and sharing one of the grand challenges of the 21st century. These experiments, in their majority, adopt computing models consisting of different Tiers with several computing centres and providing a specific set of services for the different steps of data processing such as detector calibration, simulation and data filtering, reconstruction and analysis. The computing requirements are extremely demanding and, usually, span from serial to multi-parallel or GPU-optimized jobs. The collaborative nature of these experiments demands very frequent WAN data transfers and data sharing among individuals and groups. In order to support the aforementioned demanding computing requirements we enabled Grid Computing resources, operated by EGI, within the KM3NeT computing model. In this study we describe our first advances in this field and the method for the KM3NeT users to utilize the EGI computing resources in a simulation-driven use-case.

How much higher can HTCondor fly?

The HTCondor high throughput computing system is heavily used in the high energy physics (HEP) community as the batch system for several Worldwide LHC Computing Grid (WLCG) resources. Moreover, it is the backbone of GlidelnWMS, the pilot system used by the computing organization of the Compact Muon Solenoid (CMS) experiment. To prepare for LHC Run 2, we probed the scalability limits of new versions and configurations of HTCondor with a goal of reaching 200,000 simultaneous running jobs in a single internationally distributed dynamic pool.In this paper, we first describe how we created an opportunistic distributed testbed capable of exercising runs with 200,000 simultaneous jobs without impacting production. This testbed methodology is appropriate not only for scale testing HTCondor, but potentially for many other services. In addition to the test conditions and the testbed topology, we include the suggested configuration options used to obtain the scaling results, and describe some of the changes to HTCondor inspired by our testing that enabled sustained operations at scales well beyond previous limits.

BIGhybrid: a simulator for MapReduce applications in hybrid distributed infrastructures validated with the Grid5000 experimental platform

SummaryCloud computing has increasingly been used as a platform for running large business and data processing applications. Conversely, Desktop Grids have been successfully employed in a wide range of projects, because they are able to take advantage of a large number of resources provided free of charge by volunteers. A hybrid infrastructure created from the combination of Cloud and Desktop Grids infrastructures can provide a low‐cost and scalable solution for Big Data analysis. Although frameworks like MapReduce have been designed to exploit commodity hardware, their ability to take advantage of a hybrid infrastructure poses significant challenges because of their large resource heterogeneity and high churn rate. In this paper, BIGhybrid is proposed, a simulator for two existing classes of MapReduce runtime environments: BitDew‐MapReduce designed for Desktop Grids and BlobSeer‐Hadoop designed for Cloud computing, where the goal is to carry out accurate simulations of MapReduce executions in a hybrid infrastructure composed of Cloud computing and Desktop Grid resources. This work describes the principles of the simulator and describes the validation of BIGhybrid with the Grid5000 experimental platform. Owing to BIGhybrid, developers can investigate and evaluate new algorithms to enable MapReduce to be executed in hybrid infrastructures. This includes topics such as resource allocation and data splitting. Copyright © 2015 John Wiley & Sons, Ltd.

Minimizing Energy Consumption in Scheduling of Dependent Tasks using Genetic Algorithm in Computational Grid

Energy consumption by large computing systems has become an important research theme not only because the sources of energy are depleting fast but also due to the environmental concern. Computational grid is a huge distributed computing platform for the applications that require high end computing resources and consume enormous energy to facilitate execution of jobs. The organizations which are offering services for high end computation, are more cautious about energy consumption and taking utmost steps for saving energy. Therefore, this paper proposes a scheduling technique for Minimizing Energy consumption using Adapted Genetic Algorithm (MiE-AGA) for dependent tasks in Computational Grid (CG). In MiE-AGA, fitness function formulation for energy consumption has been mathematically formulated. An adapted genetic algorithm has been developed for minimizing energy consumption with appropriate modifications in each components of original genetic algorithm such as representation of chromosome, crossover, mutation and inversion operations. Pseudo code for MiE-AGA and its components has been developed with appropriate examples. MiE-AGA is simulated using Java based programs integrated with GridSim. Analysis of simulation results in terms of energy consumption, makespan and average utilization of resources clearly reveals that MiE-AGA effectively optimizes energy, makespan and average utilization of resources in CG. Comparative analysis of the optimization performance between MiE-AGA and the state-of-the-arts algorithms: EAMM, HEFT, Min-Min and Max-Min shows the effectiveness of the model.

ATLAS FTK Challenge: Simulation of a Billion-fold Hardware Parallelism

Abstract During the current LHC shutdown period the ATLAS experiment will upgrade the Trigger and Data Acquisition system to include a hardware tracker coprocessor: the Fast TracKer (FTK). The FTK receives data from the 80 million of channels of the ATLAS silicon detector, identifying charged tracks and reconstructing their parameters at a rate of up to 100 KHz and within 100 microseconds. To achieve this performance, the FTK system identifies candidate tracks utilizing the computing power of a custom ASIC chip with associative memory (AM) designed to perform “pattern matching” at very high speed; track parameters are then calculated using modern FPGAs. A detailed simulation of this massive system has been developed with the goal of supporting the hardware design and studying its impact in the ATLAS online event selection at high LHC luminosities. We present the issues related to emulating FTK on a general-purpose CPU platform, using ATLAS computing Grid resources, and the solutions developed in order to mitigate these problems and allow the studies required to support the system design, construction and installation.

An improved approach for load balancing among heterogeneous resources in computational grids

In recent days, due to the rapid technological advancements, the grid computing has become an important area of research in distributed systems. The load balancing is a very important and complex problem in grid computing. In this paper, we propose a dynamic-distributed load-balancing technique called the improved load balancing on enhanced GridSim with deadline control (IEGDC) for computational grids. Here, we provide a new mechanism of scheduling to enhance the utilization of the resources and to prevent the resource overloading. A selection method for scheduling by considering the state of resource bandwidth and capacity of various resources is presented. We simulate the proposed load-balancing strategy on the GridSim platform. The proposed mechanism on comparison is found to outperform the existing schemes in terms of response time, resubmitted time, finished and unfinished Gridlets. The simulation results are presented.

Combined performance and availability analysis of distributed resources in grid computing

This paper presents a mathematical model to evaluate the performance of grid resources when availability of the resources is taken into account. The proposed model uses continuous time Markov chains (CTMCs) to model the failure-repair behavior of a grid resource. In grid computing environment, a resource not only may fail during task execution, but also it can cancel its membership at any time. Hence, the proposed CTMC considers the availability of a grid resource to a grid user in both failure and membership refusal situations. After modeling the availability of the resource, the mean sojourn time of grid tasks in each of the availability states is estimated. Assigning the mean sojourn times of the tasks as performance levels to each of the CTMC's states, a Markov reward model (MRM) representing the combined performance and availability measures is obtained. Computing the cumulative state probability of the CTMC and multiplying reward rates of the MRM's states to each of the corresponding state probabilities, the expected accumulated sojourn time of grid tasks in each of the grid resources is achieved. An illustrative example is presented and the results obtained from the proposed model are reported in cases where various scheduling disciplines are considered inside the grid resource to simultaneously service grid and local tasks.

Auction-Based Resource Allocation in Computational Grids

Auction techniques have been becoming the key methods in grid resource allocation. In this paper, we propose the multi-attribute reverse auction models for computational grid resource based on the designed satisfaction degree function. The fairness is also introduced into the models and protocols to match the reality. Two auction-based protocols are presented, i.e., the offline multi-attribute auction-based protocol and the on-line multi-attribute auction-based protocol. The goal is to help the resource user make multi-attribute decisions with complete information or incomplete information. Results from simulation experiments show that these two protocols can achieve market efficiency and fairness in most instances.

RESOURCE SELECTION IN GRID ENVIRONMENT BASED ON TRUST EVALUATION USING FEEDBACK AND PERFORMANCE

A grid computing is a collection of distributed computing resources owned by many organizations across boundaries and not dedicated for a single user available in the Virtual Organization. Grid computing environment provides more computational capabilities to increase the scalability and efficiency of the infra-structure. Resource selection in computational grid is the open issue for executing complex jobs. Trust is the one of the important factor for selecting the resources in computational grid. We introduced a new technique for finding the trusted resource. In our proposed method, we computed the trusted resource from user’s feedback and performance. While selecting the resource, overall trustworthiness is compared. The result of the resource selection algorithm is that most trust resource weight is above the threshold level. In our proposed algorithm job is submitted to the trust resource available in grid.

Reward-based allocation of cluster and grid resources for imprecise computation model-based applications

Utility-based resource management is becoming an emerging issue as the utilisation of cluster resources in Grid computing is growing rapidly. In this paper, we provide a new Imprecise Computation IC application model for flexible reward-based Grid resource management. An application in the proposed model consists of multiple independent jobs, in which each job has two parts: mandatory part for the minimum quality and optional part for additional computations. This application model can be applied to QoS-related Grid applications and used in adaptive resource management. We also provide scheduling algorithms for resource allocation of the IC applications based on reward. The profitable optional execution time is analysed for both space-shared and time-shared scheduling policies. Simulation results show that the proposed schemes are beneficial to both users and resource providers in terms of application acceptance rate and total reward.

On Fault Tolerance of Resources in Computational Grids

Grid computing or computational grid is always a vast research field in academic, as well as in industry also. Computational grid provides resource sharing through multi-institutional virtual organizations for dynamic problem solving. Various heterogeneous resources of different administrative domain are virtually distributed through different network in computational grids. Thus any type of failure can occur at any point of time and job running in grid environment might fail. Hence fault tolerance is an important and challenging issue in grid computing as the dependability of individual grid resources may not be guaranteed. In order to make computational grids more effective and reliable fault tolerant system is necessary. The objective of this paper is to review different existing fault tolerance techniques applicable in grid computing. This paper presents state of the art of various fault tolerance technique and comparative study of the existing algorithms.

Realistic Modeling of Resources for a Large-Scale Computational Grid System

To build a large-scale computational grid resource model with realistic characteristics, this paper proposes a statistical remedy to approximate the distributions of computational and communicational abilities of resources. After fetching the source data of resources including computing devices and networks from the real world, the gamma and normal distributions are employed to approximate probabilities for the ability distribution of grid resources. With this method, researchers can supply the computational grid simulators with required parameters relating to computational and communicational abilities. The experiment result shows that proposed methodology can lead to a better precision according to the error measurement of the simulated data and its source. The proposed method can support existing modern grid simulators with a high-precision resource model which can consider the characteristics of distribution for its computational and communicational abilities in the grid computing environment.

High availability and efficient energy consumption for cloud computing service with grid infrastructure

The main services in cloud computing are SaaS (Software as a Service), PaaS (Platform as a Service) and IaaS (Infrastructure as a Service). Among these services, server virtualization falls into IaaS which is a service for lowering server maintenance costs. Generally, the primary aim of server virtualization has been to improve system performance by replacing multiple servers with a single server with better performance. But it carries the problem of increased cost, needed for the installation of new servers for server virtualization.In this paper, grid infrastructure is used for server virtualization in which existing servers are used rather than bringing in new servers. Server virtualization service is provided using scheduling algorithms for distributed servers or resources in grid computing. To overcome potential performance limitations that come from using existing servers, mathematical models of Meta and Sleep Servers under the grid infrastructure environment are used to provide server virtualization service with high availability.

MSV3d: database of human MisSense variants mapped to 3D protein structure

The elucidation of the complex relationships linking genotypic and phenotypic variations to protein structure is a major challenge in the post-genomic era. We present MSV3d (Database of human MisSense Variants mapped to 3D protein structure), a new database that contains detailed annotation of missense variants of all human proteins (20 199 proteins). The multi-level characterization includes details of the physico-chemical changes induced by amino acid modification, as well as information related to the conservation of the mutated residue and its position relative to functional features in the available or predicted 3D model. Major releases of the database are automatically generated and updated regularly in line with the dbSNP (database of Single Nucleotide Polymorphism) and SwissVar releases, by exploiting the extensive Décrypthon computational grid resources. The database (http://decrypthon.igbmc.fr/msv3d) is easily accessible through a simple web interface coupled to a powerful query engine and a standard web service. The content is completely or partially downloadable in XML or flat file formats.Database URL: http://decrypthon.igbmc.fr/msv3d