Large-scale Computational Grid Research Articles

A job scheduling algorithm based on parallel workload prediction on computational grid

Generally, the computational grid consists of a large number of computing nodes, some of them are idle due to the uneven geographical distribution of computing requirements. This may cause workload unbalancing problems, which affect the performance of large-scale computational grids. In order to balance the computing requirements and computing nodes, we propose a job scheduling algorithm based on the workload prediction of computing nodes. We first analyze the causes of workload imbalance and the feasibility of reallocating computing resources. Secondly, we design an application and workload-aware scheduling algorithm (AWAS) by combining the previously designed workload prediction model. To reduce the complexity of the AWAS algorithm, we propose a parallel job scheduling method based on computing node workload prediction. The experiments show that the AWAS algorithm can balance the workload among different computing nodes on the real-world dataset. In addition, we propose the parallelism of workload prediction model from the perspective of internal structure and data set to make AWAS apply to more computing nodes of the large-scale computing grids. Experimental results show that the combination of the two can achieve satisfactory acceleration efficiency.

Application-aware deadline constraint job scheduling mechanism on large-scale computational grid.

Recently, computational Grids have proven to be a good solution for processing large-scale, computation intensive problems. However, the heterogeneity, dynamics of resources and diversity of applications requirements have always been important factors affecting their performance. In response to these challenges, this work first builds a Grid job scheduling architecture that can dynamically monitor Grid computing center resources and make corresponding scheduling decisions. Second, a Grid job model is proposed to describe the application requirements. Third, this paper studies the characteristics of commercial interconnection networks used in Grids and forecast job transmission time. Fourth, this paper proposes an application-aware job scheduling mechanism (AJSM) that includes periodic scheduling flow and a heuristic application-aware deadline constraint job scheduling algorithm. The rigorous performance evaluation results clearly demonstrate that the proposed application-aware job scheduling mechanism can successful schedule more Grid jobs than the existing algorithms. For successful scheduled jobs, our proposed AJSM method is the best algorithm for job average processing time and makespan.

Evaluation and comparison of performance of SDSM and CLIMGEN models in simulation of climatic variables in Qazvin plain

Climate change is found to be the most important global issue in the 21st century, so to monitor its trend is of great importance. Atmospheric General Circulation Models because of their large scale computational grid are not able to predict climatic parameters on a point scale, so small scale methods should be adapted. Among downscaling methods, statistical methods are used as they are easy to run. Two famous models, ClimGen and SDSM, were studied for daily total precipitation and temperature data in Qazvin station. For this purpose, three steps of models calibration, verification and simulation, in Qazvin station were performed and model performances in terms of similarities in produced data with those using parameters such as root mean square error (RMSE), coefficient of determination (R2) and Nash coefficient (NSE) were assessed. The results in climatic range showed that Climgen outperform in rainfall data generation while SDSM outperforms in simulating average temperatures. However, both models have high potential to simulate temperature and precipitation.

Stability Assessment Metamorphic Approach (SAMA) for Effective Scheduling based on Fault Tolerance in Computational Grid

Grid Computing allows coordinated and controlled resource sharing and problem solving in multi-institutional, dynamic virtual organizations. Moreover, fault tolerance and task scheduling is an important issue for large scale computational grid because of its unreliable nature of grid resources. Commonly exploited techniques to realize fault tolerance is periodic Checkpointing that periodically saves the job’s state. But an inappropriate checkpointing interval prevails to delay in the job execution, and reduces the throughput. With that concern, this paper endeavors to ensure better performance on computational grid with more effective and reliable fault tolerant system using a novel Stability Assessment Metamorphic Approach (SAMA). Here, the strategy used to attain fault tolerance is by adapting the checkpoints depending on the current status and past failure information of the resources dynamically, which is being maintained in the information server. Effective scheduling process can be achieved by fault tolerance based scheduling that involves in determination of deviation rate of all nodes using some high-stability assessment constraints. This evinces the job to be accomplished within the deadline with improved throughput and paves a way for making the grid environment trust worthy.

Realistic Modeling of Resources for a Large-Scale Computational Grid System

To build a large-scale computational grid resource model with realistic characteristics, this paper proposes a statistical remedy to approximate the distributions of computational and communicational abilities of resources. After fetching the source data of resources including computing devices and networks from the real world, the gamma and normal distributions are employed to approximate probabilities for the ability distribution of grid resources. With this method, researchers can supply the computational grid simulators with required parameters relating to computational and communicational abilities. The experiment result shows that proposed methodology can lead to a better precision according to the error measurement of the simulated data and its source. The proposed method can support existing modern grid simulators with a high-precision resource model which can consider the characteristics of distribution for its computational and communicational abilities in the grid computing environment.

Locality-Preserving Clustering and Discovery of Resources in Wide-Area Distributed Computational Grids

In large-scale computational Grids, discovery of heterogeneous resources as a working group is crucial to achieving scalable performance. This paper presents a resource management scheme including a hierarchical cycloid overlay architecture, resource clustering and discovery algorithms for wide-area distributed Grid systems. We establish program/data locality by clustering resources based on their physical proximity and functional matching with user applications. We further develop dynamism-resilient resource management algorithm, cluster-token forwarding algorithm, and deadline-driven resource management algorithms. The advantage of the proposed scheme lies in low overhead, fast and dynamism-resilient multiresource discovery. The paper presents the scheme, new performance metrics, and experimental simulation results. This scheme compares favorably with other resource discovery methods in static and dynamic Grid applications. In particular, it supports efficient resource clustering, reduces communications cost, and enhances resource discovery success rate in promoting large-scale distributed supercomputing applications.

Fault Tolerance In Grid Computing: State of the Art and Open Issues

Fault tolerance is an important property for large scale computational grid systems, where geographically distributed nodes co-operate to execute a task. In order to achieve high level of reliability and availability, the grid infrastructure should be a foolproof fault tolerant. Since the failure of resources affects job execution fatally, fault tolerance service is essential to satisfy QOS requirement in grid computing. Commonly utilized techniques for providing fault tolerance are job checkpointing and replication. Both techniques mitigate the amount of work lost due to changing system availability but can introduce significant runtime overhead. The latter largely depends on the length of checkpointing interval and the chosen number of replicas, respectively. In case of complex scientific workflows where tasks can execute in well defined order reliability is another biggest challenge because of the unreliable nature of the grid resources.

An integrated security-aware job scheduling strategy for large-scale computational grids

All existing fault-tolerance job scheduling algorithms for computational grids were proposed under the assumption that all sites apply the same fault-tolerance strategy. They all ignored that each grid site may have its own fault-tolerance strategy because each site is itself an autonomous domain. In fact, it is very common that there are multiple fault-tolerance strategies adopted at the same time in a large-scale computational grid. Various fault-tolerance strategies may have different hardware and software requirements. For instance, if a grid site employs the job checkpointing mechanism, each computation node must have the following ability. Periodically, the computational node transmits the transient state of the job execution to the server. If a job fails, it will migrate to another computational node and resume from the last stored checkpoint. Therefore, in this paper we propose a genetic algorithm for job scheduling to address the heterogeneity of fault-tolerance mechanisms problem in a computational grid. We assume that the system supports four kinds fault-tolerance mechanisms, including the job retry, the job migration without checkpointing, the job migration with checkpointing, and the job replication mechanisms. Because each fault-tolerance mechanism has different requirements for gene encoding, we also propose a new chromosome encoding approach to integrate the four kinds of mechanisms in a chromosome. The risk nature of the grid environment is also taken into account in the algorithm. The risk relationship between jobs and nodes are defined by the security demand and the trust level. Simulation results show that our algorithm has shorter makespan and more excellent efficiencies on improving the job failure rate than the Min–Min and sufferage algorithms.

On the Design of Fault-Tolerant Scheduling Strategies Using Primary-Backup Approach for Computational Grids with Low Replication Costs

Fault-tolerant scheduling is an imperative step for large-scale computational grid systems, as often geographically distributed nodes co-operate to execute a task. By and large, primary-backup approach is a common methodology used for fault tolerance wherein each task has a primary copy and a backup copy on two different processors. In this paper, we identify two cases that may happen when scheduling dependent tasks with primary-backup approach. We derive two important constraints that must be satisfied. Further, we show that these two constraints play a crucial role in limiting the schedulability and overloading efficiency of backups of dependent tasks. We then propose two strategies to improve schedulability and overloading efficiency, respectively. We propose two algorithms (MRC-ECT and MCT-LRC), to schedule backups of independent jobs and dependent jobs, respectively. MRC-ECT is shown to guarantee an optimal backup schedule in terms of replication cost for an independent task, while MCT-LRC can schedule a backup of a dependent task with minimum completion time and less replication cost. We conduct extensive simulation experiments to quantify the performance of the proposed algorithms.

On the design of communication-aware fault-tolerant scheduling algorithms for precedence constrained tasks in grid computing systems with dedicated communication devices

Fault-tolerant scheduling is an imperative step for large-scale computational Grid systems, as often geographically distributed nodes co-operate to execute a task. By and large, primary-backup approach is a common methodology used for fault tolerance wherein each task has a primary and a backup on two different processors. In this paper, we address the problem of how to schedule DAGs in Grids with communication delays so that service failures can be avoided in the presence of processors faults. The challenge is, that as tasks in a DAG have dependence on each other, a task must be scheduled to make sure that it will succeed when any of its predecessors fails due to a processor failure. We first propose a communication model and determine when communications between a backup and backups of its successors are necessary. Then we determine when a backup can start and its eligible processors so as to guarantee that every DAG can complete upon any processor failure. We develop two algorithms to schedule backups, which minimize response time and replication cost, respectively. We also develop a suboptimal algorithm which targets minimizing replication cost while not affecting response time. We conduct extensive simulation experiments to quantify the performance of the proposed algorithms.

Parameter optimization in 3D reconstruction on a large scale grid

Reaching as high structural resolution as possible in 3D electron microscopy of biological specimens is crucial to understanding their function and interactions. Technical and biological limitations make electron microscopy projections of such specimens quite noisy. Under those circumstances, the broadly used Weighted Back-Projection algorithm presents some limitations for 3D reconstruction. Iterative tomographic reconstruction algorithms are well suited to provide high resolution 3D structures under such noisy conditions. Nevertheless, these iterative algorithms present two major challenges: computational expensiveness and some free parameters which need to be correctly tuned to obtain the best possible resolution. This work applies global optimization techniques to search for the optimal set of parameters and makes use of the high-throughput capabilities of grid computing to perform the required computations. Fault tolerance techniques have been included in our application to deal with the dynamic nature and complexity of large scale computational grids. The approach for parameter optimization presented here has been successfully evaluated in the European EGEE grid, obtaining good levels of speedup, throughput and transfer rates.