Heterogeneous Distributed Computing Systems Research Articles

A Time-Series Model for Varying Worker Ability in Heterogeneous Distributed Computing Systems

In this paper, we consider the problem of estimating the time-dependent ability of workers participating in distributed matrix-vector multiplication over heterogeneous clusters. Specifically, we model the workers’ ability as a latent variable and introduce a log-normally distributed working rate as a function of the latent variable with parameters so that the working rate increases as the latent ability of workers increases, and takes positive values only. This modeling is motivated by the need to reflect the impact of time-dependent external factors on the workers’ performance. We estimate the latent variable and parameters using the expectation-maximization (EM) algorithm combined with the particle method. The proposed estimation and inference on the working rates are used to allocate tasks to the workers to reduce expected latency. From simulations, we observe that our estimation and inference on the working rates are effective in reducing expected latency.

Resource Search in HPC Systems using Levy Flights

Parallel applications represented by Directed Acyclic Graphs (DAGs) as Parallel Task Graphs (PTGs) requiring high execution times with large amounts of storage, are executed on High Performance Computing (HPC) Systems such as clusters. For the execution of these applications, a scheduler performs the scheduling and allocation of the resources contained in the HPC System. One of the activities of the scheduler is the search for idle resources that are geographically dispersed in the clusters to schedule and allocate them to the tasks. The search for idle resources in the clusters is a process that consumes time and system resources due to the geographical distances that must be traveled and the repetitive and permanent execution within the system. A sequential search for resources causes the scheduling and allocation of resources in the HPC System to be slowed down and paused, and increases the waiting times of the tasks that remain in the queue. The techniques that shorten the location times of resource and perform more exhaustive searches in dispersed geographic spaces can reduce the times generated by sequential searches. The open access paper: Scheduling in Heterogeneous Distributed Computing Systems Based on Internal Structure of Parallel Tasks Graphs with Meta-Heuristics presents the Array Method, a scheduler for scheduling and allocation resources in an HPC System. Array Method uses a sequential search process for the idle resources that are geographically dispersed in the clusters and save their location and characteristics in an array, which is updated every time idle resources are located in the clusters. Considering the above, this paper presents a search for idle resources using Levy random walks, a technique ´ used for searching resources in large geographical spaces; this technique avoids sequential node-by-node searches of each cluster, and promote short and long range searches over the entire geographical extent of the HPC Systems. To obtain experimental results, sequential resource search algorithm versus Levy random walks with the synthetic loads, different ´ clusters and different numbers of resources per cluster as proposed in the open access paper aforementioned, are used. Obtained results show Levy random walks ´ locates more idle resources in less time, optimizes the times of the resource searches in the clusters and update the array of available resources more frequently. With more idle resources found, the total execution times of the tasks are reduced.

A Time and Reliability Optimization Algorithm for Workflow Scheduling in Heterogeneous Distributed Computing System

In the heterogeneous distributed computing systems, efficient task scheduling is necessary for high performance. System reliability and completion time are generally the two important metrics. In this paper, we proposed an algorithm named Merging and Duplication for Makespan and Reliability (MDMR) which combines task merging and task duplication to achieve single-objective constrained optimization of workflow reliability and completion time. The MDMR algorithm is divided into three phases: task merging, entry task duplication and reliability detection with nonentry task duplication. In the task merging phase, two tasks with a sole parent–child relationship are merged into a single task, then this process is iterated until no tasks satisfy the merging condition, thus simplifying the directed acyclic graph. In the entry task duplication phase, the entry task with the lowest reliability is duplicated on the processors, which are available at that moment to increase the reliability while reducing the workflow completion time. In the reliability detection and nonentry task duplication phase, the current workflow scheduling reliability is first calculated, then the result is directly output if the reliability is higher than the target value, otherwise the nonentry task duplication is executed with the aim of increasing the reliability until the value surpasses the target. According to the experimental results, MDMR dominates the compared algorithms and ensures that the entire workflow achieves the target reliability and shortens the workflow makespan as much as possible.

Hybrid Nearest-Neighbor Ant Colony Optimization Algorithm for Enhancing Load Balancing Task Management

Many computer problems that arise from real-world circumstances are NP-hard, while, in the worst case, these problems are generally assumed to be intractable. Existing distributed computing systems are commonly used for a range of large-scale complex problems, adding advantages to many areas of research. Dynamic load balancing is feasible in distributed computing systems since it is a significant key to maintaining stability of heterogeneous distributed computing systems (HDCS). The challenge of load balancing is an objective function of optimization with exponential complexity of solutions. The problem of dynamic load balancing raises with the scale of the HDCS and it is hard to tackle effectively. The solution to this unsolvable issue is being explored under a particular algorithm paradigm. A new codification strategy, namely hybrid nearest-neighbor ant colony optimization (ACO-NN), which, based on the metaheuristic ant colony optimization (ACO) and an approximate nearest-neighbor (NN) approaches, has been developed to establish a dynamic load balancing algorithm for distributed systems. Several experiments have been conducted to explore the efficiency of this stochastic iterative load balancing algorithm; it is tested with task and nodes accessibility and proved to be effective with diverse performance metrics.

On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems

In recent years, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">coded distributed computing</i> (CDC) has attracted significant attention, because it can efficiently facilitate many delay-sensitive computation tasks against unexpected latencies in distributed computing systems. Despite such a salient feature, many design challenges and opportunities remain. In this paper, we focus on practical computing systems with heterogeneous computing resources, and design a novel CDC approach, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">batch-processing based coded computing</i> (BPCC), which exploits the fact that every computing node can obtain some coded results before it completes the whole task. To this end, we first describe the main idea of the BPCC framework, and then formulate an optimization problem for BPCC to minimize the task completion time by configuring the computation load. Through formal theoretical analyses, extensive simulation studies, and comprehensive real experiments on the Amazon EC2 computing clusters, we demonstrate promising performance of the proposed BPCC scheme, in terms of high computational efficiency and robustness to uncertain disturbances.

Energy-minimized Scheduling of Real-time Parallel Workflows on Heterogeneous Distributed Computing Systems

Today's large-scale parallel workflows are often processed on heterogeneous distributed computing platforms. From an economic perspective, computing resource providers should minimize the cost while offering high service quality. It has become well-recognized that energy consumption accounts for a large part of a computing system's total cost, and timeliness and reliability are two important service indicators. This work studies the problem of scheduling a parallel workflow that minimizes the system energy consumption under the constraints of response time and reliability. We first mathematically formulate this problem as a Non-linear Mixed Integer Programming problem. Since this problem is hard to solve directly, we present some highly-efficient heuristic solutions. Specifically, we first develop an algorithm that minimizes the schedule length while meeting reliability requirement, on top of which we propose a processor-merging algorithm and a slack time reclamation algorithm using a dynamic voltage frequency scaling (DVFS) technique to reduce energy consumption. The processor-merging algorithm tries to turn off some energy-inefficient processors such that energy consumption can be minimized. The DVFS technique is applied to scale down the processor frequency at both processor and task levels to reduce energy consumption. Experimental results on two real-life workflows and extensive synthetic parallel workflows demonstrate their effectiveness.

Artificial Intelligent Load Balance Agent on Network Traffic Across Multiple Heterogeneous Distributed Computing Systems

Distributed computing is a very common term in the field of computer science and also a very general computing paradigm which is being used by almost all the scientific and industrial computing applications. All the information technology applications have based upon the use of the distributed computing. It came on demand day by day with the evolution high speed internet. Distributed services are generally deployed on different servers somewhere at a particular geographical region and its consumers are mostly spread over the world wide. Most commercial application now a days providing its platform to access multiple different distributed computing service to provide a set of different consumer services to their customers like an example of ecommerce giant vendors Amazon and PayTM are providing many consumer services like Shopping, Bill payment, Purchasing travel ticket and Top-up mobile balance etc. on a single platform. Since the needs of customers increases day by day and they want as much as services on a single platform, so there comes up of accessing many distributed services on a single platform over some network. As soon as this kind of computer applications started to come into the commercial market, the number of its users started to increase exponentially over the time. By the time it became very difficult to handle and respond to its various users at the same. There is a high risk for the organizations of losing their customers, if they are unable to provide services with a minimal possible response time. If an application has comparatively less features but its response is fast, then it became a widely used product in the market. Hence almost all the organization invests a separate large amount of money annually on the maintenance of its computer applications to boost its performance and response. Even though there are still many challenges in this field, since organization has a limited computing and networking resources to provide their services to its users which are in millions in number. Hence, it has been always a major area of research for many engineers and scientists from all over the world to implement and deliver an algorithm that would be able to solve all these types of problem and provide the services up to the customer satisfaction. This research paper provides the systematic literaraute review on different and best of the till known load balancing system which is artificial intelligence enough to determine the resource capacity of the host server to allow only permitted number of jobs at the same time which falls under resource constraint of the server.

Scheduling in Heterogeneous Distributed Computing Systems Based on Internal Structure of Parallel Tasks Graphs with Meta-Heuristics

The problem of scheduling parallel tasks graphs (PTGs) represented by directed acyclic graphs (DAGs) in heterogeneous distributed computing systems (HDCSs) is considered an nondeterministic polynomial time (NP) problem due to the diversity of characteristics and parameters, generally opposed, intended to be optimized. The PTGs are scheduled by a scheduler that determines the best location for the sub-tasks that constitute the PTGs and is responsible for allocating the resources of the HDCS to the sub-tasks of the PTGs. To optimize scheduling and allocations, the scheduler extracts characteristics from the internal structure of the PTGs. The prevailing characteristic in existing research is the critical path (CP), which is limited to providing execution paths of PTGs; considering this limitation, we extend the array method proposed in Velarde, which extracts two additional characteristics to the CP: the layering and the density of the graph for scheduling. These characteristics are represented as integer values of the PTGs to be scheduled; the values obtained from the characteristics are stored in arrays representing populations that are evaluated with the heuristic univariate marginal distribution algorithm (UMDA) and in terms of comparison with the genetic algorithm. With the best allocations produced by the algorithms, two performance parameters are evaluated: makespan and waiting time. The results indicate that when more PTGs characteristics are considered, resource allocations are optimized, and scheduling times are reduced. The results obtained with the heuristic algorithms show that UMDA provides shorter scheduling and allocation times compared with the genetic algorithm; UMDA widely distributes the sub-tasks in the clusters, whereas the genetic algorithm compacts the assignments of the PTGs in the clusters with a longer convergence time that translates into longer scheduling and allocation times. Extensive explanations of these conclusions are provided in this work, based on the conducted experiments.

RETRACTED ARTICLE: Design and implementation of fuzzy priority deadline job scheduling algorithm in heterogeneous grid computing

In heterogeneous distributed computing system, huge amount of attention is attracted by task scheduling process in recent days. Scheduling of task is an important problem, where, overall completion of task is completed within a shortest achievable time by scheduling dissimilar task to target processors. This research work concentrates on designing and implementing a fuzzy priority deadline based task scheduling algorithm (FPDSA) having a fuzzy deadline limitation to competent job execution. This proposed FPDSA algorithm is compared with conventional task scheduling techniques like,Improvised Deadline Scheduling Algorithm (IDSA), Earliest Deadline First (EDF) and Prioritized Based Deadline Scheduling Algorithm with respect to Average Actual Execution (AAE) and amount of Non-Delayedand Delayed Jobs. For 4000 tasks, proposed algorithm achieves 7.45%, 27.94% and 30.84% less AAE than IDSA, EDF, and PDSA and for same number of tasks, computational results by proposed FPDSA for non-delayed tasks are 0.32%, 2.17%, and 1.70% higher than IDSA, EDF, and PDSA. This enhances proposed FPDSA’s performance when compared to present scheduling algorithms and illustratesFPDSA is more appropriate scheduling technique forgrid system.

Resource-aware load balancing model for batch of tasks (BoT) with best fit migration policy on heterogeneous distributed computing systems

PurposeLoad balancing is an important issue for a heterogeneous distributed computing system environment that has been proven to be a nondeterministic polynomial time hard problem. This paper aims to propose a resource-aware load balancing (REAL) model for a batch of independent tasks with a centralized load balancer to make the solution appropriate for a practical heterogeneous distributed environment having a migration cost with the objective of maximizing the level of load balancing considering bandwidth requirements for migration of the tasks.Design/methodology/approachTo achieve the effective schedule, load balancing issues should be addressed and tackled through efficient workload distribution. In this approach, the migration has been carried out in two phases, namely, initial migration and best-fit migration. Using the best-fit policy in migrations helps in the possible performance improvement by minimizing the remaining idle slots on underloaded nodes that remain unentertained during the initial migration.FindingsThe experimental results reveal that the proposed model exhibits a superior performance among the other strategies on considered parameters such as makespan, average utilization and level of load balancing under study for a heterogeneous distributed environment.Originality/valueDesign of the REAL model and a comparative performance evaluation with LBSM and ITSLB have been conducted by using MATLAB 8.5.0.

Efficient task pruning mechanism to improve robustness of heterogeneous computing systems

In heterogeneous distributed computing (HC) systems, diversity can exist in both computational resources and arriving tasks. In an inconsistently heterogeneous computing system, task types have different execution times on heterogeneous machines. A method is required to map arriving tasks to machines based on machine availability and performance, maximizing the number of tasks meeting deadlines (defined as robustness). For tasks with hard deadlines (e.g., those in live video streaming), tasks that miss their deadlines are dropped. The problem investigated in this research is maximizing the robustness of an oversubscribed HC system. A way to maximize this robustness is to prune (i.e., defer or drop) tasks with low probability of meeting their deadlines to increase the probability of other tasks meeting their deadlines. In this paper, we first provide a mathematical model to estimate a task’s probability of meeting its deadline in the presence of task dropping. We then investigate methods for engaging probabilistic dropping. We propose methods to dynamically determine task dropping and deferring threshold probabilities. Next, we develop a pruning system and a pruning-aware mapping heuristic, which we extend to engender fairness across various task types. We present the pruning mechanism as an independent component that can be applied to any mapping heuristic to improve the system robustness. To reduce overhead of the pruning mechanism, we propose approximation methods that remarkably reduce the number of mathematical calculations and improve the practicality of deploying the mechanism in heterogeneous or even homogeneous computing systems. We show the cost and energy gains of the pruning mechanism. Simulation results, harnessing a selection of mapping heuristics, show efficacy of the pruning mechanism in improving robustness (on average by ≃22%) and cost in an oversubscribed HC system by up to ≃33%.

Resource-Aware Dynamic Load Balancing System in Heterogeneous Distributed Computing Systems through Swarm Intelligence

Resource-Aware Dynamic Load Balancing System in Heterogeneous Distributed Computing Systems through Swarm Intelligence

A hybrid Method to augment the efficiency of Distributed Computing System by DAG -Using Finest Task Allocation with Dual Mode Processors

Distributed computing system creates or provides a platform having multiple computing nodes linked in a specified manner. On the basis of literature review of last few decades it has been noticed that most of distributed computing researchers have shown their effort to maintain load balancing between processors ,effective task scheduling and optimizing different parameters affecting execution cost and throughput .With these above scenario an additional parameter “Self reconfiguration of CPU” is also a countable parameter to augment the efficiency of distributed computing system .Through this research paper we want to present new approach of adaptive scheduling algorithm which is the mix output of effective task allocation to processor involved in computing and self-reconfiguration of those processors as per need of computing. By this proposed method we will optimize the execution cost, service rate and maximize the throughput as an outcome of organized processors consist in heterogeneous distributed computing system, resulting provide the considerable enhancement in the performance of Distributed computing environment.

Maximizing reliability and performance with reliability-driven task scheduling in heterogeneous distributed computing systems

Computing machines and communication links may fail permanently with nonzero probability in heterogeneous distributed computing systems (HDCSs), and the results of running applications (i.e., large-scale parallel image processing and neuroimaging) on these systems will expect to deteriorate over time. Therefore, the reliability and performance of applications on HDCS remain an imperative and open issue, especially when the parallel applications are scheduled on graphics processing unit architectures. It is urgent to tackle the problem of maximizing performance and reliability considering the impact of communication and machine failures. This work presents a rigorous probabilistic theory to analytically characterize the performance and reliability of an effective task scheduling in the presence of processor and communication failure. An optimal communication path search algorithm considering Reliability overhead and a reliability-driven lookahead scheduling algorithm for precedence constrained tasks are developed. The theoretical model and experimental data, which are based on randomly generated emulation applications represented by directed acyclic graph, reveal that the proposed algorithms significantly outperform previously existing scheduling algorithms in terms of expected makespan, reliability, and schedule length ratio. The weaknesses of the algorithms related to the input parameters are also observed.

On the role of application and resource characterizations in heterogeneous distributed computing systems

Loosely coupled applications composed of a potentially very large number (from tens of thousands to even billions) of tasks are commonly used in high-throughput computing and many-task computing paradigms. To efficiently execute large-scale computations which can exceed the capability in a single type of computing resources within expected time, we should be able to effectively integrate resources from heterogeneous distributed computing (HDC) systems such as clusters, grids, and clouds. In this paper, we quantitatively analyze the performance of three different real scientific applications consisting of many tasks on top of HDC systems based on a partnership of distributed computing clusters, grids, and clouds to understand the application and resource characteristics, and show practical issues that normal scientific users can face during the course of leveraging these systems. Our experimental study shows that the performance of a loosely coupled application can be significantly affected by the characteristics of a HDC system, along with hardware specification of a node, and their impacts on the performance can vary widely depending on the resource usage pattern of each application. We then devise a preference-based scheduling algorithm that can reflect characteristics and resource usage patterns of various loosely coupled applications running on top of HDC systems from our experimental study. Our preference-based scheduling algorithm can allocate the resources from different HDC systems to loosely coupled applications based on the preferences of the applications for the HDC systems. We evaluate the overall system performance over various preference types, using trace-based simulations, which can be determined based on different factors such as CPU specifications and application throughputs. Our simulation results demonstrate the importance of understanding the application and resource characteristics on effective scheduling of loosely coupled applications on the HDC systems.

A New Duplication Task Scheduling Algorithm in Heterogeneous Distributed Computing Systems

A New Duplication Task Scheduling Algorithm in Heterogeneous Distributed Computing Systems

A New Duplication Task Scheduling Algorithm in Heterogeneous Distributed Computing Systems

The efficient scheduling algorithm is critical to achieve high performance in parallel and distributed systems. The main objective of task scheduling is to assign the tasks onto the available processors with the aim of producing minimum schedule length and without violating the precedence constraints. So we developed new algorithm called Mean Communication Node with Duplication MCND algorithm to achieve high performance task scheduling. The MCND algorithm has two phases namely, task priority and processor selection. Our algorithm takes into account the average of parents' communication costs for each task to reduce the overhead communication. The algorithm uses new task duplication algorithm. We build a simulation to compare the MCND algorithm with CPOP with duplication algorithm. The algorithms are applied on real application. From results, the MCND algorithm shows the best results

Priority-Based Task Scheduling in the Cloud Systems Using a Memetic Algorithm

Task scheduling is one of the major issues to achieve high performance in distributed systems such as Grid, Peer-to-Peer and cloud environment. Generally, there are two phases in heuristics-based task scheduling algorithms in heterogeneous distributed computing systems (HeDCSs). These phases are task prioritization and processor assigning respectively. Heuristic-based task scheduling algorithms may use different policies to assign priority to subtasks which produce different makespans in a heterogeneous computing system. Thus, a suitable scheduling algorithm is one that can efficiently assign a priority to tasks in order to minimize makespan. Recently, memetic algorithms (MAs) have been used as evolutionary or population-based global search approaches with local search heuristic to optimize NP-complete problems. Recent studies on MAs have discovered their success on a wide variety of real-world problems. Since the task scheduling problem is an NP-complete, in this paper, a new task scheduling algorithm on cloud environment using multiple priority queues and a memetic algorithm (MPQMA) is proposed. The proposed method uses a genetic algorithm (GA) along with hill climbing to assign a priority to each subtask while using a heuristic-based earliest finish time (EFT) approach to search for a solution for the task-to-processor mapping. The basic idea of our approach is using the advantage of MA to increase the convergence speed of the solutions. We implemented the algorithm on Azure Cloud Service by C# language where the experimental results for the set of randomly generated graphs revealed that the proposed MPQMA algorithm outperformed the existing three task scheduling algorithms in terms of makespan with fast convergence to the optimized solution.

A Multi-Class Task Scheduling Strategy for Heterogeneous Distributed Computing Systems

Performance enhancement is one of the most important issues in high performance distributed computing systems. In such computing systems, online users submit their jobs anytime and anywhere to a set of dynamic resources. Jobs arrival and processes execution times are stochastic. The performance of a distributed computing system can be improved by using an effective load balancing strategy to redistribute the user tasks among computing resources for efficient utilization. This paper presents a multi-class load balancing strategy that balances different classes of user tasks on multiple heterogeneous computing nodes to minimize the per-class mean response time. For a wide range of system parameters, the performance of the proposed multi-class load balancing strategy is compared with that of the random distribution load balancing, and uniform distribution load balancing strategies using simulation. The results show that, the proposed strategy outperforms the other two studied strategies in terms of average task response time, and average computing nodes utilization.

A Survey on ANN Based Task Scheduling Strategies in Heterogeneous Distributed Computing Systems

Heterogeneous Distributed Computing Systems (HeDCS) efficiently utilize the heterogeneity of diverse computational resources which are interlinked through high speed networks for executing a group of computing intensive applications. Directed acyclic graphs (DAGs) are usually used to represent these parallel applications with varied computational requirements and constraints. The optimal scheduling of the given set of precedence constrained tasks to available resources is a core concern in HeDCS and is known to be NP Complete problem. Non deterministic nature of application programs and heterogeneous environment are the main challenges in designing, implementing and analyzing phases of task scheduling techniques. A myriad of heuristic and meta-heuristic approaches have been proposed in the literature to solve this complex problem. The basic purpose of this study is to cover ANN based task scheduling strategies in the distributed computing environment perspective. Further existing scheduling heuristics could be classified in a new state of art classification including the description of frequently used parameters in the mentioned scheduling strategies. The flexible and powerful nature of ANN for identifying the data patterns, underlying time and other constraints and learning capabilities have shown to be a promising candidate among other heuristics.Nepal Journal of Science and Technology Vol. 16, No.1 (2015) pp. 69-78