Makespan Of Applications Research Articles

Dynamic Multiworkflow Offloading and Scheduling Under Soft Deadlines in the Cloud-Edge Environment

Computing offloading and scheduling are emerging, accompanied by smart city services, Internet of Things, and other rich cloud services. Cloud-assisted mobile edge computing provides malleable resources but introduces considerable overhead. Due to the geographical distributions and distances of the edge, cloud, and mobile terminal, promoting cost efficiency and reducing the makespan of applications are urgent targets. For multiple mobile applications, the soft deadline constraint for each application is common in the edge-cloud environment. When multiple users initiate application requests, scheduling on the target offloaded platform commences. Some workflows, especially newly arrived workflows, might suffer from “starving to death,” which has the risk of being missed. This article describes a dynamic, multiworkflow offloading, and scheduling, adaptive heterogeneous earliest-finish-time-based algorithm, which can lightly run on a distributed network. We explore an innovative model composed of a multiobjective makespan and cost for a soft deadline constraint optimization problem. Abundant data are adopted to test the offloading and scheduling effect on a variety of platform parameters, and comparative experiments demonstrate the validity and efficiency of the adaptive system.

Microservices architectural based secure and failure aware task assignment schemes in fog‐cloud assisted Internet of things

The Internet of Things (IoT) paradigm has applications in many domains and is growing these days progressively. The applications are e-business, e-healthcare, and e-transportation. Recently, the container microservices-based Mobile Cloud Computing (MCC) has gained popularity, a lightweight framework compared to the monolithic virtual machine-based system. MCC combines fog nodes or cloud nodes with a base station to run the applications. However, storing the sensitive data of IoT applications on the untrusted nodes and failure of services are critical challenges in the existing architecture. This study proposes a novel microservices-based by combining fog and cloud services with efficient schemes. The first scheme is the Latency Aware Task Assignment Algorithm, which determines the optimal assignment of tasks to minimize the makespan of all applications. The second scheme is Fully Homomorphism Encryption, which ensures data security before an offload to any external assistance for execution. The final one is the Failure Aware, which handles any transient failure during application execution in the architecture. The experimental results show that the recommended architecture improved resource utilization, and the proposed schemes satisfied the security demand while reducing the makespan of applications.

Network-aware task selection to reduce multi-application makespan in cloud

One new metric that plays a vital role in evaluating the cloud service is the multi-application makespan. There are usually multiple applications without a deadline in the cloud, while the makespan of each application should be minimized, such as parallel applications for training neural networks. However, the previous scheduling rule, called moving the computation tasks of applications closer to data, fails to reduce multi-application makespan. Because it is unknown which application the task belongs to, the number of optimized tasks (close-to-data tasks) is unbalanced among multiple applications, which causes a big gap between the makespan of multiple applications. To address this issue, we propose a cooperative scheduler that considers the application-task relationship, whose goal is reducing the multi-application makespan. Firstly, we propose a cost sharing game model to guide the balance of optimized task selection between multiple applications, in which the cost is referred to as the makespan. Specifically, we develop a network-aware method that can accurately estimate the makespan with a given task selection strategy. In the end, we evaluate the cooperative scheduler in the actual Hadoop cluster with diverse network environments. Experimental results demonstrate that the gap between multi-application makespan decreases. Moreover, compared to the baseline schedulers, the longest makespan decreases by 61.5%, and the network traffic is saved by more than 50%.

Decentralized executions of privacy awareness data analytics workflows in the cloud

SummaryNowadays, with the development of cloud computing technology, an increasing number of enterprises or organizations have migrated applications to the cloud environment. Because of privacy concerns, a company may store sensitive data on a local server or a private cloud. As a result, the data analytics tasks have to be performed only in the local environment even when the analysis results will be shared to the outside. In this paper, we design a decentralized workflow system to speed up the execution of a Privacy‐Awareness Data Analytics application (PADA). Specifically, a workflow system with actor layer and engine layer for distributed execution of PADA proposed. The actors are responsible for performing actual data analysis tasks on the data side while the engines are deployed into different regions of the public cloud and communicate with actors to execute the whole workflow. Since engines in different regions have different communication latencies when invoking an actor, the optimization engine deployment scheme should be found to minimize the total execution time of a PADA. A path search‐based heuristic algorithm is designed to select suitable cloud regions to place the engines in the public cloud. The experimental results indicate that the proposed algorithm is effective in reducing application makespan.

Combining analytical modeling, realistic simulation and real experimentation for the optimization of Monte-Carlo applications on the European Grid Infrastructure

Evaluating the performance of distributed systems through real experimentation is resource-consuming and by essence very difficult to reproduce. Conversely, analytical modeling and simulation facilitate investigation, but their level of realism needs to be evaluated to avoid misinterpretation. In this paper, we combine production experiments and realistic simulation for performance modeling and optimization of application workflows deployed on the European Grid Infrastructure (EGI), one of the largest distributed systems in the world. We use a validated simulator to (i) exhaustively evaluate an analytical model of the application makespan and (ii) study the influence and calibrate the parameters of the application workflow, in particular the checkpointing period. Experimental results show that the model fits the simulated makespan with a relative error of at most 15%, and that simulation allows us to validate analytical models in a more exhaustive manner than what is possible with production experiments. Results also show that, provided that the simulator is correctly validated and instantiated, simulation can be safely used for exhaustive parameter studies, allowing for a quick and fine tuning of sensitive application parameters.

On the impact of process replication on executions of large-scale parallel applications with coordinated checkpointing

Processor failures in post-petascale parallel computing platforms are common occurrences. The traditional fault-tolerance solution, checkpoint–rollback–recovery, severely limits parallel efficiency. One solution is to replicate application processes so that a processor failure does not necessarily imply an application failure. Process replication, combined with checkpoint–rollback–recovery, has been recently advocated. We first derive novel theoretical results for Exponential failure distributions, namely exact values for the Mean Number of Failures To Interruption and the Mean Time To Interruption. We then extend these results to arbitrary failure distributions, obtaining closed-form solutions for Weibull distributions. Finally, we evaluate process replication in simulation using both synthetic and real-world failure traces so as to quantify average application makespan. One interesting result from these experiments is that, when process replication is used, application performance is not sensitive to the checkpointing period, provided that period is within a large neighborhood of the optimal period. More generally, our empirical results make it possible to identify regimes in which process replication is beneficial.

Fault-tolerant scheduling on parallel systems with non-memoryless failure distributions

As large parallel systems increase in size and complexity, failures are inevitable and exhibit complex space and time dynamics. Most often, in real systems, failure rates are increasing or decreasing over time. Considering non-memoryless failure distributions, we study a bi-objective scheduling problem of optimizing application makespan and reliability. In particular, we determine whether one can optimize both makespan and reliability simultaneously, or whether one metric must be degraded in order to improve the other. We also devise scheduling algorithms for achieving (approximately) optimal makespan or reliability. When failure rates decrease, we prove that makespan and reliability are opposing metrics. In contrast, when failure rates increase, we prove that one can optimize both makespan and reliability simultaneously. Moreover, we show that the largest processing time (LPT) list scheduling algorithm achieves good performance when processors are of uniform speed. The implications of our findings are the accelerated completion and improved reliability of parallel jobs executed across large distributed systems. Finally, we conduct simulations to investigate the impact of failures on the performance, which is done using an actual application of biological sequence comparison.

Energy-aware task scheduling in heterogeneous computing environments

Efficient application scheduling is critical for achieving high performance in heterogeneous computing (HC) environments. Because of such importance, there are many researches on this problem and various algorithms have been proposed. Duplication-based algorithms are one kind of well known algorithms to solve scheduling problems, which achieve high performance on minimizing the overall completion time (makespan) of applications. However, they pursuit of the shortest makespan overly by duplicating some tasks redundantly, which leads to a large amount of energy consumption and resource waste. With the growing advocacy for green computing systems, energy conservation has been an important issue and gained a particular interest. An existing technique to reduce energy consumption of an application is dynamic voltage/frequency scaling (DVFS), whose efficiency is affected by the overhead of time and energy caused by voltage scaling. In this paper, we propose a new energy-aware scheduling algorithm with reduced task duplication called Energy-Aware Scheduling by Minimizing Duplication (EAMD), which takes the energy consumption as well as the makespan of an application into consideration. It adopts a subtle energy-aware method to search and delete redundant task copies in the schedules generated by duplication-based algorithms, and it is easier to operate than DVFS, and produces no extra time and energy consumption. This algorithm not only consumes less energy but also maintains good performance in terms of makespan compared with duplication-based algorithms. Two kinds of DAGs, i.e., randomly generated graphs and two real-world application graphs, are tested in our experiments. Experimental results show that EAMD can save up to 15.59 % energy consumption for HLD and HCPFD, two classic duplication-based algorithms. Several factors affecting the performance are also analyzed in the paper.

Self-adjusting resource sharing policies in Federated Grids

The majority of non-coordinated decentralized meta-schedulers in a Federated Grid perform scheduling strategies without taking into account resources’ current load or specific resource owners’ internal demands, leading to suboptimal schedules. Clearly, these policies increase the number of job migrations, the number of messages generated per re-scheduled job, and also the application makespan. The main purpose of the present study is to analyze the effect of applying self-adjusting resource sharing policies to previously proposed performance based scheduling strategies. For example, when a resource is near saturation or has an internal peak demand, it can decide not to accept new external jobs. On the other hand, when a job owner receives the previous action, it can decide not to submit temporally more jobs to that resource. In this way, the proposed self-adjusting resource sharing policies save time and communication bandwidth by reducing the number of jobs migrations, and thus, avoiding the generation of the corresponding messages per re-scheduled job. At the same time, the new resource sharing strategies improve application makespan and resource performance objective functions while maintaining infrastructure owners complete autonomy.

Cost and Makespan Trade-off Management for Scheduling Workflow on Utility Grids

Reaching a high performance in Utility Grids, among other things, requires an efficient workflow application scheduling on shared resources. The shared resources among users are autonomously managed in these environments. Application tasks are submitted to a scheduler; next the scheduler selects a resource for executing the task.Finally, the resource schedules the assigned tasks according to its own allocation policy. Therefore, users have no explicit control on allocating resources to application task. As a result Quality of Service cannot be guaranteed. In this paper, a heuristic algorithm known as Minimum First-fit Cost-Makespan Trade-off (MinFCMT) is developed in order to effectively schedule an application in Utility Grids so that the application makespan and allocation-cost can be minimized. To evaluate the MinFCMT heuristic algorithm a widespread simulation of a real parallel workload models plus the synthetic workflow is used according to the existing application scheduling algorithm. The results demonstrated that the MinFCMT algorithm is far more effective than the existing algorithms in terms of time-complexity of algorithm.

Scheduling Grid Applications with Software Requirements

The aim of task scheduling is to minimize the makespan of applications, exploiting the best possible way to use shared resources. Applications have requirements which call for customized environments for their execution. One way to provide such environments is to use virtualization on demand. This paper presents two schedulers based on integer linear programming which schedule virtual machines (VMs) in grid resources and tasks on these VMs. The schedulers differ from previous work by the joint scheduling of tasks and VMs and by considering the impact of the available bandwidth on the quality of the schedule. Experiments show the efficacy of the schedulers in scenarios with different network configurations.

A workflow model for heterogeneous computing environments

This work presents the details of a model for fully decentralised scientific workflow management designed to support the efficient execution of applications in heterogeneous and unpredictable computing environments. The proposed model involves a novel application of the overlay metacomputer concept, built on a graph-oriented peer to peer framework. The use of peer to peer overlay on both batch and cycle scavenging systems simultaneously enables the aggregation and virtual homogenisation of heterogeneous resource collectives, and makes possible the execution of complex scientific workflow applications on otherwise batch oriented processors. Overlaying a peer to peer workflow management system on multiple independent sites decouples task expression from resource allocation, i.e., the host resource managers allocate processing elements to host the peers, allowing the peers execute multiple workflow tasks. This facilitates the efficient execution of very fine grained workflow expressions, which is shown to reduce both application makespan and wasted cycles when using processing elements allocated by a pre-emptive resource manager. Quantitative results are presented based on performance observed in a large scale heterogeneous multi-user environment.