Job Response Time Research Articles

Data-Driven Derivation of an Analytic Model for Parallel Servers With Job Replication

The job replication problem has been studied recently as a mechanism to improve performance and availability of systems with $n$ n parallel servers, each with its own queue. A dispatcher using some policy sends $d ~(1 \leq d \leq n)$ d ( 1 ≤ d ≤ n ) copies of a job to $d$ d of the servers. Copies are eliminated from the system as soon as the first copy completes from any of the $d$ d servers. This article introduces a data-driven method to derive closed-form expressions for the average response time and other metrics of jobs as a function of the degree of replication $d$ d . This method consists of developing a simulator for the system in order to generate a very large number of datasets for a wide range of input parameters. A statistical and visualization analysis of the data provides the analytical models. It is important to emphasize the difference between using simulation methods to obtain the value of metrics (e.g., average response time) of a computer system given values of input parameters and using our data-driven method to obtain closed-form expressions that relate output metrics to input parameters. The latter is the focus of our approach. The analysis presented here covers results for homogeneous and heterogeneous servers with exponentially distributed service times and for homogeneous servers with hypo-exponentially and hyper-exponentially distributed service times. This article also presents a closed-form equation for the optimal replication degree for the case of homogeneous servers with hypo-exponentially distributed service times.

Frequency scaling in multilevel queues

In this paper, we study a variant of PS+PS multilevel scheduling, which we call the PS+IS queue. Specifically, we use Processor Sharing (PS) at both queues, but with linear frequency scaling on the second queue, so that the latter behaves like an Infinite Server (IS) queue. The goals of the system are low response times for small jobs in the first queue, and reduced power consumption for large jobs in the second queue. The novelty of our model includes the frequency scaling at the second queue, and the batch arrival process at the second queue induced by the busy period structure of the first queue which has strictly higher priority. We derive a numerical solution for the PS+IS queueing system in steady-state, and then study its properties under workloads obtained from fitting of TCP flow traces. The simulation results confirm the efficacy of the PS+IS model.

VM Consolidation and Migration for IAAS In Cloud Storage

Cloud computing is capable of handling a huge amount of growing work in a predestined manner for the usage of the business customers. The main enabling technology for cloud computing is virtualization which generalize the physical infrastructure and makes it easy to use and manage. In this project virtualization is used to allocate resources based on their needs and also supports green computing concept. “Skewness” concept is introduced here in which the same is minimized to combine various workloads to improve the utilization of the server. Managing the customer demand creates the challenges of on demand resource allocation. So can implement Virtual Machine (VM) technology has been employed for resource provisioning. It is expected that using virtualized environment will reduce the average job response time as well as executes the task according to the availability of resources. Hence VMs are allocated to the user based on characteristics of the job. The VM live migration technology makes the VM and PM (Physical machine) mapping possible when the execution is running.

Limited Busy Periods in Response Time Analysis for Tasks Under Global EDF Scheduling

Response time analysis (RTA) is a kind of effective methods to test the schedulability of real-time tasks under the global earliest deadline first (G-EDF) scheduling. The main idea of the RTA methods is to judge the schedulability of the task set by comparing the worst-case response time (WCRT) and deadline of each task. For obtaining a safe WCRT, existing RTA methods assume that the response times of carry-in jobs are equal to the WCRTs of the tasks generating these jobs. This assumption causes the low accuracy of existing RTA methods, since the busy period lengths of carry-in jobs are usually limited and the response times of these jobs thus cannot always reach the assumed WCRTs. This article proposes a new RTA method that has a higher accuracy than existing RTAs. The main idea of the new method is to limit the busy period lengths of carry-in jobs in the WCRT calculation. We also propose an efficiency improvement scheme for the new method. A set of experiments is conducted to evaluate the performances of the new method and the efficiency improvement scheme. The experimental results show that the new method has a higher acceptance ratio than the state-of-the-art RTA methods and the efficiency improvement scheme can significantly reduce the execution time of the new method.

Low Latency Big Data Processing without Prior Information

Job scheduling plays an important role in improving the overall system performance in big data processing frameworks. Simple job scheduling policies, such as Fair and FIFO scheduling, do not consider job sizes and may degrade the performance when jobs of varying sizes arrive. More elaborate job scheduling policies make the convenient assumption that jobs are recurring, and complete information about their sizes is available from their prior runs. In this paper, we design and implement an efficient and practical job scheduler for big data processing systems to achieve better performance even without prior information about job sizes. The superior performance of our job scheduler originates from the design of multiple level priority queues, where jobs are demoted to lower priority queues if the amount of service consumed so far reaches a certain threshold. In this case, jobs in need of a small amount of service can finish in the topmost several levels of queues, while jobs that need a large amount of service to complete are moved to lower priority queues to avoid head-of-line blocking. Our new job scheduler can effectively mimic the shortest job first scheduling policy without knowing the job sizes in advance. To demonstrate its performance, we have implemented our new job scheduler in YARN, a popular resource manager used by Hadoop/Spark, and validated its performance with experiments in both real testbeds including Amazon EC2 and large-scale trace-driven simulations. Our experimental and simulation results have strongly confirmed the effectiveness of our design: our new job scheduler can reduce the average job response time of the Fair scheduler by up to 45 percent and achieve better fairness at the same time.

LSTPD: Least Slack Time-Based Preemptive Deadline Constraint Scheduler for Hadoop Clusters

Big data refers to numerous forms of complex and large datasets which need distinctive computational platforms in order to be analyzed. Hadoop is one of the popular frameworks for analytics of big data. In Hadoop, a big job is split into multiple small tasks and then they are distributed to worker nodes in a parallel way using MapReduce to speed up computational processes. In this aspect, it is important how to improve throughput performance. MapReduce jobs require quick responses from the worker nodes to complete them under their deadlines. The existing scheduling schemes for Hadoop such as FIFO, fair, and capacity schedulers cannot guarantee the quick response requirement satisfying a prior deadline. Thus, Hadoop system needs to improve response time and completion time for the heterogeneous MapReduce jobs. In this paper, we propose an efficient preemptive deadline constraint scheduler based on least slack time and data locality. In order for better allocation of tasks and load balancing, we first analyze the task scheduling behaviors of the Hadoop platform. Based on that, we propose a novel preemptive approach which considers the remaining execution time of the job being executed in deciding preemption. The experimental results show that the proposed scheme significantly reduces the job execution time and queue waiting time, compared to existing schemes.

OnDisc: Online Latency-Sensitive Job Dispatching and Scheduling in Heterogeneous Edge-Clouds

In edge-cloud computing, a set of servers (called edge servers) are deployed near the mobile devices to allow these devices to offload their jobs to and subsequently obtain their results from the edge servers with low latency. One fundamental problem in edge-cloud systems is how to dispatch and schedule the jobs so that the job response time (defined as the interval between the release of the job and the arrival of the computation result at the device) is minimized. In this paper, we propose a general model for this problem, where the jobs are generated in arbitrary order and at arbitrary times at the mobile devices and then offloaded to servers with both upload and download delays. Our goal is to minimize the total weighted response time of all the jobs. The weight is set based on how latency-sensitive the job is. We derive the first online job dispatching and scheduling algorithm in edge-clouds, called OnDisc , which is scalable in the speed augmentation model; that is, OnDisc is $(1 + \varepsilon )$ -speed $O(1/\varepsilon )$ -competitive for any small constant $\varepsilon >0$ . Moreover, OnDisc can be easily implemented in distributed systems. We also extend OnDisc with a fairness knob to incorporate the trade-off between the average job response time and the degree of fairness among jobs. Extensive simulations based on a real-world data-trace from Google show that OnDisc can reduce the total weighted response time dramatically compared with heuristic algorithms.

Semi-Clairvoyant Scheduling in Data Analytics Systems

Popular data analytics systems including Apache Hadoop, Dryad, and Apache Spark abstract jobs as directed acyclic graphs (DAGs). Speeding up completions for DAG jobs matter in practice in order to support real-time decisions. State-of-the-art works propose clairvoyant schedulers to optimize these goals, however, they assume complete job information as a prior knowledge which includes the precise DAG structure, and fine-grained resource requirement and duration time of each task. This assumption limits their applicability. In this paper, to be more practical, we relax the complete prior knowledge assumption and rely solely on partial prior information, based on which, we design a semi-clairvoyant task scheduler Cobra operating within each job. When managing resources for a job, Cobra adaptively adjusts its resource desires in a multiplicative-increase multiplicative-decrease manner on the basis of the its current resource utilization and the presence of current waiting tasks. When assigning tasks to run on the allocated resources, Cobra strives to satisfy task locality preference by tolerating each task waiting for some time that is bounded by a parameterized threshold. Even with the partial prior job information, when a set of jobs in which each employing Cobra as its task scheduler, run on a cluster that employs the fair job scheduler, we theoretically prove the produced makespan and average job response time are O(1)-competitive in different settings. We implement our design in Spark on YARN system, and use experiments from both real deployments and simulations on Google's trace to verify the performance promotion and sensitivity of Cobra.

Optimized load balancing technique in cloud computing systems

Cloud computing” is a term, which involves virtualization, distributed computing, Networking, software and web services. A cloud consists of several elements such As clients, datacenter and distributed servers. It includes fault tolerance, high availability, Scalability, flexibility, reduced overhead for users, reduced cost of ownership, on Demand services etc. Central to these issues lies the establishment of an effective load balancing algorithm. The load can be CPU load, memory capacity, delay or network load. Load Balancing is the process of distributing the load among various nodes of a distributed System to improve both resource utilization and job response time while also avoiding A situation where some of the nodes are heavily loaded while other nodes are idle or Doing very little work. Load balancing ensures that all the processor in the system or Every node in the network does approximately the equal amount of work at any instant of time. This technique can be sender initiated, receiver initiated or symmetric type (Combination of sender initiated and receiver initiated types).

Fairness in responsive parallelism

Research on parallel computing has historically revolved around compute-intensive applications drawn from traditional areas such as high-performance computing. With the growing availability and usage of multicore chips, applications of parallel computing now include interactive parallel applications that mix compute-intensive tasks with interaction, e.g., with the user or more generally with the external world. Recent theoretical work on responsive parallelism presents abstract cost models and type systems for ensuring and reasoning about responsiveness and throughput of such interactive parallel programs. In this paper, we extend prior work by considering a crucial metric: fairness. To express rich interactive parallel programs, we allow programmers to assign priorities to threads and instruct the scheduler to obey a notion of fairness. We then propose the fairly prompt scheduling principle for executing such programs; the principle specifies the schedule for multithreaded programs on multiple processors. For such schedules, we prove theoretical bounds on the execution and response times of jobs of various priorities. In particular, we bound the amount, i.e., stretch, by which a low-priority job can be delayed by higher-priority work. We also present an algorithm designed to approximate the fairly prompt scheduling principle on multicore computers, implement the algorithm by extending the Standard ML language, and present an empirical evaluation.

Elastic neural network method for load prediction in cloud computing grid

<span>Cloud computing still has no standard definition, yet it is concerned with Internet or network on-demand delivery of resources and services. It has gained much popularity in last few years due to rapid growth in technology and the Internet. Many issues yet to be tackled within cloud computing technical challenges, such as Virtual Machine migration, server association, fault tolerance, scalability, and availability. The most we are concerned with in this research is balancing servers load; the way of spreading the load between various nodes exists in any distributed systems that help to utilize resource and job response time, enhance scalability, and user satisfaction. Load rebalancing algorithm with dynamic resource allocation is presented to adapt with changing needs of a cloud environment. This research presents a modified elastic adaptive neural network (EANN) with modified adaptive smoothing errors, to build an evolving system to predict Virtual Machine load. To evaluate the proposed balancing method, we conducted a series of simulation studies using cloud simulator and made comparisons with previously suggested approaches in the previous work. The experimental results show that suggested method betters present approaches significantly and all these approaches.</span>

Hybrid Cloud Adaptive Scheduling Strategy for Heterogeneous Workloads

With the advent of the era of big data, many companies have taken the most important steps in the hybrid cloud to handle large amounts of data. In a hybrid cloud environment, cloud burst technology enables applications to be processed at a lower cost in a private cloud and burst into the public cloud when the resources of the private cloud are exhausted. However, there are many challenges in hybrid cloud environment, such as the heterogeneous jobs, different cloud providers and how to deploy a new application with minimum monetary cost. In this paper, the efficient job scheduling approach for heterogeneous workloads in private cloud is proposed to ensure high resource utilization. Moreover, the task scheduling method based on BP neural network in hybrid cloud is proposed to ensure that the tasks can be completed within the specified deadline of the user. The experimental results show that the efficient job scheduling approach can veffectively reduce the job response time and improve the throughput of cluster. The task scheduling method can reduce the response time of tasks, improve QoS satisfaction rate and minimize the cost of public cloud.

Load Balance Based Job Scheduling in Geo-Distributed Clouds

In the wake of rapid development of Internet, more and more people could access it from any places in the world, which leads to the characteristic of geographical distribution of the data. Only one single cloud cannot deal with such data efficiently due to high delay and transmission cost. The geo-distributed clouds can alleviate it. However, because of the varied locations of geo-distributed clouds, how to balance the workloads of geo-distributed clouds is a crucial problem. In this paper, an efficient load balance based job scheduling in geo-distributed clouds is proposed in order to minimize the average waiting time, average response time of jobs and improve the system throughput. First, the clouds are divided into idle or busy state to get the job execution time in each cloud by Logistic regression. Then, the job scheduling problem is modeled as a $$ M/M/C $$ queue in each cloud. In addition, Lagrange Multiplier is given to derive the optimal job arrival rate of each cloud. Finally, the experimental results show that our proposed algorithm in this paper can decrease the average waiting time, average execution time and average response time of jobs, and improve system throughput.

TS-BatPro: Improving Energy Efficiency in Data Centers by Leveraging Temporal–Spatial Batching

With the rapid scaling of data centers, understanding their power characteristics and optimizing data center energy consumption is a critical task. Typically, data centers are provisioned for peak load even when they are mostly operating at low utilization levels. This results in wasteful energy consumption requiring smart energy saving strategies. At the same time, latency critical application workloads have stringent quality of service (QoS) constraints that need to be satisfied. Optimizing data center energy with QoS constraints is challenging since different workloads can have variabilities in job sizes and distinct system utilization levels. Also, server configuration (e.g., the number of cores per server) can be different across data centers. Therefore, a single configuration for energy management that works well across these various scenarios, is not practical. In this paper, we propose TS-BatPro, a novel framework that judiciously integrates spatial and temporal job batching to save energy for multicore data center servers while meeting the QoS constraints for application workloads. TS-BatPro performs effective global job batching and scheduling by modeling job performance and power characteristics of multicore servers without any hardware modifications. TS-BatPro works on commodity server platforms and comprises two components: 1) a temporal batching engine that batches incoming jobs such that the servers can continue to reside in low-power states and 2) a spatial batching engine that prioritizes scheduling job batches to a small subset of servers. We develop a prototype of TS-BatPro on physical testbed with a cluster of servers and evaluate TS-BatPro on a variety of workloads. Our results show that TS-BatPro is able to achieve significant amount of energy savings under various job response time constraints and traffic patterns.

OPTIMAL ENERGY DISTRIBUTION WITH ENERGY PACKET NETWORKS

We use the Energy Packet Network (EPN) to investigate an optimal energy distribution problem for the computer-communication system which is powered by intermittent renewable energy sources. The objective is to find an optimal energy distribution to minimize the proposed cost function which computes penalty costs caused by the overall average response time of jobs and the energy loss. In this EPN system, we consider the energy can be lost through storage leakages, or due to empty workstations which will consume energy even no job needs to be processed. Related numerical examples with different sets of parameter values are presented in the paper to evaluate the system performance and to examine the obtained analytical solution. Then a special case is considered to study the optimal system performance when the total energy harvesting rate is sufficiently large.

Cost-aware scheduling for ensuring software performance and reliability under heterogeneous workloads of hybrid cloud

Cloud computing is a rapidly growing paradigm in software engineering that offers different services. The hybrid cloud is the best choice for the enterprise to benefit by taking resources on lease from the public cloud only if private cloud resources are not sufficient. However, the key is how to provide better cloud services and improve software performance in the hybrid cloud for software engineers. In this paper, the efficient job scheduling method in the private cloud is proposed by considering the heterogeneity of hybrid cloud resources to guarantee the software performance and reliability. The experimental results show that the efficient job scheduling method can effectively reduce the average job response time and improve the system throughput. Moreover, the task scheduling method based on BP neural network in the hybrid cloud is proposed by considering both the cost and deadline constraints to ensure the quality of service (QoS) for software. The experimental results show that the task scheduling method can improve the QoS, maximize the resources utilization of private cloud and minimize the cost of hybrid cloud resources.

BoPF

Even though batch, interactive, and streaming applications all care about performance, their notions of performance are different. For instance, while the average completion time can suffciently capture the performance of a throughout-sensitive batch-job queue (TQ) [5], interactive sessions and streaming applications form latencysensitive queues (LQ): each LQ is a sequence of small jobs following an ON-OFF pattern. For these jobs [7], individual completion times or latencies are far more important than the average completion time or the throughput of the LQ. Indeed, existing "fair" schedulers are inherently unfair to LQ jobs: when LQ jobs are present (ON state), they must share the resources equally with TQ jobs, but when they are absent (OFF state), batch jobs get all the resources. In the long run, TQs receive more resources than their fair shares because today's schedulers such as Dominant Resource Fairness [4] make instantaneous decisions Clearly, it is impossible to achieve the best response time for LQ jobs under instantaneous fairness. In other words, there is a hard tradeoff between providing instantaneous fairness for TQs and minimizing the response time of LQs. However, instantaneous fairness is not necessary for TQs because average-completion time over a relatively long time horizon is their most important metric. This sheds light on the following question: how well can we simultaneously accommodate multiple classes of workloads with performance guarantees, in particular, isolation protection for TQs in terms of long-term fairness and low response times for LQs? This work serves as our first step in answering the question by designing BoPF: the first multi-resource scheduler that achieves both isolation protection for TQs and response time guarantees for LQs in a strategyproof way. The key idea is "bounded" priority for LQs: as long as the burst is not too large to hurt the long-term fair share of TQs and other LQs, they are given higher priority so jobs can be completed as quickly as possible.

Reliable Access Control for Mobile Cloud Computing (MCC) With Cache-Aware Scheduling

In mobile cloud computing (MCC), data are stored and processed by more powerful computing platforms located in clouds to enhance the capability of mobile devices. Although MCC brings many benefits to users and organizations, security has become the main concern for MCC because it involves third party environment where data confidentiality is not a guarantee. To overcome this challenge, attribute-based encryption (ABE) technique, a promising cryptography method was introduced to support both data confidentiality and access control simultaneously. However, most of the ABE used single certification authority (CA) and disjoint attribute authority (AA) to verify user identity which can be a single point of failure. Therefore, a new approach was proposed in this research to deal with the single point of failure in the authority. Furthermore, the cache-based scheduling algorithm was upgraded to improve the response time of user jobs. The experimental results show that the approach performed better compared to its competitor in terms of reliability. Additionally, it significantly reduced the number of read operations compared to its counterpart.

Replica-aware task scheduling and load balanced cache placement for delay reduction in multi-cloud environment

With the development of content-sharing and collaborative computing services such as online social networks, scientific workflow, there are huge amounts of data generated. To process this tremendous amount of data, multi-cloud system that integrates multiple clouds together to provide a unified service in a collaborative manner has been introduced. However, task scheduling in such heterogeneous multi-cloud environment is very challenging. To reduce response delay caused by cross-data centers file access, we proposed a replica-aware task scheduling algorithm based on data replication. For speeding up data access in multi-cloud cooperative caches, we presented a load balanced cache placement algorithm based on Bayesian networks. In our scheduling algorithm, combined transferring computation with transferring data, resource matching is accomplished according to node locality. Only non-local unassigned and failed map tasks’ input data are replicated and transferred in advance to target nodes to expedite task execution. In our cache placement method, based on Bayesian networks the next execute task is predicted. In accordance with caching profit and recycling cost, cache prefetching files are selected. For each prefetching file, according to load balancing, target placement node is determined. Extensive experimental results show that the performance of our proposed replica-aware task scheduling algorithm is better than benchmark scheduling algorithms in terms of node locality ratio and job response time, and our load balanced cache placement algorithm outperforms the baseline caching algorithms in performance of prefetching hit ratio and execution time saving ratio.

Towards operational cost minimization for cloud bursting with deadline constraints in hybrid clouds

In hybrid clouds, there is a technique named cloud bursting which can allow companies to expand their capacity to meet the demands of peak workloads in a low-priced manner. In this work, a cost-aware job scheduling approach based on queueing theory in hybrid clouds is proposed. The job scheduling problem in the private cloud is modeled as a queueing model. A genetic algorithm is applied to achieve optimal queues for jobs to improve the utilization rate of the private cloud. Then, the task execution time is predicted by back propagation neural network. The max–min strategy is applied to schedule tasks according to the prediction results in hybrid clouds. Experiments show that our cost-aware job scheduling algorithm can reduce the average job waiting time and average job response time in the private cloud. In additional, our proposed job scheduling algorithm can improve the system throughput of the private cloud. It also can reduce the average task waiting time, average task response time and total costs in hybrid clouds.