Cluster Scheduling Research Articles

Investigation of the influence of the number of modules of a multicluster technological complex and the operations performed by them on the overall performance of the installation using simulation simulations

Variety of compositions of multicluster tools determines the development of methods aimed at analysis and search of feasible compositions, which meet the requirements and optimal performance (throughput) criteria. Very often in the industry, there is a task to find optimum utilization of the cluster tool. To find a solution various approaches are used, such as input in the system several flows of the semifinished products, increasing overall number of the arms of the robot, etc. In this article cluster tool and throughput dependence on quantity of performed operations of the least loaded module are investigated. It is shown, that with the help of this approach one can easily simplify both finding the solution of the task to composite cluster tool with flows of two different types of wafers and determining schedule of the wafers for maximum and minimum throughput. Relevance of the article is stipulated by substantive and complex choices of both finding optimal composition of the cluster tool and scheduling semifinished products flows (in accordance with variations of the operations in modules) at both the design and exploitation stage.

Fine-grained scheduling in multi-resource clusters

In multi-resource clusters, many schedulers allocate resources based on fixed quantities. However, fixed allocations can easily lead to resource fragmentation and over-commitment problems, which may result in lower resource utilization and performance degradation. This paper proposes a fine-grained method (FGM) to improve the allocation granularity of resource allocation. This method divides tasks into execution stages according to the task requirement estimated using similar tasks at the runtime. Then, task resource requirements are matched with the available server resources by stages to refine two aspects of allocation granularity: allocation duration and allocation quantity. In addition, the FGM may over-allocate resources deliberately to further improve resource utilization and performance. The paper tested the FGM in three environments using both online and offline workloads. The test results show that the FGM can resolve resource fragmentation and over-commitment problems by significantly improving resource utilization and performance with acceptable fairness and scheduling response times.

Range smart cluster monitor based guesstimate approach for resource scheduling in small size clusters

Performing scheduling of tasks with low energy consumption with high performance is one of the major concerns in distributed computing. Most of the existing systems have achieved improved energy efficiency but compromised with QoS metrics such as makespan and resource utilization. A resource scheduling strategy for wireless clusters is proposed by making careful considerations on decisions that would im-prove the battery life of nodes. The proposed strategy also incorporates monitoring system with in the clusters for optimizing the system performance as well as energy consumption. The system ensures “Any case zero loss" performance wherein each cluster will be monitored by at least one cluster monitor. This is implemented by using predictive calculation at each cluster monitor to communicate only if absolutely essential, during assigning jobs to resources, selecting optimal resources by assigning the jobs to the most power efficient resource among the available idle resources within the cluster. The experimental result ensures improved system performance with low power consumption in homogeneous computing environment. The resource sharing strategy is experimentally analyzed, considering the important performance metrics such as starvation deadline, turnaround time, miss hit count through simulations. Significant results were observed with improved efficiency.

IQCAR: A Demonstration of an Inter-Query Contention Analyzer for Cluster Computing Frameworks.

Unpredictability in query runtimes can arise in a shared cluster as a result of resource contentions caused by inter-query interactions. iQCAR - inter Query Contention AnalyzeR is a system that formally models these interferences between concurrent queries and provides a framework to attribute blame for contentions. iQCAR leverages a multi-level directed acyclic graph called iQC-Graph to diagnose the aberrations in query schedules that lead to these resource contentions. The demonstration will enable users to perform a step-wise deep exploration of such resource contentions faced by a query at various stages of its execution. The interface will allow users to identify top-k victims and sources of contentions, diagnose high-contention nodes and resources in the cluster, and rank their impacts on the performance of a query. Users will also be able to navigate through a set of rules recommended by iQCAR to compare how application of each rule by the cluster scheduler resolves the contentions in subsequent executions.

Designing cluster schedulers for internet-scale services

Embracing failures for improving availability.

Accelerated subcutaneous immunotherapy in pediatric population – Systematic review

BackgroundAccelerated subcutaneous immunotherapy (SCIT) schedules represent an alternative to conventional SCIT, providing immunotherapy benefits in a shorter period of time. The objectives of this systematic review were to assess clinical and immunological efficacy as well as safety of accelerated SCIT build-up schedules for the treatment of respiratory allergy in pediatric patients. MethodsStudies were located by searching PubMed, using “immunotherapy” and “desensitization” as keywords. The selection of studies, published from January 1st, 2006, to December 31th, 2015, was performed in two stages: screening of titles and abstracts, and assessment of the full papers identified as relevant, considering the inclusion criteria. Data were extracted in a standardized way and synthesized qualitatively to assess efficacy and safety of accelerated schedules in respiratory allergy. ResultsEleven trials were included: two evaluated rush SCIT and nine assessed cluster SCIT. This review demonstrated that rush and cluster schedules are clinically and immunological efficacious, with faster effect than conventional schedules. No relevant difference with respect to clinical outcomes was noticed between subgroups (pediatric, adult and mixed populations). Regarding safety, most local adverse reactions were mild and there were neither life-threatening systemic reactions nor fatal events. No relevant differences in the incidence and severity of either local or systemic reactions between the accelerated schedule group and control group were registered. ConclusionsAccelerated SCIT build-up schedules are effective in the treatment of respiratory allergy in pediatric patients, representing a safe alternative to the conventional schedules with the advantage of achieving clinical effectiveness sooner.

Safety of a Two-visit Cluster Schedule with Venom Immunotherapy

Safety of a Two-visit Cluster Schedule with Venom Immunotherapy

Structures and State-of-Art Research of Cluster Scheduling in Big Data Background

Structures and State-of-Art Research of Cluster Scheduling in Big Data Background

An Energy Efficient Schedule for IEEE 802.15.4/ZigBee Cluster Tree WSN with Multiple Collision Domains and Period Crossing Constraint

Cluster scheduling respecting collision avoidance is a complex problem in cluster-tree wireless sensor networks (WSNs). The difficulty of the problem also increases significantly when the traffic is organized as time-constrained data flows with opposite directions. Thus, in this paper, we seek a collision-free cluster schedule that meets all the data flow deadlines as given in time units. In this context, we have found an elegant approach that expresses the deadline of each flow as an integer number of the length of the schedule period (i.e., period crossing constraints). Consequently, the data flow timeliness requirements become easier to be tackled. Due to the scarce resources of the WSNs, the minimization of the energy consumption of the nodes is a problem of paramount importance. Therefore, the objective is to maximize the lifetime of the network by maximizing the time when the nodes stay in low-power mode. In this paper, we present a novel heuristic scheduling algorithm to obtain the desired schedule. The algorithm is based on very interesting formulations of graph theory problems. Thus, it is efficient in both computational time (instances with thousands of devices are solved in a short time) and solution quality (evaluated over smaller size instances while comparing it with optimal solutions obtained by integer linear programming).

О задаче эффективного управления вычислительной инфраструктурой

At present, big companies such as Amazon, Google, Facebook, Microsoft, Yahoo! own huge datacenters with thousands of nodes. These clusters are used simultaneously by many users. The users submit jobs containing one or more tasks. Task flow is usually a mix of short, long, interactive, batch, and tasks with different priorities. Cluster scheduler decides on which server to run the task, where the task is then run as a process, container or a virtual machine. Scheduler optimizations are important as they provide higher server utilization, lower latency, improved load balancing, and fault tolerance. Achieving good task placement is hard. The problem has multiple dimensions and requires algorithmically complex optimizations. This increases placement latency and limits cluster scalability. In this paper we consider different cluster scheduler architectures and optimization problems.

Cluster Scheduling for Data Centers

This installment of Research for Practice features a curated selection from Malte Schwarzkopf, who takes us on a tour of distributed cluster scheduling, from research to practice, and back again. With the rise of elastic compute resources, cluster management has become an increasingly hot topic in systems R&D, and a number of competing cluster managers including Kubernetes, Mesos, and Docker are currently jockeying for the crown in this space.

Cluster-based multicore real-time mixed-criticality scheduling

Abstract Cluster-based scheduling is recently gaining importance to be applied to mixed-criticality real-time systems on multicore processors platform. In this approach, the cores are grouped into clusters, and tasks that are partitioned among different clusters are scheduled by global scheduler in each cluster. This research work introduces a new cluster-based task allocation scheme for the mixed-criticality real-time task sets on multicore processors. For task allocation, smaller clusters sizes (sub-clusters) are used for mixed-criticality tasks in low criticality mode, while relatively larger cluster sizes are used for high criticality tasks in high criticality mode. In this research paper, the mixed-criticality task set is allocated to clusters using worst-fit heuristic. The tasks from each cluster are also allocated to its sub-clusters, using the same worst-fit heuristic. A fixed-priority response time analysis approach based on Audsley’s approach is used for the schedulability analysis of tasks in each cluster and sub-cluster. If the high criticality job is not completed after its worst case execution time in low mode, then the system is switched to high criticality mode. After mode switch, all the low criticalities tasks are discarded and only high criticality tasks are further executed in high criticality mode. Simulation results indicate that the percentage of schedulable task sets significantly increases under cluster scheduling as compared to partitioned and global mixed-criticality scheduling schemes.

Scheduling Executors with Time-varying Resource Demands on Data-Parallel Computation Frameworks

Efficiently scheduling execution instances of data-parallel computing frameworks, such as Spark and Dryad, on a multi-tenant environment is critical to applications’ performance and systems’ utilization. To this end, one has to avoid resource fragmentation and over-allocation so that both idleness and contention of resources can be minimized. To make effective scheduling decisions, a scheduler has to be informed of and exploit resource demands of individual execution instances, including both short-lived tasks and long-lived executors. The issue becomes particularly challenging when resource demands greatly vary over time within each instance. Prior studies often assume that a scheduling instance is either short lived or of gradually varying resource demands. However, when in-memory computing platforms, such as Spark, become increasingly popular, the assumption no longer holds. The execution instance for scheduling becomes executor, which executes an entire application once it is scheduled. Usually it is not short lived. Its resource demands are significantly time-varying. To address the inefficacy of current cluster schedulers, we propose a scheduling approach, namely Prophet, which takes resource demand variation within each executor into the scheduling decision. It leverages the fact that execution of a data-parallel application is pre-defined by a DAG structure and resource demands at various DAG stages are highly predictable. With this knowledge, Prophet schedules executors to minimize resource fragmentation and over-allocation. To deal with unexpected re-source contention, Prophet adaptively backs off selected task(s) to reduce the contention. We have implemented Prophet in Apache Yarn running Spark. We evaluated it on a 16-server cluster, using 10 categories of a total of 90 application benchmarks. Compared to Yarn’s default capacity and fair schedulers, Prophet reduces application make span by up to 39% and reduces their median completion time by 23%.

Load feedback-based resource scheduling and dynamic migration-based data locality for virtual hadoop clusters in openstack-based clouds

With cloud computing technology becoming more mature, it is essential to combine the big data processing tool Hadoop with the Infrastructure as a Service (IaaS) cloud platform. In this study, we first propose a new Dynamic Hadoop Cluster on IaaS (DHCI) architecture, which includes four key modules: monitoring, scheduling, Virtual Machine (VM) management, and VM migration modules. The load of both physical hosts and VMs is collected by the monitoring module and can be used to design resource scheduling and data locality solutions. Second, we present a simple load feedback-based resource scheduling scheme. The resource allocation can be avoided on overburdened physical hosts or the strong scalability of virtual cluster can be achieved by fluctuating the number of VMs. To improve the flexibility, we adopt the separated deployment of the computation and storage VMs in the DHCI architecture, which negatively impacts the data locality. Third, we reuse the method of VM migration and propose a dynamic migration-based data locality scheme using parallel computing entropy. We migrate the computation nodes to different host(s) or rack(s) where the corresponding storage nodes are deployed to satisfy the requirement of data locality. We evaluate our solutions in a realistic scenario based on OpenStack. Substantial experimental results demonstrate the effectiveness of our solutions that contribute to balance the workload and performance improvement, even under heavy-loaded cloud system conditions.

PeerMsm: a Scalable P2P Model for Streaming Media

Streaming media over Internet is becoming one of the main Internet services, but the limitation of network and server resource makes existing systems the bottleneck of servicing. In this paper, we propose the scalable servicing system based on P2P networks. Through constructing the model of PeerMsm and the FSM of public host, meanwhile, utilizing a new caching and cluster scheduling algorithm, our system makes the media server high QoS and low cost so as to satisfy demands of more users.

An approximate bandwidth allocation algorithm for tradeoff between fairness and throughput in WSN

In this paper, we investigate the problem of bandwidth allocation in wireless sensor network (WSN) under signal to noise plus interference ratio interference model, which aims at finding a tradeoff between fairness and network throughput. Specifically, we propose an approximate algorithm to solve this problem since it has been proved to be NP-hard. Different from similar algorithms proposed in previous issues, we maximize two utility functions, which are the newly defined bandwidth utility function for fairness and network throughput, by jointly considering sensor nodes association and resource allocation in WSN. In addition, we formulate a new utility function with respect to bandwidth allocation, with the method of weighted sum of two objectives as one objective function, which will find a tradeoff between fairness and throughput. Consequently, the problem is decomposed into two sub-problems and solved in two stages, which are cluster formation stage and scheduling stage. In the first stage, we let sensor nodes join to cluster head nodes, which can determine the association of sensor nodes; in the second stage, the total utility function is maximized by allocating time slots for tradeoff between fairness and throughput. Finally, simulation results demonstrate that our algorithm can achieve better performance than compared algorithms.

Back-end scheduling methods of virtual clusters for energy conservation in event-driven WSNs

As sensor nodes of wireless sensor networks (WSNs) are generally battery-powered devices, the critical aspects to face concern how to conserve the energy for nodes, so that the network lifetime can be extended to reasonable times. In this paper, we aim at the energy conservation in event driven WSNs from the perspective of back-end scheduling for virtual clusters in cloud. Taking the Xen Credit scheduler as a baseline, we carry out a comparative study of three representative approaches (i.e., hybrid co-scheduling, balance scheduling, and dynamic switching-frequency scaling (DSS)). Then we discuss challenges in further optimising the energy conservation from the scheduling of virtual clusters. At last, we present and evaluate our proposed approach. Experimental results show that our approach outperforms the existing scheduling approaches of virtual clusters in the terms of reducing the energy consumption of WSNs.

Multi Attribute D-S Evidence Theory Based OCC for Shared-State Scheduling in Large Scale Cluster

With the growth of big data problems, nowadays the size of cloud-scale computing clusters is growing rapidly to run complicated parallel processing jobs. To full utilize cluster resources, the cluster management system is being challenged by the scaling cloud size and the often more complicated application requirements. Omega scheduling software provides a flexible and scalable shared-state scheduling architecture for large scale cluster scheduling. One of its key ideas is using an optimistic concurrency control (OCC) algorithm to let parallel schedulers concurrently make decisions. However, there are few studies exploring to extend OCC for a shared-state scheduling architecture. Furthermore, most of the traditional’ shared-state scheduling architectures also use the same OCCs as Omega does. In this paper, we present a multi attributeDempster–Shafer (D-S) evidence theory based OCC for shared-state scheduling. This OCC adaptsthe multi attribute D-S evidence theory to help making conflict decisions for some scheduling transactions. Experiments’ results show that our method can obtain in some respects more optimized scheduling results compared to coarse-grained conflict detection of Omega.

Using graph isomorphism for mapping of data flow applications on reconfigurable computing systems

The tremendous increase in the computing capacity of the embedded architectures has led to widespread deployment of embedded applications. These applications generally exhibit similar patterns in their specification such as filters in which multiply and accumulate operations are repetitive. If such patterns are identified and used for the system design, trade-off between the area and delay can be achieved. This paper proposes a new methodology which allows to implements a design by retrieving similar patterns known as graph isomorphs and interfaces them as HW accelerators in the system-on-chip design flow. An effective algorithm that converges in polynomial time has been proposed to find such similar subgraphs. In the next phase of the design flow, an algorithm has been proposed which performs the scheduling of clusters and minimizes the time overhead. All algorithms have been written in python for parsing the data flow description and test the correctness of the proposed work. The proposed design flow has been applied to five different programs which are sine, cosine, exponent, matrix multiplication and discrete cosine transform (DCT). These have been described as a data flow graph and have been used for results comparison. An estimation table showing the HW and SW parameter of the data flow operators has been developed for timing and area analysis of the programs. The work is an effort to show the clustering and scheduling of a standalone specification which is mapped on static reconfigurable fabric. Reconfigurable computing systems (RCS) are a popular platform for embedded computing applications as they offer a wide exploration in the design space by allowing HW, SW or HW–SW (hybrid) implementation depending on computational demand and resource requirement. These systems have inspired the designers to find new frameworks for achieving the optimized system characteristics under the given constraints. Any static or dynamic HW hardware optimization in an application can be proposed, implemented and easily verified on the chip. The results presented show the comparison of the proposed approach with SW and HW implementation of DCT design on the Xilinx ML507 board. HW timer has been used to find the execution of each implementation. The experimental verification of the proposed algorithms shows that static IP core design flow gives better results.

Insect venom allergies : Update 2016 for otorhinolaryngologists

Due to the increasing incidence of hymenoptera venom allergies and the potentially life-threatening reactions, it is important for otolaryngologists working in allergology to have an understanding of modern diagnostic and treatment standards for this allergic disease. Molecular diagnosis with recombinant single allergens from bee and wasp venom components improves the diagnostics of insect venom allergies, particularly in patients with double-positive extract-based test results. Detection of specific sensitizations to bee or wasp venom enables double sensitizations to be better distinguished from cross-reactivity. Based on patient history and test results, the patient is initially advised on avoidance strategies and prescribed an emergency medication kit. Then, the indication for allergen-specific immunotherapy (AIT) is evaluated. The dose-increase phase can be performed using conventional, cluster, rush, or ultra-rush schedules, whereby rapid desensitization (rush AIT) performed in the clinic seems to be particularly effective as initial treatment.