Resource Usage Profiles Research Articles

An Effective Methodology for Measuring Software Resource Usage

In the last several years, an increasing number of attacks targeting services running on computing systems have been designed and deployed. Such attacks typically tend to overuse some of the system resources, e.g., CPU, memory, or storage. Current solutions are attack specific, i.e., they are designed for a specific malware or a specific set of malware that share the same resource usage profile and are mainly based on the underlying hypothesis that it is possible to measure the amount of resources used by each service under control. Such possibility is clearly available within modern operating systems, but using such measuring tools is nontrivial and limits their applicability. The advent of user-level virtualization systems may change this attitude. In this paper, we propose and validate a methodology for simplifying the measurements of resources used by a software service. Our approach uses the isolation properties of a containerized virtualization system in order to properly measure all the resources used by a specific service, even if it splits itself into different workers. At the same time, using the resource limitation functionalities provided by the virtualization system, we propose a methodology that can be used to limit the effects of malware by limiting the amount of resources that can be accessed on a system. In order to demonstrate the effectiveness of our approach, we consider the specific case of attacks targeting the overuse of power consumption.

REDUCING RENEWABLE RESOURCE DEMAND FLUCTUATION USING SOFT PRECEDENCE RELATIONS IN PROJECT SCHEDULING

Renewable resource levelling is the core of the scheduling process. A perfect schedule ensures that resource supply corresponds to the demand at every unit of project time. A classic approach to resource levelling in schedules with predefined project completion dates consists in manipulating processes start dates. Resource deployment can be also improved by considering alternative construction processes execution modes with various crew formations, and by allowing activities to be split. There are other possibilities: in many practical cases, the activities’ precedence logic predefined in the network model can be changed with no harm to the project outcome. Within the structure of the project network model, some precedence relations between activities would definitely be of fixed (hard) character, whereas some might allow the activities to be executed at the same time or arranged in a variety of logical sequences. The authors use soft precedence relations that let the processes run in reversed order or that can be cancelled, in search for improved resource usage profiles. The benefits of scheduling with soft precedence relations are demonstrated by an example.

Adaptive resource provisioning method using application-aware machine learning based on job history in heterogeneous infrastructures

With the remarkable growth in cloud computing, computing resources can be manipulated on-demand in most scientific fields. This enables scientists to strategically select their experimental environment. Since it is hard to offer cloud resources in accordance with application characteristics, efficient resource provisioning methods are needed. This paper proposes an adaptive resource provisioning method using an application-aware machine learning technique that is based on the job history in heterogeneous infrastructures. The proposed resource provisioning method is built on two main concepts. First, it provides application-aware resource provisioning through the profiling of scientific application in a heterogeneous computing infrastructure. A resource provisioning model uses the resource usage profiles of scientific applications and job history data in heterogeneous computing infrastructures. In addition to the multilayer perceptron machine learning method, an error back-propagation approach is applied to analyze job history to re-learn the error of the output value. Second, it offers an adaptive resource scaling that is invoked by the availability of resource changes. An adaptive resource management method results in an enhancement of the overall balance between the performance and utilization of a system. For the experiments with the two CPU-intensive applications according to the method, a heterogeneous infrastructure comprising clusters and cloud environments is used. Experimental results indicate that the use of the proposed method can gratify user requests (cost and execution time) regarding its application and enhance resource usage effectiveness.

Affinity aware scheduling model of cluster nodes in private clouds

Running applications on a cloud environment without checking and meeting their allocation and performance requirements may lead to unexpected application slowdown and infrastructure under-utilization. In addition, competition for same shared resources may cause performance degradation when applications with similar resource usage profiles are scheduled concurrently. This paper presents an affinity-based model for scheduling virtual machines that host and run scientific applications on a private cloud environment. The main contributions of the proposed model are: i) an approach that exploits the concept of affinity relations among competitive applications; ii) a set of experiments using consolidated HPC benchmarks and their analysis to assess the performance of two concurrent applications; and iii) a novel scheduling algorithm based on an affinity relation among competitive applications.

Advance Resource Reservation based on Context Aware Workload

Cloud computing gives the facility of provisioning resources on rent in pay as-you-go fashion, due to which resource demand changes dynamically over time. Such type of dynamic resource demand leads to mainly two types of problems such as Over-Provisioning and Under-Provisioning. The former leads to violations of Service Level Objectives (SLOs) whereas latter leads to wastage of resources, as the system is not being used to full capacity all the time. Also some cloud having limited number of resources cannot satisfy all the requests at a time. To handle such scenario advance reservation techniques are used, so that the resources available could be used efficiently with minimum possible provisioning cost and at the same time satisfying service level objectives. In the proposed technique, history of resource usage profile of tasks is maintained. For each task submitted to cloud, pattern finding technique is used a task with similar resource usage requirement from history of resource usage profile. After that check-pointing mechanism is used to monitor completion of new task based on resource usage profile of task found in history. Such type of monitoring helps in order to estimate the amount resources that will be released in short duration and based on that resources can be reserved in advance as per user’s request. Hence problem of under as well as over provisioning could be solved up to great extent at the same time meeting the Service Level Objectives. Keywords: Resource Provisioning, Check-pointing, Advance Reservation, Profiling, K-means clustering.

Project resources scheduling and leveling using Multi-Attribute Decision Models: Models implementation and case study

Project scheduling is one of the most vital processes in Project Management. It is a widely discussed topic in academic and practical circles due to its importance and complexity. Manpower, machines, materials and equipment are used for the execution of project activities, but these mostly have limited availability, which can constrain project scheduling procedures. Project resources might exceed or fall short of the resource demand in a project's time horizon. These considerations present issues to project managers who must try to properly allocate among these demands in order to achieve a near optimal utilization during a project's lifetime. Resource leveling is among the greatest challenges faced by project managers as the success of a project largely depends on it. This is because peaks and valleys in the resource usage histogram are responsible for cost overruns due to the necessary recruitment, dismissal and training of the personnel Moreover, issues may arise regarding the efficient management of available resources given that large peaks correspond to fluctuations in resource allocation during a project's life cycle or construction period. To address these issues, resource leveling provides procedures and frameworks that ensure the efficient management of resources to obtain smooth resource usage profiles. These procedures attempt to identify activities that should be delayed to resolve resource over-allocations under time and cost constraints. Given the existence of a variety of available rules that could be followed by project managers to prioritize activities, the paper at hand examines the implementation of five Multi-Attribute Decision Making models and how they perform in the scheduling of a solar park construction project. Namely these models are the Weighted Sum Method, Analytic Hierarchy Process, PROMETHEE, TOPSIS, Ordered Weighted Average (OWA) and Hybrid Weighted Average (HWA). Finally, the derived results are discussed in comparison with those obtained by the standard resource leveling procedures of MS-Project.

A general framework for static profiling of parametric resource usage

AbstractFor some applications, standard resource analyses do not provide the information required. Such analyses estimate the total resource usage of a program (without executing it) as functions on input data sizes. However, some applications require knowing how such total resource usage is distributed over selected parts of a program. We propose a novel, general, and flexible framework for setting up cost equations/relations which can be instantiated for performing a wide range of resource usage analyses, including both static profiling and the inference of the standard notion of cost. We extend and generalize standard resource analysis techniques, so that the relations generated include additional Boolean control variables for switching on or off different terms in the relations, as required by the desired resource usage profile. We also instantiate our framework to perform static profiling of accumulated cost (also parameterized by input data sizes). Such information is much more useful to the software developer than the standard notion of cost: it identifies the parts of the program that have the greatest impact on the total program cost, and which therefore should be optimized first. We also report on an implementation of our framework within the CiaoPP system, and its instantiation for accumulated cost, and provide some experimental results. In addition to generality, our new method brings important advantages over our previous approach based on a program transformation, including support for non-deterministic programs, better and easier integration in the compiler, and higher efficiency.

Collaborative Agents for Distributed Load Management in Cloud Data Centers Using Live Migration of Virtual Machines

Load management in cloud data centers must take into account 1) hardware diversity of hosts, 2) heterogeneous user requirements, 3) volatile resource usage profiles of virtual machines (VMs), 4) fluctuating load patterns, and 5) energy consumption. This work proposes distributed problem solving techniques for load management in data centers supported by VM live migration. Collaborative agents are endowed with a load balancing protocol and an energy-aware consolidation protocol to balance and consolidate heterogeneous loads in a distributed manner while reducing energy consumption costs. Agents are provided with 1) policies for deciding when to migrate VMs, 2) a set of heuristics for selecting the VMs to be migrated, 3) a set of host selection heuristics for determining where to migrate VMs, and 4) policies for determining when to turn off/on hosts. This paper also proposes a novel load balancing heuristic that migrates the VMs causing the largest resource usage imbalance from overloaded hosts to underutilized hosts whose resource usage imbalances are reduced the most by hosting the VMs. Empirical results show that agents adopting the distributed problem solving techniques are efficient and effective in balancing data centers, consolidating heterogeneous loads, and carrying out energy-aware server consolidation.

PRISM: Fine-Grained Resource-Aware Scheduling for MapReduce

MapReduce has become a popular model for data-intensive computation in recent years. By breaking down each job into small map and reduce tasks and executing them in parallel across a large number of machines, MapReduce can significantly reduce the running time of data-intensive jobs. However, despite recent efforts toward designing resource-efficient MapReduce schedulers, existing solutions that focus on scheduling at the task-level still offer sub-optimal job performance. This is because tasks can have highly varying resource requirements during their lifetime, which makes it difficult for task-level schedulers to effectively utilize available resources to reduce job execution time. To address this limitation, we introduce PRISM, a fine-grained resource-aware MapReduce scheduler that divides tasks into phases, where each phase has a constant resource usage profile, and performs scheduling at the phase level. We first demonstrate the importance of phase-level scheduling by showing the resource usage variability within the lifetime of a task using a wide-range of MapReduce jobs. We then present a phase-level scheduling algorithm that improves execution parallelism and resource utilization without introducing stragglers. In a 10-node Hadoop cluster running standard benchmarks, PRISM offers high resource utilization and provides 1.3× improvement in job running time compared to the current Hadoop schedulers.

A Bin Packing Heuristic for On-Line Service Placement and Performance Control

The ever-increasing size and complexity of cloud computing, data centers, virtualization, web services, and other forms of distributed computing make automated and effective service management increasingly important. This article treats the service placement problem as a novel generalization of the on-line vector packing problem. This generalization of the service placement problem does not require a priori knowledge of the service resource profiles, allows for resource profiles to change over time, and allows services to be moved once placed on a server. An on-line self-organizing model profiles resource supplies and demands arranging services in a placement based on their resulting quality rating. A policy-driven asymmetric matrix norm quantifies the quality of the placement allowing for administrative preferences regarding service performance versus service inclusion. Service resource usage profiles' variations cause changes in their assigned placement quality; forcing new, better server placements to be found. Because some placements perform better, a proportional integral derivative controller for performance feedback adjusts the services' actual profile according to service's individual response times. This large scale system autonomically organizes placement of services in response to changes in demand and network disruptions. This article presents theorems which demonstrate the theoretical basis for the model. The article includes empirical results from the implementation of this model in a self-organizing testbed of web servers and services.

Black box scheduling for resource intensive virtual machine workloads with interference models

Modern datacenters consist of increasingly powerful hardware. Achieving high levels of utilization on this hardware often requires the execution of multiple concurrent workloads. Virtualization has emerged as an efficient means to isolate workloads by partitioning large physical resources using self-contained virtual machine images. Despite the many advantages, some challenges regarding performance isolation still need to be addressed. Unmanaged multiplexing of resource intensive workloads has the potential to cause unexpected variances in workload performance.In this paper, we address this issue using performance models based on the runtime characteristics of virtualized workloads. A set of resource intensive workloads is benchmarked with increasing degrees of multiplexing. Resource usage profiles are constructed using the metrics made available by the Xen hypervisor. Based on these profiles, performance degradation is predicted using several existing modeling techniques. In addition, we propose a novel approach using both the classification and regression capabilities of support vector machines. Application clustering is used to identify several application types with distinct performance profiles. Finally, we evaluate the developed performance models by introducing several new scheduling techniques. We demonstrate that the integration of these models in the scheduling logic can significantly improve the overall performance of multiplexed workloads.

PDB23 Cost Effectiveness of Angipars™ in the Treatment of Diabetic Foot Ulcers in Iran

The primary objective of this study was to estimate the cost effectiveness of treating diabetic foot ulcers with ANGIPARS™ plus good wound care (GWC) compared with GWC alone in Iranian health care settings. A secondary objective was to analyse the effect of different treatment practices on the economics of caring for diabetic foot ulcers. A 6-month Markov computer simulation model was used to assess the cost effectiveness in Iran of treating diabetic foot ulcers with ANGIPARS™ plus GWC versus GWC alone. Transition probabilities were taken from a prospective study of 20 patients and ANGIPARS™ efficacy was based on 20-week healing rates in a recent meta-analysis of clinical trials involving 50 patients. Country-specific treatment cost data were collected in collaboration with local economic consultations and combined with the disease model to estimate the incremental cost per ulcer-free month gained. The model was then run using hypothetical low- and high-intensity resource usage profiles to investigate the economics of caring for diabetic foot ulcers. Over the course of 6-month, individuals who received ANGIPARS™ plus GWC were, on average, predicted to spend an additional 0.81 months (24% longer) free of ulcers and to experience a 9% lower risk of undergoing a lower extremity amputation than individuals who received GWC alone. Consequently, ANGIPARS™ plus GWC was estimated to be net cost saving in Iran. ANGIPARS™ may be a cost-effective treatment for neuropathic diabetic foot ulcers in a wide range of Iranian settings.

An approach to enhance the efficiency of opportunistic grids

SUMMARYOpportunistic grid computing middleware has as a main concern the need to preserve the performance of the local applications running on machines that donate resources to the grid. This concern, together with the fact that it happens in an extremely dynamic environment, causes the adoption of a treatment based on thetextitbest‐effort principle for grid applications. This means that efficient application management schemes are usually not employed, which results in less than optimal performance as grid applications often need to be restarted due to (often temporary) resource claims by local user applications. This paper presents a method to improve the performance of grid applications, taking into account resource usage profiles for local applications, trying to identify when such resource claims are temporary and avoiding costly actions such as the migration of grid tasks. The approach is proposed as an extension to the InteGrade middleware and its evaluation shows promising results for the efficient management of grid applications. Copyright © 2011 John Wiley & Sons, Ltd.

Using economic regulation to prevent resource congestion in large-scale shared infrastructures

In this paper we study the problem of large-scale resource congestion from the control and regulation point of view. Applications and services running in large-scale shared infrastructures like Grids or PlanetLab have different resource usage profiles and different resource consumption strategies according to their specific requirements. However, users of these types of infrastructure tend to prefer a subset of available nodes to execute their tasks. As a result, this pattern of user behaviour usually leads to an unfair distribution of work between nodes — i.e. some nodes are highly loaded while the others remain almost idle. We find that most current research focuses on short-term and per-resource scheduling, and the issue of efficient resource allocation in the long term, and system wide, is not yet appropriately studied. Thus, there is a need for controlling, distributing and limiting the capacity of each participant to consume resources considering the state of the system as a whole. Our main contribution is the introduction of a novel macro-scheduling (long-term and system-wide) mechanism for resource capacity self-regulation in which virtual currency or money is used as a tool to govern resource and service usage in massively distributed settings, which are otherwise hard to control. We show by simulation that our approach successfully redistributes the load in a fair and economically efficient manner.

Cost effectiveness of Becaplermin in the treatment of diabetic foot ulcers in four European countries.

The primary objective of this study was to estimate the cost effectiveness of treating diabetic foot ulcers with becaplermin (Regranex) plus good wound care (GWC) compared with GWC alone in a variety of European healthcare settings. A secondary objective was to analyse the effect of different treatment practices on the economics of caring for diabetic foot ulcers. Markov-based simulation study from the perspective of a national health system. A 12-month Markov computer simulation model was used to assess the cost effectiveness in 4 European countries of treating diabetic foot ulcers with becaplermin plus GWC versus GWC alone. Transition probabilities were taken from a prospective study of 183 patients and becaplermin efficacy was based on 20-week healing rates in a recent meta-analysis of clinical trials involving 449 patients. Country-specific treatment cost data were collected in collaboration with local economic consultations and combined with the disease model to estimate the incremental cost per ulcer-free month gained. The model was then run using hypothetical low- and high-intensity resource usage profiles to investigate the economics of caring for diabetic foot ulcers. Over the course of 1 year, individuals who received becaplermin plus GWC were, on average, predicted to spend an additional 0.81 months (24% longer) free of ulcers and to experience a 9% lower risk of undergoing a lower extremity amputation than individuals who received GWC alone. Consequently, becaplermin plus GWC was estimated to be net cost saving in Sweden, Switzerland and the UK. In France, the addition of becaplermin was estimated to add $US19 (1999 values) for each additional ulcer-free month gained. There were substantial intercountry differences in treatment practices and the costs of treating diabetic foot ulcers. Becaplermin may be a cost-effective treatment for neuropathic diabetic foot ulcers in a wide range of European settings. In Sweden, Switzerland and the UK, becaplermin may even be cost saving. Substantial intercountry differences in resource patterns appear, at least partly, to be the logical outcome of differences in unit costs.