Virtual Machine Consolidation Research Articles

A Study on Energy Consumption of DVFS and Simple VM Consolidation Policies in Cloud Computing Data Centers Using CloudSim Toolkit

The exponential growth in cloud services has led to an all-time high use and expansion of cloud computing frameworks. The frameworks operating in data centers, reportedly account for about two-hundredths of total energy consumed around the globe. Numerous methods have been proposed to curb or reduce the consumption. One such method is Virtual Machine consolidation. In this paper, we simulate the operation of a data center with varying number of hosts across different operating hours with support of CloudSim and estimate the energy consumption of non-power aware hosts, dynamic voltage and frequency scaling—enabled hosts, and two popular VM consolidation policies, namely—local regression minimum utilization and static threshold random selection. We then compare the above techniques to show the various levels of energy consumption. We also have a brief look at the SLA violation rates of the two consolidation policies.

Nash equilibrium based replacement of virtual machines for efficient utilization of cloud data centers

Workload uncertainty has been increased with the integration of the Internet of Things to the computing grid i.e. edge computing and cloud data centers. Therefore, efficient resource utilization in cloud data centers become more challenging. Dynamic consolidation of virtual machines on optimal number of processing machines can increase the efficiency of resource utilization in cloud data centers. This process requires the migration of virtual machines from the under-utilized and over-utilized processing machines to other suitable machines. In this work, the problem of efficient replacement of virtual machines is solved using a game theory based well known technique, Nash Equilibrium (NE). We designed a nash equilibrium based dual on two players, over-load manager and under-load manager, to deduce the dominant strategy profiles for various scenarios during consolidation cycles. Dominant strategy profile is the set of strategies where every player has no incentive in deviation, thus leading to equilibrium position. A virtual machines redeployment algorithm, Nash Equilibrium based Virtual Machines Replacement (NE-VMR), has been proposed on the basis of the dominant strategy profiles for efficient consolidation. Experiment results show that NE-VMR is a more efficient server consolidation technique, saved 30% energy and improved 35% quality of service as compared to baselines.

Embedding individualized machine learning prediction models for energy efficient VM consolidation within Cloud data centers

The fast growth in demand for utility-based IT services has lead to the formation of large scale Cloud data centers. The electrical energy consumption of these data centers results in high operational costs and carbon dioxide emissions. Cloud data centers benefit from the use of virtualization technology to reduce their energy consumption. This technology enables a Cloud data center to allocate its physical resources (CPU, memory, hard disk, network bandwidth) on demand and balance loads between their physical hosts by live migration of Virtual Machines (VMs). However, the migration of VMs can result in Service Level Agreement Violations (SLAVs) and consequently low Quality of Service (QoS). Hence, in this paper, we propose an energy aware VM consolidation algorithm that minimizes SLAVs.Dynamic VM consolidation has three stages: a) Detecting over- and under-utilized hosts; b) Selecting one or more VMs for migration from those hosts; c) Finding destination hosts for the selected VMs. Therefore, the proposed VM consolidation algorithm contains different models for each stage. For the first stage, we developed different fine-tuned Machine Learning (ML) prediction models for individual VMs to predict the best time to trigger migrations from hosts. For the second stage, we lexicographically consider migration time and host CPU usage when selecting VMs to migrate. Finally, a new method based on the Best Fit Decreasing (BFD) algorithm was developed to select a destination host for the VMs being migrated. Our algorithm was compared to a baseline VM consolidation algorithm that used Local Regression for detecting over-utilized hosts, minimum migration time for the VM selection stage and power-aware best fit for the host selection stage. The comparison demonstrated that our VM consolidation algorithm improved energy consumption and SLAVs by 26% and 50%, respectively.

An On-Line Virtual Machine Consolidation Strategy for Dual Improvement in Performance and Energy Conservation of Server Clusters in Cloud Data Centers

As data centers are consuming massive amount of energy, improving the energy efficiency of cloud computing has emerged as a focus of research. However, it is challenging to reduce energy consumption while maintaining system performance without increasing the risk of Service Level Agreement violations. Most of the existing consolidation approaches for virtual machines (VMs) consider system performance and Quality of Service (QoS) metrics as constraints, which usually results in large scheduling overhead and impossibility to achieve effective improvement in energy efficiency without sacrificing some system performance and cloud service quality. In this article, we first define the metrics of peak power efficiency and optimal utilization for heterogeneous physical machines (PMs). Then we propose Peak Efficiency Aware Scheduling (PEAS), a novel strategy of VM placement and reallocation for achieving dual improvement in performance and energy conservation from the perspective of server clusters. PEAS allocates and reallocates VMs in an on-line manner and always attempts to maintain PMs working in their peak power efficiency via VM consolidation. Extensive experiments on Cloudsim show that PEAS outperforms several energy-aware consolidation algorithms with regard to energy consumption, system performance as well as multiple QoS metrics.

The $$CiS^2$$: a new metric for performance and energy trade-off in consolidated servers

The increased use of cloud services has turned the virtualization in the main technology that supports cloud datacentres. To reduce the increment of power consumption caused in datacenters due to the addition of physical servers, system administrators are using virtual machine consolidation (VMC) techniques which tries to allocate the adequate number of virtual machines per physical server. Therefore, VMC increases server resources utilization and as a consequence its performance degradation and the energy consumption too. Then, a trade-off between the performance and the energy consumption exists when consolidating virtual machines. This trade-off is difficult to quantify and also to determine the servers efficiency taking into account a specific number of allocated virtual machines. Because of this, it is crucial for system administrators having a simple metric that assists the VMC making-decision process. In this paper, we propose the $$CiS^2$$ index, a metric to quantify this performance-energy trade-off. Also, this index can help system administrators to decide about the servers’ efficiency through benchmarking and to select the most efficient server through a proposed algorithm. Besides, we propose a simple graphical representation of the index to distinguish graphically the efficient and non-efficient server consolidations. We validate the index in a theoretical manner and performing real experiments in different physical servers under CPU workload saturation. Obtained results show that the proposed index reflects the performance-energy trade-off behaviour and it helps systems’ administrators when consolidating virtual machines.

Virtual Machine Consolidation with Multiple Usage Prediction for Energy-Efficient Cloud Data Centers

Virtual machine consolidation aims at reducing the number of active physical servers in a data center so as to decrease the total power consumption. In this context, most of the existing solutions rely on aggressive virtual machine migration, thus resulting in unnecessary overhead and energy wastage. Besides, virtual machine consolidation should take into account multiple resource types at the same time, since CPU is not the only critical resource in cloud data centers. In fact, also memory and network bandwidth can become a bottleneck, possibly causing violations in the service level agreement. This article presents a virtual machine consolidation algorithm with multiple usage prediction (VMCUP-M) to improve the energy efficiency of cloud data centers. In this context, multiple usage refers to both resource types and the horizon employed to predict future utilization. Our algorithm is executed during the virtual machine consolidation process to estimate the long-term utilization of multiple resource types based on the local history of the considered servers. The joint use of current and predicted resource utilization allows for a reliable characterization of overloaded and underloaded servers, thereby reducing both the load and the power consumption after consolidation. We evaluate our solution through simulations on both synthetic and real-world workloads. The obtained results show that consolidation with multiple usage prediction reduces the number of migrations and the power consumption of the servers while complying with the service level agreement.

A Virtual Machine Consolidation Algorithm Based on Ant Colony System and Extreme Learning Machine for Cloud Data Center

The energy consumption issue of large-scale data centers is attracting more and more attention. Virtual machine consolidation can significantly reduce energy consumption by migrating virtual machines from one physical machine to another. However, excessive virtual machine consolidation can lead to dangerous Service Level Agreement (SLA) violations. Therefore, how to balance between effective energy consumption and SLA violations avoidance effectively is a paradox to be mediated. The virtual machine consolidation problem is NP-hard. The traditional heuristic algorithm is easy to fall into the local optimal and some meta-heuristic algorithms can help to avoid it. However, the existing meta-heuristic algorithms are with high complexity. Therefore, we propose a lower complexity multi-population ant colony system algorithm with the Extreme Learning Machine (ELM) prediction (ELM_MPACS). The algorithm firstly predicts the host state employing ELM and then the virtual machine on the overloaded host will be migrated to the normal host, while the virtual machine on the underloaded host will be consolidated to another underloaded host with higher utilization. Multiple populations concurrently construct migration plans and local search further optimizes the results obtained by each population to reduce SLA violations. We compare ELM_MPACS with the benchmark, heuristic and meta-heuristic algorithms. The experimental results have shown that compared with these algorithms, our algorithm reduces energy consumption, migration times and SLA violations effectively.

Cluster-based optimal VM placement using crow search algorithm for cloud data centres

The extensive use of computational power from data centres causes huge energy consumption. A good virtual machine (VM) placement strategy would make better consolidation of VMs in a data centre (DC) that reduces energy consumption. However, it is hard to balance hosts in DCs due to the workload fluctuation by application and scaling of VMs. The optimal decision on VM placement and consolidation is an NP-hard problem and many researchers have proposed solutions to tackle this problem but they lack efficient exploitation of the mechanisms. Therefore, this paper proposes a hierarchical cluster-based approach with a meta-heuristic crow search algorithm (CSA) for the optimal selection of hosts to place the VMs and consolidate the maximum number of VMs on a minimum number of hosts. The work is simulated in CloudSim using real workload traces. Experimental results show that proposed work reduces energy consumption, SLA violations and VM migrations while ensuring better resource utilisation.

Energy-efficient and Dynamic Consolidation of Virtual Machines in OpenStack-based Private Cloud

Abstract Dynamic Virtual Machines (VMs) consolidation is an efficient technique for enhancing resource utilization and reducing energy consumption by the physical servers. Over-utilized host affects the degradation of resource utilization and VM performance. Determining when it is best to migrate VMs from an over-utilized host to an under-utilized host is the aim of dynamic Live VM Migration (LVM). It influences the resource utilization and Quality of Services (QoS) offered by cloud providers. In the OpenStack cloud, LVM is done manually, and it uses first-fit as a default migration technique, which is not efficient for resource utilization. The paper proposes a novel approach for the dynamic consolidation of VMs in the OpenStack cloud. The CPU utilization, RAM utilization, and the number of instances of each host are monitored over a period of time, and overload detection of a host is carried out using the SVM classification model. The Load balancing consolidation is performed based on the SVM classification result. Further, an energy-efficient consolidation achieved to optimize the energy of physical servers. We carry out the performance analysis of the proposed work in the OpenStack-based real-time testbed of five-node. The result reveals the fair distribution of load within the servers and the significant benefits of energy-efficient VM consolidation resulting up to 15% energy savings.

Comparative analysis of VM consolidation algorithms for cloud computing

Virtual machine consolidation is a major solution for addressing the issue of increasing energy consumption by cloud computing data centers. A lot of work is done on developing algorithms for detecting underloaded, overloaded hosts, selection of virtual machines and their placement to perform the consolidation. These algorithms are usually tested on publicly available Planet lab workload. There is a need to know how benchmarks algorithms used in consolidation of virtual machines respond to other workloads. This paper is an attempt to evaluate these algorithms on Google workload trace. An importer is made to use this dataset by extending the CloudSim toolkit. The comparison of results using Planet lab and Google workload traces is made which shows the difference of 46.41%, 14.84%, 12.86% and 44.83% respectively in terms of number of virtual machine migrations, service level agreement violation time per active host, number of hosts shutdown, and energy consumption. The objective comparison of results illustrated that there is a need to test the proposed algorithms on multiple datasets in order to be assessed as optimal.

Hybrid Algorithm Naive Bayesian Classifier and Random Key Cuckoo Search for Virtual Machine Consolidation Problem

The trade-off between Energy consumption and SLA violation presents a serious challenge in cloud computing environments. A non-aggressive virtual machine consolidation algorithm is a good approach to reduce the consumed energy as well as SLA violation. A well-known strategy to deal with the virtual machine consolidation problem consists of four steps: host overloading detection, host under-loading detection, virtual machine selection and virtual machine placement. In this paper, the previous strategy is modified by merging the last two steps virtual machine selection and virtual machine placement, to avoid any poor solutions caused by solving both steps separately. In the host overloading/under-loading detection steps, we classified host status into five classes: Over-Utilized, Nearly Over-Utilized, Normal Utilized, Under-Utilized and Switched Off, then an algorithm, based on the Naive Bayesian Classifier, was introduced in order to detect the future host state for minimizing the number of virtual machine migrations; as a result, the energy consumption and performance degradation due to migrations will be minimized. In the virtual machine selection and placement steps, we introduced an algorithm based on the Random Key Cuckoo Search to reduce the energy consumption and enhance the SLA violation. To assess the algorithm, real data traces for 10 days, were used to verify the proposed algorithms. The experimental results proved that the proposed algorithms can significantly reduce the consumed energy as well as the SLA violation in data centers.

Predicting Multi-Attribute Host Resource Utilization Using Support Vector Regression Technique

Studies on the resource workload demand in cloud computing environment aim at reducing resource wastage by optimizing the resource utilization in a cloud data center. Based on this goal, most of the existing approaches rely on resource management mechanisms such as resource allocation and Virtual Machine (VM) consolidation to reach an ideal solution for reducing resource wastage. Because of instability and high variability of the cloud resource usage and workloads, there is a demand for cloud providers to apply the prediction methods for forecasting the future cloud resource utilization. This paper employs a supervised statistical learning method, i.e., Support Vector Regression Technique (SVRT), to forecast the future usage of multi-attribute host resource. The method is particularly suitable to handle a non-linear cloud resource workload. To improve the prediction accuracy of SVRT, we decide Radial Basis Function as the kernel function of SVRT and apply Sequential Minimal Optimization Algorithm (SMOA) for the training and regression estimation of the prediction method. Besides, compared with the existing work, we consider the multi-attribute cloud resources other than the single resource. The method is employed under eight sets of real-world workloads, which are collected from BitBrain (BB), PlanetLab (PL) and Google Cluster Workload Traces (GCWT). Series of experiments conducted on the workload dataset show the effectiveness of our approach. Based on evaluation metrics, the final results show that the accuracy was enhanced by approximately 4%-16% and the error percentage was reduced by approximately 8%-60% compared with the state-of-the-art methods.

A Review on Virtual Machine Positioning and Consolidation Strategies for Energy Efficiency in Cloud Data Centers

The cloud data center consumes massively more and more energy which is considered inacceptable. Therefore further efforts are needed to improve the energy efficiency of such data centers by using Server Consolidation to minimize the number of Active Physical Machines (APMs) in a data center setting. Strategies for positioning and transformation of VM maintain their usefulness as a roadmap to maximum consolidation. The latest techniques do complex restructuring, thus optimizing VM's positioning. The paper provides a detailed state-of - the-art strategies for VM positioning and consolidation that help improve energy efficiency in cloud data centers. A comparison is provided here between the strategies that revealed the worthiness, limitations and suggestions of strengthening other methods along the way.

CPU and RAM Energy-Based SLA-Aware Workload Consolidation Techniques for Clouds

Cloud computing offers hardware and software resources delivered as services. It provides solutions for dynamic as well as “pay as you go” provision of resources. Energy consumption of these resources is high which leads to higher operational costs and carbon emissions in data centers. A number of research studies have been conducted on energy efficiency of data centers, but most of them concentrate on single factor energy consumption, i.e., energy consumed by CPU only, and energy consumption by Random Access Memory (RAM) is neglected. However, recently the focus has been turned towards impact of energy consumption by RAM on data centers. Studies have shown that RAM consumes about 25% of joint energy consumed by a server’s CPU and RAM. In this paper, two energy-aware virtual machine (VM) consolidation schemes are proposed that take into account a server’s capacity in terms of CPU and RAM to reduce the overall energy consumption. The proposed schemes are compared with existing schemes using CloudSim simulator. The results show that the proposed schemes reduce the energy cost with improved Service Level Agreement (SLA).

A Dynamic Virtual Machine Resource Consolidation Strategy Based on a Gray Model and Improved Discrete Particle Swarm Optimization

With the continuous expansion of the scale of cloud datacenters, high energy consumption has become a problem. Virtual machine (VM) consolidation is an effective method of energy management, but overly aggressive consolidation has caused issues such as high service level agreement violation (SLA) rates and excessive migrations. To reduce energy consumption while ensuring the quality of service of a datacenter, a VM dynamic consolidation strategy based on a gray model and an improved discrete particle swarm algorithm (GM-DPSO) is proposed. This strategy uses a gray prediction algorithm for load detection and adjusts the size of the threshold according to the load condition. Then, we select VMs for overloaded and underloaded hosts and complete the placement of VMs through a VM placement strategy based on an improved discrete particle swarm algorithm. This method aims to improve the load balance, establish a mapping between VMs and hosts, use particles to search for the best VM placement in the global scope, and reduce the probability of SLAV. Our algorithm considers the impact of VM migration on system energy consumption and service quality and takes sufficient measures to reduce the number of migrations. To verify the effectiveness and practicality of this strategy, experiments were conducted on actual workloads, and the results were compared with those of other strategies. The experimental results show that the GM-DPSO greatly improves service quality while reducing energy consumption and SLAV by 34.53% and 97.53%, respectively. The methods and theories in this paper provide strong theoretical and practical engineering guidance for large-scale cloud deployment.

Performance-to-Power Ratio Aware Resource Consolidation Framework Based on Reinforcement Learning in Cloud Data Centers

Dynamic consolidation of virtual machines (VMs) is presented as a significant technique of energy conservation in cloud environments, which can eliminate the hotspot of overloaded hosts and switch the under loaded hosts to sleep mode through live migration of virtual machines. However, since the fact that migrating VM consumes a certain amount of extra resources, the process of reallocation can cause Service Level Agreement (SLA) violations. In this paper, a novel proactive framework which considers both predicted resource utilization and Performance-to-power Ratio (PPR) of heterogeneous hosts is proposed to perform dynamic VM consolidation to achieve balance of performance and energy. More precisely, a workload predictor is proposed based on the modified Weighted Moving Average (WMA) algorithm, representing the support for dynamic resource allocation; a cluster controller is proposed based on reinforcement learning for exploring the optimal matching relationship between resource requests and host at different PPR levels; a resource allocator is designed based on greedy strategy for achieving the trade-off between energy consumption and application performance across the cluster. Moreover, the framework is implemented based on distributed architecture and off-line learning pattern, which are able to not only scale up quickly but also improve the computing efficiency of the system. To validate the effectiveness of the proposed method, we have performed experimental evaluation on CloudSim with real-world workload traces of PlanetLab, and the simulation results demonstrate that it reduces the energy consumption up to 45.25% and effectively deals with high Quality of Service (QoS) requirements and heterogeneous distributed infrastructures in comparison with other competitive approaches.

PAPSO: A Power-Aware VM Placement Technique Based on Particle Swarm Optimization

With the widespread usage of cloud computing to benefit from its services, cloud service providers have invested in constructing large scale data centers. Consequently, a tremendous increase in energy consumption has arisen in conjunction with its results, including a remarkable rise in costs of operating and cooling servers. Besides, increasing energy consumption has a significant impact on the environment due to emissions of carbon dioxide. Dynamic consolidation of Virtual Machines (VMs) into the minimal number of Physical Machines (PMs) is considered as one of the magic solutions to manage power consumption. The virtual machine placement problem is a critical issue for good VM consolidation. This paper proposes a Power-Aware technique depending on Particle Swarm Optimization (PAPSO) to determine the near-optimal placement for the migrated VMs. A discrete version of Particle Swarm Optimization (PSO) is adopted based on a decimal encoding to map the migrated VMs to the best appropriate PMs. Furthermore, an effective minimization fitness function is employed to reduce power consumption without violating the Service Level Agreement (SLA). Specifically, PAPSO consolidates the migrated VMs into the minimum number of PMs with a major constraint to decrease the number of overloaded hosts as much as possible. Therefore, the number of VM migrations can be reduced drastically by taking into consideration the main sources for VM migrations; overloaded hosts and underloaded ones. PAPSO is implemented in CloudSim and the experimental results on random workloads with different sizes of VMs and PMs show that PAPSO does not violate SLA and outperforms the Power-Aware Best Fit Decreasing algorithm (PABFD). It can reduce about 8.01%, 39.65%, 66.33%, and 11.87% on average in terms of consumed energy, number of VM migrations, number of host shutdowns and the combined metric Energy SLA Violation (ESV), respectively.

Adaptive Markov‐based approach for dynamic virtual machine consolidation in cloud data centers with quality‐of‐service constraints

SummaryDynamic virtual machine (VM) consolidation is one of the emerging technologies that has been considered for low‐cost computing in cloud data centers. Quality‐of‐service (QoS) assurance is one of the challenging issues in the VM consolidation problem since it is directly affected by the increase of resource utilization due to the consolidations. In this paper, we take advantage of Markov chain models to propose a novel approach for VM consolidation that can be used to explicitly set a desired level of QoS constraint in a data center to ensure the QoS goals while improving system utilization. For this purpose, an energy‐efficient and QoS‐aware best fit decreasing algorithm for VM placement is proposed, which considers QoS objective when determining the location of a migrating VM. This algorithm employs an online transition matrix estimator method to deal with the nonstationary nature of real workload data. We also propose new policies for detecting overloaded and underloaded hosts. The performance of our proposed algorithms is evaluated through simulations. The results show that the proposed VM consolidation algorithms in this paper outperforms the benchmark algorithms in terms of energy consumption, service‐level agreement violations, and other cost factors.

Efficient virtual machine placement algorithms for consolidation in cloud data centers

Dynamic Virtual Machine (VM) consolidation is a successful approach to improve the energy efficiency and the resource utilization in cloud environments. Consequently, optimizing the online energy-performance tradeoff directly influences quality of service. In this study, algorithms named as CPU Priority based Best-Fit Decreasing (CPBFD) and Dynamic CPU Priority based Best-Fit Decreasing (DCPBFD) are proposed for VM placement. A number of VM placement algorithms are implemented and compared with the proposed algorithms. The algorithms are evaluated through simulations with real-world workload traces and it is shown that the proposed algorithms outperform the known algorithms. The simulation results clearly show that CPBFD and DCPBFD provide the least service level agreement violations, least VM migrations, and efficient energy consumption.

A Novel Energy Efficient Scheduling for VM Consolidation and Migration in Cloud Data Centers

A Novel Energy Efficient Scheduling for VM Consolidation and Migration in Cloud Data Centers