Virtual Machine Consolidation Algorithm Research Articles

An efficient dynamic migration and consolidation method of VMs based on improved K-nearest neighbor algorithm and attribute weighting

Virtual machine (VM) consolidation and migration that only consider current workload can result in excessive unnecessary migrations. To address this issue, a VM consolidation algorithm based on resource utilization prediction is proposed. An improved K-nearest neighbor (KNN) classification algorithm weighted by attribute inconsistency is proposed to predict the workload of both the host and the VMs. Firstly, two distributions are partitioned according to the neighboring relationship for comparing consistency. Then, an inconsistency evaluation function based on earth mover’s distance (EMD) is designed to measure the inconsistency between the neighboring sample set of each sample under each attribute and the equivalent partition refined by the decision attribute. Finally, the inconsistency level of the neighboring samples is transformed into the importance of the corresponding attribute to implement the attribute weighting KNN classifier. When selecting the source host and target host for VM migration, both current and predicted overloads are considered to avoid unnecessary VM migrations. Simulation tests were performed with random and realistic workloads, and the results show that the proposed method can reduce the overall energy consumption of the host, while also reducing service level agreement (SLA) violations and VM migration.

A resource scheduling method for cloud data centers based on thermal management

With the rapid growth of cloud computing services, the high energy consumption of cloud data centers has become a critical concern of the cloud computing society. While virtual machine (VM) consolidation is often used to reduce energy consumption, excessive VM consolidation may lead to local hot spots and increase the risk of equipment failure. One possible solution to this problem is to utilize thermal-aware scheduling, but existing approaches have trouble realizing the balance between SLA and energy consumption. This paper proposes a novel method to manage cloud data center resources based on thermal management (TM-VMC), which optimizes total energy consumption and proactively prevents hot spots from a global perspective. Its VM consolidation process includes four phases where the VMs scheduler uses an improved ant colony algorithm (UACO) to find appropriate target hosts for VMs based on server temperature and utilization status obtained in real-time. Experimental results show that the TM-VMC approach can proactively avoid data center hot spots and significantly reduce energy consumption while maintaining low Service Level Agreement (SLA) violation rates compared to existing mainstream VM consolidation algorithms with workloads from real-world data centers.

An Efficient Virtual Machine Consolidation Algorithm for Cloud Computing

With the rapid development of integration in blockchain and IoT, virtual machine consolidation (VMC) has become a heated topic because it can effectively improve the energy efficiency and service quality of cloud computing in the blockchain. The current VMC algorithm is not effective enough because it does not regard the load of the virtual machine (VM) as an analyzed time series. Therefore, we proposed a VMC algorithm based on load forecast to improve efficiency. First, we proposed a migration VM selection strategy based on load increment prediction called LIP. Combined with the current load and load increment, this strategy can effectively improve the accuracy of selecting VM from the overloaded physical machines (PMs). Then, we proposed a VM migration point selection strategy based on the load sequence prediction called SIR. We merged VMs with complementary load series into the same PM, effectively improving the stability of the PM load, thereby reducing the service level agreement violation (SLAV) and the number of VM migrations due to the resource competition of the PM. Finally, we proposed a better virtual machine consolidation (VMC) algorithm based on the load prediction of LIP and SIR. The experimental results show that our VMC algorithm can effectively improve energy efficiency.

A Virtual Machine Consolidation Algorithm Based on Dynamic Load Mean and Multi-Objective Optimization in Cloud Computing

High energy consumption and low resource utilization have become increasingly prominent problems in cloud data centers. Virtual machine (VM) consolidation is the key technology to solve the problems. However, excessive VM consolidation may lead to service level agreement violations (SLAv). Most studies have focused on optimizing energy consumption and ignored other factors. An effective VM consolidation should comprehensively consider multiple factors, including the quality of service (QoS), energy consumption, resource utilization, migration overhead and network communication overhead, which is a multi-objective optimization problem. To solve the problems above, we propose a VM consolidation approach based on dynamic load mean and multi-objective optimization (DLMM-VMC), which aims to minimize power consumption, resources waste, migration overhead and network communication overhead while ensuring QoS. Fist, based on multi-dimensional resources consideration, the host load status is objectively evaluated by using the proposed host load detection algorithm based on the dynamic load mean to avoid an excessive VM consolidation. Then, the best solution is obtained based on the proposed multi-objective optimization model and optimized ant colony algorithm, so as to ensure the common interests of cloud service providers and users. Finally, the experimental results show that compared with the existing VM consolidation methods, our proposed algorithm has a significant improvement in the energy consumption, QoS, resources waste, SLAv, migration and network overhead.

I-Neat: An intelligent framework for adaptive virtual machine consolidation

With the increasing use of cloud computing, high energy consumption has become one of the major challenges in cloud data centers. Virtual Machine (VM) consolidation has been proven to be an efficient way to optimize energy consumption in data centers, and many research works have proposed to optimize VM consolidation. However, the performance of different algorithms is related with the characteristics of the workload and system status; some algorithms are suitable for Central Processing Unit (CPU)-intensive workload and some for web application workload. Therefore, an adaptive VM consolidation framework is necessary to fully explore the potential of these algorithms. Neat is an open-source dynamic VM consolidation framework, which is well integrated into OpenStack. However, it cannot conduct dynamic algorithm scheduling, and VM consolidation algorithms in Neat are few and basic, which results in low performance for energy saving and Service-Level Agreement (SLA) avoidance. In this paper, an Intelligent Neat framework (I-Neat) is proposed, which adds an intelligent scheduler using reinforcement learning and a framework manager to improve the usability of the system. The scheduler can select appropriate algorithms for the local manager from an algorithm library with many load detection algorithms. The algorithm library is designed based on a template, and in addition to the algorithms of Neat, I-Neat adds six new algorithms to the algorithm library. Furthermore, the framework manager helps users add self-defined algorithms to I-Neat without modifying the source code. Our experimental results indicate that the intelligent scheduler and these novel algorithms can effectively reduce energy consumption with SLA assurance.

Dynamic Virtual Machine Consolidation Algorithm Based on Balancing Energy Consumption and Quality of Service

Virtual machine consolidation (VMC) is an effective way to solve the problems of high power consumption and low utilization in cloud data centers. However, large-scale virtual machine migrations (VMMs) can result in additional workloads, service-level agreement violations (SLAVs), and considerable energy consumption (EC). Existing studies have made great progress in this respect, but the following problems remain: first, the potential overload of the physical host is not considered in the load detection of the physical host; second, the resource-demand scaling of physical hosts is not considered during virtual machine (VM) placement, which results in the lack of accuracy in selecting suitable hosts. In view of the above problems, this study firstly constructs a virtual resource consolidation model based on green energy conservation (GEC-VRCM), which defines the specific process and related attributes of VMC, which is beneficial to improve the consolidation efficiency of virtual resources. Second, based on this model, we propose a dynamic virtual machine consolidation algorithm based on balancing energy consumption and quality of service (EQ-DVMCA) to achieve efficient consolidation of virtual resources. Finally, experiments show that, compared with the selected 12 benchmark algorithms and two advanced VMC algorithms, EQ-DVMCA not only reduces the number of VMMs and EC, but also maintains a high level of Quality of Service (QoS) and achieves a balance between EC and QoS.

Application of virtual machine consolidation in cloud computing systems

Cloud systems play a vital and significant role in our daily lives due to various internet services. For instance, email services, social networks, and others. Consequently, their energy consumption has also become an increasingly essential concern in Cloud Computing Systems (CCSs). Virtualization technologies are widely used to facilitate the management of CCSs and reduce their energy consumption. Virtualization enables live migration of Virtual Machines (VMs) where several VMs can be loaded on some Physical machines (PMs) called VM consolidation. A VM consolidation algorithm can be an effective technique for reducing energy consumption, operational cost, hardware cost, Service Level Agreements (SLAs) compliance/violation, CO2 emissions, and enhancing the hardware and service reliability, performance, and hardware lifetime, load balancing, and utilization in CCSs. Essentially, VM consolidation must minimize energy utilization and service quality in the cloud system. This study presents a taxonomy comprising resource assignment method, metrics, objective functions, migration methods, algorithmic methods, co-location criteria of VMs, architectures, workload dataset, and evaluation approaches in VM consolidation CCSs. Also, we reviewed related work regarding the resources of PMs, algorithm methods, metrics, architectures, and the objectives in static/dynamic VM consolidation.

An improved thermodynamic simulated annealing-based approach for resource-skewness-aware and power-efficient virtual machine consolidation in cloud datacenters

Cloud computing attracted great attention in both industry and research communities for the sake of its ubiquitous, elasticity and economic services. The first class concern of cloud providers is power management for both reducing their total cost of ownership and green computing objectives. To reach the goal, a system framework is presented which has different modules. The main concentration of the paper is on virtual machine (VM) consolidation module which launches users requested VMs on the minimum number of active servers to reduce datacenter total power consumption (TPC). In this paper, the VMs consolidation is abstracted to two-dimensional bin-packing problem and also is formulated to an integer linear programming. Since the papers in the literature scarcely are aware of skewness in resources of requested VMs and for discrete nature of search space, this paper presents the resource skewness-aware VMs consolidation algorithm based on improved thermodynamic simulated annealing approach because resource skewness potentially compels the algorithm to activate additional servers. The proposed SA-based algorithm is validated in extensive scenarios with different resource skewness in comparison with two heuristics and two meta-heuristics. The average results reported from different scenarios proves superiority of proposed algorithm in comparison with other approaches in terms of the number of used servers, TPC, and total resource wastage of datacenter.

Energy and resource efficient workflow scheduling in a virtualized cloud environment

High energy consumption (EC) is one of the leading and interesting issue in the cloud environment. The optimization of EC is generally related to scheduling problem. Optimum scheduling strategy is used to select the resources or tasks in such a way that system performance is not violated while minimizing EC and maximizing resource utilization (RU). This paper presents a task scheduling model for scheduling the tasks on virtual machines (VMs). The objective of the proposed model is to minimize EC, maximize RU, and minimize workflow makespan while preserving the task’s deadline and dependency constraints. An energy and resource efficient workflow scheduling algorithm (ERES) is proposed to schedule the workflow tasks to the VMs and dynamically deploy/un-deploy the VMs based on the workflow task’s requirements. An energy model is presented to compute the EC of the servers. Double threshold policy is used to perceive the server’ status i.e. overloaded/underloaded or normal. To balance the workload on the overloaded/underloaded servers, live VM migration strategy is used. To check the effectiveness of the proposed algorithm, exhaustive simulation experiments are conducted. The proposed algorithm is compared with power efficient scheduling and VM consolidation (PESVMC) algorithm on the accounts of RU, energy efficiency and task makespan. Further, the results are also verified in the real cloud environment. The results demonstrate the effectiveness of the proposed ERES algorithm.

Embedding individualized machine learning prediction models for energy efficient VM consolidation within Cloud data centers

The fast growth in demand for utility-based IT services has lead to the formation of large scale Cloud data centers. The electrical energy consumption of these data centers results in high operational costs and carbon dioxide emissions. Cloud data centers benefit from the use of virtualization technology to reduce their energy consumption. This technology enables a Cloud data center to allocate its physical resources (CPU, memory, hard disk, network bandwidth) on demand and balance loads between their physical hosts by live migration of Virtual Machines (VMs). However, the migration of VMs can result in Service Level Agreement Violations (SLAVs) and consequently low Quality of Service (QoS). Hence, in this paper, we propose an energy aware VM consolidation algorithm that minimizes SLAVs.Dynamic VM consolidation has three stages: a) Detecting over- and under-utilized hosts; b) Selecting one or more VMs for migration from those hosts; c) Finding destination hosts for the selected VMs. Therefore, the proposed VM consolidation algorithm contains different models for each stage. For the first stage, we developed different fine-tuned Machine Learning (ML) prediction models for individual VMs to predict the best time to trigger migrations from hosts. For the second stage, we lexicographically consider migration time and host CPU usage when selecting VMs to migrate. Finally, a new method based on the Best Fit Decreasing (BFD) algorithm was developed to select a destination host for the VMs being migrated. Our algorithm was compared to a baseline VM consolidation algorithm that used Local Regression for detecting over-utilized hosts, minimum migration time for the VM selection stage and power-aware best fit for the host selection stage. The comparison demonstrated that our VM consolidation algorithm improved energy consumption and SLAVs by 26% and 50%, respectively.

Virtual Machine Consolidation with Multiple Usage Prediction for Energy-Efficient Cloud Data Centers

Virtual machine consolidation aims at reducing the number of active physical servers in a data center so as to decrease the total power consumption. In this context, most of the existing solutions rely on aggressive virtual machine migration, thus resulting in unnecessary overhead and energy wastage. Besides, virtual machine consolidation should take into account multiple resource types at the same time, since CPU is not the only critical resource in cloud data centers. In fact, also memory and network bandwidth can become a bottleneck, possibly causing violations in the service level agreement. This article presents a virtual machine consolidation algorithm with multiple usage prediction (VMCUP-M) to improve the energy efficiency of cloud data centers. In this context, multiple usage refers to both resource types and the horizon employed to predict future utilization. Our algorithm is executed during the virtual machine consolidation process to estimate the long-term utilization of multiple resource types based on the local history of the considered servers. The joint use of current and predicted resource utilization allows for a reliable characterization of overloaded and underloaded servers, thereby reducing both the load and the power consumption after consolidation. We evaluate our solution through simulations on both synthetic and real-world workloads. The obtained results show that consolidation with multiple usage prediction reduces the number of migrations and the power consumption of the servers while complying with the service level agreement.

A Virtual Machine Consolidation Algorithm Based on Ant Colony System and Extreme Learning Machine for Cloud Data Center

The energy consumption issue of large-scale data centers is attracting more and more attention. Virtual machine consolidation can significantly reduce energy consumption by migrating virtual machines from one physical machine to another. However, excessive virtual machine consolidation can lead to dangerous Service Level Agreement (SLA) violations. Therefore, how to balance between effective energy consumption and SLA violations avoidance effectively is a paradox to be mediated. The virtual machine consolidation problem is NP-hard. The traditional heuristic algorithm is easy to fall into the local optimal and some meta-heuristic algorithms can help to avoid it. However, the existing meta-heuristic algorithms are with high complexity. Therefore, we propose a lower complexity multi-population ant colony system algorithm with the Extreme Learning Machine (ELM) prediction (ELM_MPACS). The algorithm firstly predicts the host state employing ELM and then the virtual machine on the overloaded host will be migrated to the normal host, while the virtual machine on the underloaded host will be consolidated to another underloaded host with higher utilization. Multiple populations concurrently construct migration plans and local search further optimizes the results obtained by each population to reduce SLA violations. We compare ELM_MPACS with the benchmark, heuristic and meta-heuristic algorithms. The experimental results have shown that compared with these algorithms, our algorithm reduces energy consumption, migration times and SLA violations effectively.

Hybrid Algorithm Naive Bayesian Classifier and Random Key Cuckoo Search for Virtual Machine Consolidation Problem

The trade-off between Energy consumption and SLA violation presents a serious challenge in cloud computing environments. A non-aggressive virtual machine consolidation algorithm is a good approach to reduce the consumed energy as well as SLA violation. A well-known strategy to deal with the virtual machine consolidation problem consists of four steps: host overloading detection, host under-loading detection, virtual machine selection and virtual machine placement. In this paper, the previous strategy is modified by merging the last two steps virtual machine selection and virtual machine placement, to avoid any poor solutions caused by solving both steps separately. In the host overloading/under-loading detection steps, we classified host status into five classes: Over-Utilized, Nearly Over-Utilized, Normal Utilized, Under-Utilized and Switched Off, then an algorithm, based on the Naive Bayesian Classifier, was introduced in order to detect the future host state for minimizing the number of virtual machine migrations; as a result, the energy consumption and performance degradation due to migrations will be minimized. In the virtual machine selection and placement steps, we introduced an algorithm based on the Random Key Cuckoo Search to reduce the energy consumption and enhance the SLA violation. To assess the algorithm, real data traces for 10 days, were used to verify the proposed algorithms. The experimental results proved that the proposed algorithms can significantly reduce the consumed energy as well as the SLA violation in data centers.

Adaptive Markov‐based approach for dynamic virtual machine consolidation in cloud data centers with quality‐of‐service constraints

SummaryDynamic virtual machine (VM) consolidation is one of the emerging technologies that has been considered for low‐cost computing in cloud data centers. Quality‐of‐service (QoS) assurance is one of the challenging issues in the VM consolidation problem since it is directly affected by the increase of resource utilization due to the consolidations. In this paper, we take advantage of Markov chain models to propose a novel approach for VM consolidation that can be used to explicitly set a desired level of QoS constraint in a data center to ensure the QoS goals while improving system utilization. For this purpose, an energy‐efficient and QoS‐aware best fit decreasing algorithm for VM placement is proposed, which considers QoS objective when determining the location of a migrating VM. This algorithm employs an online transition matrix estimator method to deal with the nonstationary nature of real workload data. We also propose new policies for detecting overloaded and underloaded hosts. The performance of our proposed algorithms is evaluated through simulations. The results show that the proposed VM consolidation algorithms in this paper outperforms the benchmark algorithms in terms of energy consumption, service‐level agreement violations, and other cost factors.

An intelligent ensemble learning approach for energy efficient and interference aware dynamic virtual machine consolidation

Inefficient resource usage is one of the greatest causes of high energy consumption in cloud data centers. Virtual Machine (VM) consolidation is an effective method for improving energy related costs and environmental sustainability for modern data centers. While dynamic VM consolidation algorithms can improve energy efficiency, virtualisation technologies cannot guarantee performance isolation between co-located VMs resulting in interference issues. We address the problem by introducing an energy and interference aware VM consolidation algorithm which uses predictive modelling to classify workloads using their resource usage features to make more informed consolidation decisions. The use of ensemble methods plays a pivotal role for improving predictive performance for many different problems. Using recent workload data from Microsoft Azure we present a comparative analysis of several ensemble methods using state-of-the-art prediction models and propose an ensemble based VM consolidation algorithm. Our empirical results demonstrate how our approach improves energy efficiency by 34% while also reducing service violations by 77%.

Migration Cost and Energy-Aware Virtual Machine Consolidation Under Cloud Environments Considering Remaining Runtime

By live migration technology, multiple virtual machines (VMs) can be consolidated into a fewer physical servers and the idle ones can be shut down or switched to low-power mode, thus reducing the energy consumption of cloud data centers. However, live migration can result in performance degradation of migrated VMs, or even interrupting their services. At the same time, live migration can also aggravate the overheads of data transmissions and produce additional energy consumption in cloud data centers. All these negative influences belong to migration cost (MC) caused by VM migration, which becomes an important cost factor that can’t be ignored. Otherwise, another important concern, remaining runtime of the migrated VM, also has influence on the efficiency of VM consolidation, which is not well addressed as well. This paper investigates MC-aware VM consolidation problem and formulates the problem as a multi-constraint optimization model by considering migration cost and remaining runtime of VMs. Based on the proposed model, a heuristic algorithm, called MC-aware VM consolidation (MVC) algorithm, is developed. Finally, based on a real-world cloud trace, we conduct extensive experimental studies to verify the validity of the proposed model and algorithm. Experimental results show that, compared with some popular algorithms, MVC algorithm can effectively decrease the migration cost and, at the same time guarantee the energy consumption within a certain low level.

A Workload-Aware VM Consolidation Method Based on Coalitional Game for Energy-Saving in Cloud

Virtual Machine (VM) consolidation technique plays an important role in energy management and load-balancing of cloud computing systems. Dynamic VM consolidation is a promising consolidation approach in this direction, which aims at using least active physical machines (PMs) through appropriately migrating VMs to reduce resource consumption. The resulting optimization problem is well-acknowledged to be NP-hard optimization problems. In this paper, we propose a novel merge-and-split-based coalitional game-theoretic approach for VM consolidation in heterogeneous clouds. The proposed approach first partitions PMs into different groups based on their workload levels, then employs a coalitional-game-based VM consolidation algorithm (CGMS) in choosing members from such groups to form effective coalitions, performs VM migrations among the coalition members to maximize the payoff of every coalition, and finally keeps PMs running in a high energy-efficiency state. The simulation results based on three scenarios clearly suggest that our proposed approach outperforms traditional ones in terms of energy-saving, and also achieve a fair level of load balance.

Review of the quality of service scheduling mechanisms in cloud

Cloud computing provides on demand access to a large pool of heterogeneous computational and storage resources to users over the internet. Optimal scheduling mechanisms are needed for the efficient management of these heterogeneous resources. The optimal scheduler can improve the Quality of Services (QoS) as well as maintaining efficiency and fairness among these tasks. In large scale distributed systems, the performance of these scheduling algorithms is crucial for better efficiency. Now the cloud customers are charged based upon the amount of resources they are consumed or held in reserve. Comparing these scheduling algorithms from different perspectives is needed for further improvement. This paper provides a comparative study about different resource allocation, load balancing and virtual machine consolidation algorithms in cloud computing. These algorithms have been evaluated in terms of their ability to provide QoS for the tasks and Service Level Agreement (SLA) guarantee amongst the jobs served. This study identifies current and future research directions in this area for QoS enabled cloud scheduling.

Adaptive selection of dynamic VM consolidation algorithm using neural network for cloud resource management

Cloud resource management becomes more important with the increasing usage of cloud resources. With various cloud options available, cloud provider may have different priority in managing the resource through resource scheduling and provisioning. Dynamic VM (Virtual Machine) consolidation algorithm is one of the techniques which can be used to reduce energy consumption through VM migration. Higher VM migration may lead to lower energy consumption and higher SLA violation. Although previous research has successfully decreased energy consumption and SLA violation, cloud providers may need to manage trade-offs between energy and SLA violation through availability of priority in the system. This paper proposes neural network-based adaptive selection of VM consolidation algorithms which adaptively chooses appropriate algorithm according to cloud provider’s goal priority and environment parameters. Dataset generation and performance evaluation using simulations on real-world PlanetLab VMs workload trace showed that adaptive selector produced better average performance score than independent methods on various evaluation priority.

Energy efficient workflow scheduling with virtual machine consolidation for green cloud computing

Cloud computing offers utility-based IT services on-demand to the users on a pay-per-use-basis. The cloud centers consist of physical machines (PMs) with virtual machines (VMs). These data centers consume a large amount of energy due to the improper resource utilization and lack of efficient schedu ling algorithms to perform the task-resource mapping. These issues lead to huge energy consumption along with high maintenance costs and carbon emissions. In this paper, a Power Efficient Scheduling and VM Consolidation (PESVMC) algorithm is proposed to address these issues and the associated challenges. The numerous existing research works concentrated on the application of energy management techniques to hardware level support for the reduction of energy consumption. The proposed algorithm emphasizes on the software level by taking the flexibility of the virtualization technology and it consists of two phases, VM Scheduling phase, and VM Consolidation phase. In the Scheduling phase, the tasks with maximum runtime are allocated to VM, which is expected to consume minimal energy. In the VM Consolidation phase, overloaded and underloaded hosts are determined based on the double-threshold scheme. Further, Live Migration technique is applied for migrating the VMs from over-utilized or underutilized hosts to other hosts with the normal utilization. A power efficient utilization factor is introduced to determine the underloaded hosts. This utilization factor is proven to reduce the number of migrations, which can cause additional energy consumption. Energy efficient Scheduling combined with VM Consolidation is successful in maximizing the resource utilization and minimizing the energy consumption. The experimental evaluation is performed using WorkflowSim and the proposed algorithm achieves significant energy conservation and resource utilization.