Virtual Machine Consolidation Research Articles

Efficient cloud data center: An adaptive framework for dynamic Virtual Machine Consolidation

Cloud computing is a thriving and ever-expanding sector in the industry world. This growth has sparked increased interest from organizations seeking to harness its potential. However, the sheer volume of services and offerings in this field has resulted in a noticeable surge in related data. With the rapid evolution and growing demand, cloud computing resource management faces a fresh set of challenges. Resource limitations, such as high maintenance costs, elevated Energy Consumption (EC), and adherence to Service Level Agreements (SLA), are critical concerns for both the cloud computing industry and its user organizations. In this context, taking a proactive approach to resource management and Virtual Machine Consolidation (VMC) has become imperative. The logical management of resources and the consolidation of Virtual Machines (VMs) in a manner that aligns with the requirements and demands of service providers and users have garnered widespread attention. The goal of this proposed paper is to focus on addressing the VMC problem within a unified framework, divided into two main phases. The first phase deals with host workload detection and prediction, while the subsequent phase tackles the selection and allocation of appropriate VMs. In our proposed method, for the first time, we use a Granular Computing (GRC) model, which is an efficient, scalable, and human-centric computational approach. This model exhibits behaviors similar to intelligent human decision-making, as it can simultaneously consider all factors and criteria involved in the problems. We evaluated our proposed method through simulations using CloudSim on various types of workloads. Experimental results demonstrate that our proposed algorithm outperforms other algorithms in all measurement metrics.

Deep Learning Modified Reinforcement Learning with Virtual Machine Consolidation for Energy-Efficient Resource Allocation in Cloud Computing

Cloud computing has attracted significant attention because of the growing service demands of businesses that outsource computationally intensive tasks to the data center. Meanwhile, the infrastructure of a data center is comprised of hardware resources that consume a great deal of energy and release harmful levels of carbon dioxide. Cloud data centers demand massive amounts of electrical power as modern applications and organizations grow. To prevent resource waste and promote energy efficiency, virtual machines (VMs) must be dispersed over numerous physical machines (PMs) in a data center in the cloud. The actual allocation of VMs to PMs can involve more complex decision-making processes, such as considering the resource utilization, load balancing, performance requirements, and constraints of the system. Advanced techniques, like intelligent placement algorithms or dynamic resource allocation, may be employed to optimize resource utilization and achieve efficient VM distribution across multiple PMs. Cloud service suppliers aim to lower operational expenses by reducing energy consumption while offering clients competitive services. Minimizing large-scale data center power usage while maintaining the quality of service (QoS), especially for social media-based cloud computing systems, is crucial. Consolidating VMs has been highlighted as a promising method for improving resource efficiency and saving energy in data centers. This research provides deep learning augmented reinforcement learning (RL)-based energy efficient and QoS-aware virtual machine consolidation (VMC) approach to meet the difficulties. The proposed deep learning modified reinforcement learning-virtual machine consolidation (DLMRL-VMC) model can motivate both cloud providers and customers to distribute cloud infrastructure resources to achieve high CPU utilization and good energy efficiency as measured by power usage effectiveness (PUE) and data center infrastructure efficiency (DCiE). The suggested model, DLMRL-VMC, offers a VM placement approach based on resource usage and dynamic energy consumption to determine the best-matched host and VM selection strategy, Average Utilization Migration Time (AUMT). Based on AUMT, deep learning modified reinforcement learning (DLMRL) will choose a VM with a low average CPU utilization and a short migration time. The DLMRL-VMC Energy-efficient, Resource Allocation strategy is evaluated on the trace of the CloudSim VM to attain good PUE and CPU utilization.

An efficient dynamic migration and consolidation method of VMs based on improved K-nearest neighbor algorithm and attribute weighting

Virtual machine (VM) consolidation and migration that only consider current workload can result in excessive unnecessary migrations. To address this issue, a VM consolidation algorithm based on resource utilization prediction is proposed. An improved K-nearest neighbor (KNN) classification algorithm weighted by attribute inconsistency is proposed to predict the workload of both the host and the VMs. Firstly, two distributions are partitioned according to the neighboring relationship for comparing consistency. Then, an inconsistency evaluation function based on earth mover’s distance (EMD) is designed to measure the inconsistency between the neighboring sample set of each sample under each attribute and the equivalent partition refined by the decision attribute. Finally, the inconsistency level of the neighboring samples is transformed into the importance of the corresponding attribute to implement the attribute weighting KNN classifier. When selecting the source host and target host for VM migration, both current and predicted overloads are considered to avoid unnecessary VM migrations. Simulation tests were performed with random and realistic workloads, and the results show that the proposed method can reduce the overall energy consumption of the host, while also reducing service level agreement (SLA) violations and VM migration.

Energy and Performance Centric Resource Allocation Framework in Virtual Machine Consolidation Using Reinforcement Learning Approach

Abstract:: Virtual machines are used to reduce cloud platform application performance, management costs, and access irregularities. Virtual machines are frequently vulnerable to delays, overburdening workloads, and other obstacles while consolidating and migrating servers. To significantly disperse loads among virtual machines, dynamic consolidation techniques are implemented to control energy dissipation, monitor overloading, and address underloading problems. Background:: The process of consolidation involves more calculations and resources in order to transfer services between virtual machines, provided that Service Level Agreements are observed. Methods:: The suggested approach promotes the use of cutting-edge architecture to combine virtual machines, and, therefore, strike a balance between performance and energy requirements. The main design considerations for the suggested Dynamic Weightage algorithm, which includes the clustering approach in relation to reinforcement learning approaches, are overall resource needs and Performance to Power Ratio (PPR). A cluster of ideal virtual machines is created, and resources are distributed according to performance and energy requirements. Virtual machine resource requests are converted into a matching relationship factor, which represents the individual hosts while taking PPR into account. The overall workload associated with virtual machine consolidation is also provided by these estimations. It is noted that there is little energy trade-off and that performance is maintained at a nominal level across the cluster. The architecture is put into practice throughout offline platforms, which are dispersed ecosystems that allow for increased system performance and scaling. Results:: The CloudSim simulator is used to validate the system using datasets that are obtained from PlanetLab. According to the data, energy saving has produced yields of up to 47% and promising quality of service attributes. Conclusion:: The validation of the system is performed using the CloudSim simulator with datasets from PlanetLab. The results indicate significant energy conservation, up to 47%, along with promising quality of service parameters. The proposed architecture is compared with other state-of-the-art algorithms for distributed architectures and heterogeneous environments, showcasing its efficiency. The conclusion emphasizes the prioritization of VM consolidation and energy efficiency in the proposed architecture, which has been tested on a Proliant G7- based data center using a variety of hosts. Notably, the CloudSim Toolkit is highlighted as outperforming OpenStack-based techniques in simulation results.

On the scalability of the speedup considering the overhead of consolidating virtual machines in servers for data centers

AbstractVirtualization technologies are extensively utilized in data centers, particularly cloud computing. This facilitates data center management and diminishes the number of physical machines (servers) and, subsequently, their cooling requirements, leading to cost, space, and power consumption reductions. When applications in data centers are executing independent parallel transactions, but with similar performance requirements, the appropriate level of virtual machine consolidation on a server poses a fundamental challenge for capacity planning. This article introduces a method to evaluate the performance speedup achieved through virtualization on any server and the effects of virtualization and consolidation overheads on physical or virtual machine scalability. This research formalizes the speedup and overheads, using classical computer architecture statements. but at the same time proposes a new method to analyze these overhead amounts and types, showing the scalability and efficiency of different consolidations in the same server and its comparison against no consolidation. This work also proposes a new way to determine the optimal number of physical servers and the optimal number of consolidated virtual machines for a given transaction workload. The real experimentation was performed with different workload sizes, types of virtualizations and different servers. The method presented also facilitates the representation of linear scalability against the real degree of parallelism of either physical machines or consolidated virtual machines for a given transaction workload, as well as striking the right balance between speedup and energy in virtual server consolidation.

A Review Study on Energy Consumption in Cloud Computing

Cloud computing has become a fundamental technology for a wide range of computing services, yet its increasing energy demands present substantial environmental and economic challenges. With the rapid growth of Cloud services and applications, increasing number of researches have been focused on energy saving. The need to reduce energy costs is a constant challenge of cloud providers and data centers. This paper offers an extensive review of the issues surrounding energy consumption in cloud computing, with a focus on algorithms associated with the situational awareness, consolidation, allocation, placement/migration, and scheduling of virtual machines and containers. We conduct a critical analysis of studies from 2018 to 2023, comparing various methodologies aimed at achieving energy efficiency without sacrificing performance. This review delineates current trends, identifies gaps in existing research, and proposes directions for future investigations. Our study emphasizes the necessity of cultivating sustainable practices in cloud computing and provides valuable insights into the practical implementation of energy-efficient solutions in cloud environments.

A cut-and-solve algorithm for virtual machine consolidation problem

The virtual machine consolidation problem attempts to determine which servers to be activated, how to allocate virtual machines to the activated servers, and how to migrate virtual machines among servers such that the summation of activated, allocation, and migration costs is minimized subject to the resource constraints of the servers and other practical constraints. In this paper, we first propose a new mixed integer linear programming formulation for the virtual machine consolidation problem. We show that compared with existing formulations, the proposed formulation is much more compact in terms of smaller numbers of variables or constraints, which makes it suitable for solving large-scale problems. We then develop a cut-and-solve algorithm, a tree search algorithm to efficiently solve the virtual machine consolidation problem to optimality. The proposed cut-and-solve algorithm is based on a novel relaxation of the virtual machine consolidation problem that provides a stronger lower bound than the natural continuous relaxation of the virtual machine consolidation problem, making a smaller search tree. By extensive computational experiments, we show that (i) the proposed formulation significantly outperforms existing formulations in terms of solution efficiency; (ii) compared with standard mixed integer linear programming solvers, the proposed cut-and-solve algorithm is much more efficient; and (iii) compared with an existing heuristic algorithm, the proposed cut-and-solve algorithm is better able to balance workload distribution across servers and achieve higher resource utilization.

Dynamic Optimization of Energy Efficiency and Service Level Agreement violation in Cloud Data Centers via Virtual Machine Consolidation

With the escalating proliferation of cloud computing services, cloud data center providers are actively in pursuit of solutions not only to curtail energy consumption but also to efficiently cater to the unique demands of their users, thereby enhancing user satisfaction. The process of consolidating virtual machines emerges as a viable strategy for striking a balance between energy consumption and Service Level Agreement violation (SLAv), potentially surmounting this predicament faced by cloud service providers. This study aims to minimize energy consumption and SLAv to their utmost feasible extent by utilizing the First Median Threshold (FMT) technique for physical host selection and virtual machine allocation, in conjunction with the Maximum Ratio of CPU utilization to memory Utilization (MRCU) method for virtual machine selection within the framework of virtual machine consolidation. This dual-pronged approach not only translates into cost savings for cloud providers but also ensures the delivery of optimal services to users. Furthermore, a novel method has been introduced for selecting virtual machines for migration and appropriately choosing physical hosts to achieve equilibrium between two critical parameters: energy consumption and SLAv. The proposed methodology is implemented and evaluated using the CloudSim simulator, utilizing real-world PlanetLab data. The comprehensive analysis of the simulation results reveals that the proposed approach can significantly reduce energy consumption while effectively mitigating SLAv for users.

Towards virtual machine scheduling research based on multi-decision AHP method in the cloud computing platform

Virtual machine scheduling and resource allocation mechanism in the process of dynamic virtual machine consolidation is a promising access to alleviate the cloud data centers of prominent energy consumption and service level agreement violations with improvement in quality of service (QoS). In this article, we propose an efficient algorithm (AESVMP) based on the Analytic Hierarchy Process (AHP) for the virtual machine scheduling in accordance with the measure. Firstly, we take into consideration three key criteria including the host of power consumption, available resource and resource allocation balance ratio, in which the ratio can be calculated by the balance value between overall three-dimensional resource (CPU, RAM, BW) flat surface and resource allocation flat surface (when new migrated virtual machine (VM) consumed the targeted host’s resource). Then, virtual machine placement decision is determined by the application of multi-criteria decision making techniques AHP embedded with the above-mentioned three criteria. Extensive experimental results based on the CloudSim emulator using 10 PlanetLab workloads demonstrate that the proposed approach can reduce the cloud data center of number of migration, service level agreement violation (SLAV), aggregate indicators of energy comsumption (ESV) by an average of 51.76%, 67.4%, 67.6% compared with the cutting-edge method LBVMP, which validates the effectiveness.

Energy and Performance-Efficient Dynamic Consolidate VMs Using Deep-Q Neural Network

With cloud computing facing higher levels of big data than ever, the processor scale is rapidly expanding. Large clusters place a heavy burden on cloud service providers and the environment. High energy consumption decreases the economic benefits of cloud service providers, while enormous power demands pressure on the environment. The dynamic consolidation of virtual machines (VMs), which uses live migration technology to optimize resource usage and reduce energy consumption, is sufficient for saving energy while ensuring high performance with the desired level of quality of service (QoS) between cloud providers and users. In this paper, we propose a novel machine-learning algorithm called deep-Q neural network VM consolidation (DQNVMC) that combines the Q-leaning approach with deep learning neural network to find an approximately optimal solution. Furthermore, based on the real workload trace in the cloud environment, the experiments show that DQNVMC effectively reduces energy consumption while meeting the high performance of QoS requirements.

Design of an energy efficient dynamic virtual machine consolidation model for smart cities in urban areas

The growing smart cities in urban areas are becoming more intelligent day by day. Massive storage and high computational resources are required to provide smart services in urban areas. It can be provided through intelligence cloud computing. The establishment of large-scale cloud data centres is rapidly increasing to provide utility-based services in urban areas. Enormous energy consumption of data centres has a destructive effect on the environment. Due to the enormous energy consumption of data centres, a massive amount of greenhouse gases (GHG) are emitted into the environment. Virtual Machine (VM) consolidation can enable energy efficiency to reduce energy consumption of cloud data centres. The reduce energy consumption can increase the Service Level Agreement (SLA) violation. Therefore, in this research, an energy-efficient dynamic VM consolidation model has been proposed to reduce the energy consumption of cloud data centres and curb SLA violations. Novel algorithms have been proposed to accomplish the VM consolidation. A new status of any host called an almost overload host has been introduce, and determined by a novel algorithm based on the Naive Bayes Classifier Machine Learning (ML) model. A new algorithm based on the exponential binary search is proposed to perform the VM selection. Finally, a new Modified Power-Aware Best Fit Decreasing (MPABFD) VM allocation policy is proposed to allocate all VMs. The proposed model has been compared with certain well-known baseline algorithms. The comparison exhibits that the proposed model improves the energy consumption by 25% and SLA violation by 87%.

Fuzzy correlational analysis for dynamic consolidation of virtual machines in cloud computing environment

This theoretical research in fuzzy correlation analysis integrates data uncertainty analysis by measuring the strength of the linear relationship restricted to two fuzzy sets. As the relevant contribution to this research area, this paper presents the axiomatic definition of the n-dimensional generalized fuzzy correlation coefficient (nGCC), assigning to n input fuzzy sets an output value in the interval [−1,1]. Thus, the properties of general overlap functions and fuzzy negations are studied, discussing binary non-normed restricted dissimilarity functions, and the n-dimensional non-normed conjunctive functions. This study provides new methods for the n-dimensional generalized fuzzy correlation coefficient analysis, regarding their applications in solving multi-criteria and decision-making problems founded on fuzzy logic extensions. The n-dimensional non-normed conjunctive aggregation functions are also introduced, as range domain extensions from [0,1] to [−1.1], covering the interpretation of negative to positive variable associations. The proposal correlation analysis promotes a better evaluation and data selection even when more than one algorithm is applied to evaluate the reduction methods in the defuzzification process based on Interval-valued Fuzzy Logic. We also investigate the relevance to determine a reliable result from the fuzzy inference system based on the nGCC methodology. This correlation methodology applied to the Interval Fuzzy Load Balancing for Cloud Computing (Int-FLBCC) model contributes as a flexible approach for virtual machines dynamic consolidation enabling improvements in resource usage and power efficiency, improving the computational system’s energy efficiency. So, the nGCC methodology extends the Int-FLBCC model by adding other degrees of reliability to the results obtained with diverse evaluations through n-dimensional generalized fuzzy correlation coefficient expressions, exploring average aggregations as arithmetic and exponential means and the median operator.

An efficient proactive VM consolidation technique with improved LSTM network in a cloud environment

An efficient proactive VM consolidation technique with improved LSTM network in a cloud environment

Modeling and Simulation of Real-Time Virtual Machine Allocation in a Cloud Data Center

For dynamic resource scheduling in cloud data centers, a novel lightweight simulation system is proposed; two existing simulation systems at the application level for cloud computing are reviewed; and results gained using the suggested simulation system are examined and discussed. The usage of resources and energy efficiency in cloud data centers can be improved by load balancing and the consolidation of virtual machines. An aspect of dynamic virtual machine consolidation that directly affects resource usage and the quality of service the system is delivering is the timing of when it is ideal to reallocate Virtual Machines from an overloaded host [1]. Because server overloads result in a lack of resources and a decline in application performance, they have an impact on quality of service. In order to determine the best answer, existing approaches to the problem of host overload detection typically rely on statistical analysis inspired by nature. These strategies' drawbacks include the fact that they provide less-than-ideal outcomes and prevent the explicit articulation of a Quality-of-Service target. By optimizing the mean inter-migration time under the defined Quality of Service target ideally, we present a novel method for detecting host overload for any stationary workload that is known and a particular state configuration [2]. We demonstrate that our technique exceeds the best benchmark algorithm and offers over 88%of the performance of the ideal offline algorithm through simulations with real-world workload traces from more than a thousand Virtual Machines.

Host utilization prediction using Taylor Kernel Convolutional Neural Network (TKCNN) and workflow scheduling for smart irrigation cloud data centers

One of the most significant economic sectors in these nations is agriculture, which emphasises the significance of controlling the water resources at hand to maintain the survival of this industry. Sensor and cloud computing (CC) technology are widely used in many real-time applications, particularly in agriculture. To ensure Quality of Service in the CC environment, Virtual Machine (VM) Migration as part of VM Consolidation is very important (QoS). Predicting host utilization utilizing utilization history and lowering energy consumption while satisfying Service Level Agreements (SLAs) is a challenging task as a result of the large fluctuation in the consumption of cloud resources and the dynamic workloads.CC technologies are widely used in many real-time applications, particularly in agriculture for smart irrigation cloud data centers difficult task. To solve this issue proposed work introduced a Taylor Kernel Convolutional Neural Network (TKCNN) algorithm, which trains the past history for host usage prediction in smart irrigation cloud data centers. The Optimum Energy and Resource Aware Workflow Scheduling- Host Utilization Prediction (OERES-HUP) method, which attempts to schedule the task workflow to VMs engaging in computing, was influenced by the well-known Fuzzy Multi-Verse Optimizer (FMVO) algorithm. Additionally, it focuses on dynamically deploying and underplaying VMs in accordance with job requirements. The effectiveness of scheduling algorithms is evaluated using key metrics including Resource Utilization (RU), Energy Consumption or Task (ECT), Total Energy Consumption (TEC), Makespan, and Execution Time per Task (ETT).The outcomes are a good representation of how effective the suggested algorithm is compared to current techniques. To test the efficacy of the suggested model, the cloud environment is simulated using the Cloud Sim simulator.

A resource scheduling method for cloud data centers based on thermal management

With the rapid growth of cloud computing services, the high energy consumption of cloud data centers has become a critical concern of the cloud computing society. While virtual machine (VM) consolidation is often used to reduce energy consumption, excessive VM consolidation may lead to local hot spots and increase the risk of equipment failure. One possible solution to this problem is to utilize thermal-aware scheduling, but existing approaches have trouble realizing the balance between SLA and energy consumption. This paper proposes a novel method to manage cloud data center resources based on thermal management (TM-VMC), which optimizes total energy consumption and proactively prevents hot spots from a global perspective. Its VM consolidation process includes four phases where the VMs scheduler uses an improved ant colony algorithm (UACO) to find appropriate target hosts for VMs based on server temperature and utilization status obtained in real-time. Experimental results show that the TM-VMC approach can proactively avoid data center hot spots and significantly reduce energy consumption while maintaining low Service Level Agreement (SLA) violation rates compared to existing mainstream VM consolidation algorithms with workloads from real-world data centers.

An energy efficient fog level resource management scheme for software defined cities

The proliferation of Internet of Things (IoT) and smart city services have paved the path for complex network architectures including edge and fog layer resources in order to meet such services’ Quality of Service (QoS). Such services should also be energy efficient for sustainable cities. Though energy efficiency has been widely studied over the past decade, yet most such solutions have focused on device or sensor level solutions or alternatively on cloud/datacenter level solutions. With the prominence of Software Defined Networks (SDNs) in recent times owing to its flexible and scalable network management features, SDN based fog level resource management has become an attractive alternative. This paper addresses fog level energy-aware resource management with the goal of minimizing energy consumption while QoS fulfillment by means of Virtual Machine (VM) consolidation through live migrations. Since any decent sized smart city would have millions of devices and large numbers of edge/fog instances, it would lead any solution mechanism to a NP hard problem. This paper proposes a novel set based Differential Evolutionary (S-DE) algorithm. Simulations have been carried out to test the efficacy of the proposed S-DE solution and the results demonstrate the efficiency of the proposed method.

A Kernel search algorithm for virtual machine consolidation problem in cloud computing

Virtual machine consolidation refers to the process of reallocating virtual machines across a set of target servers. It can be formulated as a mixed integer linear programming problem, which is known to be NP-hard. In this paper, we propose a kernel search (KS) heuristic algorithm based on hard variable fixing to quickly obtain high-quality solutions for large-scale virtual machine consolidation problems (VMCPs). Since existing variable fixing strategies in KS algorithms may result in some VMCP instances being infeasible, our proposed KS algorithm employs a more efficient strategy to choose a set of fixed variables based on their corresponding reduced costs. Our numerical results on VMCP instances demonstrate that our proposed KS algorithm significantly outperforms mixed integer linear programming solvers in terms of CPU time. Moreover, our proposed strategy of variable fixing improves the efficiency of the KS algorithm significantly, with negligible degradation in solution quality.

Proactive and dynamic load balancing model for workload spike detection in cloud

Dynamic Load Balancing (DLB) has been an ever-green research topic since the beginning of cloud computing. DLB model distributes the available cloud resources (i.e., CPU, RAM, Storage, BW, etc.) among the VMs using a systematic approach to ensure efficient resource utilization. Overloading, an abnormal condition, occurs when the host encounters a resource shortage due to the high workload of Federated Learning. VM migration and consolidation are reliable load-balancing techniques to fix dynamic overloading issues. As part of the migration, the oversubscribed host VMs are moved to the other hosts till the load is regulated. Besides the high workload, the sudden spike in workload also leads the hosts to instant overloading. The former DLB models are efficient in handling general overloading conditions. Still, they are now suffering from limitations in taking the spike overloading due to 1) a sudden spike in workloads, 2) instant overloading, 3) recurrent migrations, and 4) SLA violations. To address these limitations in handling spike overloading, in this paper, we proposed a proactive and dynamic load balancing model (DLBM), which is designed to handle spike overloading with optimal solutions. The offered dynamic load balancing model with a spike detection algorithm helps detect workload spikes in advance to prevent the hosts from overloading and SLA violations. Experimental conducted using the Cloudsim tool and the planet lab workload dataset prove that the proposed DLBM identified the spikes in workload and efficiently controlled the recurrent migrations.

BalCon — resource balancing algorithm for VM consolidation

Cloud providers handle substantial number of requests to create and delete virtual machines (VMs) on a daily basis, where the unknown sequence of requests eventually leads to resource fragmentation. To mitigate this issue, periodic consolidation of VMs into fewer number of physical hosts is an important cost-saving procedure, closely related to the vector bin-packing problem. In this paper, we propose the BalCon algorithm for consolidation that steadily reduces the number of active hosts and keeps migration costs low. BalCon classifies the cluster’s state and selects one of three heuristics to balance resources for superior consolidation. To evaluate BalCon’s performance with respect to optimality, we introduce integer programming models. BalCon finds 99.7% of the optimal solutions for over 750 problem instances. This outstanding result was achieved due to the Force Step of our algorithm, which is the key improvement detail for common heuristics. We compare BalCon with a modified Sercon heuristic using Huawei and synthetic datasets with two resources for allocation.