Large Number Of Virtual Machines Research Articles

Clustering based EO with MRF technique for effective load balancing in cloud computing

PurposeCloud computing (CC) refers to the usage of virtualization technology to share computing resources through the internet. Task scheduling (TS) is used to assign computational resources to requests that have a high volume of pending processing. CC relies on load balancing to ensure that resources like servers and virtual machines (VMs) running on real servers share the same amount of load. VMs are an important part of virtualization, where physical servers are transformed into VM and act as physical servers during the process. It is possible that a user’s request or data transmission in a cloud data centre may be the reason for the VM to be under or overloaded with data.Design/methodology/approachVMs are an important part of virtualization, where physical servers are transformed into VM and act as physical servers during the process. It is possible that a user’s request or data transmission in a cloud data centre may be the reason for the VM to be under or overloaded with data. With a large number of VM or jobs, this method has a long makespan and is very difficult. A new idea to cloud loads without decreasing implementation time or resource consumption is therefore encouraged. Equilibrium optimization is used to cluster the VM into underloaded and overloaded VMs initially in this research. Underloading VMs is used to improve load balance and resource utilization in the second stage. The hybrid algorithm of BAT and the artificial bee colony (ABC) helps with TS using a multi-objective-based system. The VM manager performs VM migration decisions to provide load balance among physical machines (PMs). When a PM is overburdened and another PM is underburdened, the decision to migrate VMs is made based on the appropriate conditions. Balanced load and reduced energy usage in PMs are achieved in the former case. Manta ray foraging (MRF) is used to migrate VMs, and its decisions are based on a variety of factors.FindingsThe proposed approach provides the best possible scheduling for both VMs and PMs. To complete the task, improved whale optimization algorithm for Cloud TS has 42 s of completion time, enhanced multi-verse optimizer has 48 s, hybrid electro search with a genetic algorithm has 50 s, adaptive benefit factor-based symbiotic organisms search has 38 s and, finally, the proposed model has 30 s, which shows better performance of the proposed model.Originality/valueUser’s request or data transmission in a cloud data centre may cause the VMs to be under or overloaded with data. To identify the load on VM, initially EQ algorithm is used for clustering process. To figure out how well the proposed method works when the system is very busy by implementing hybrid algorithm called BAT–ABC. After the TS process, VM migration is occurred at the final stage, where optimal VM is identified by using MRF algorithm. The experimental analysis is carried out by using various metrics such as execution time, transmission time, makespan for various iterations, resource utilization and load fairness. With its system load, the metric gives load fairness. How load fairness is worked out depends on how long each task takes to do. It has been added that a cloud system may be able to achieve more load fairness if tasks take less time to finish.

Energy and Reliability-Aware Task Scheduling for Cost Optimization of DVFS-Enabled Cloud Workflows

Due to the increasing complexity, the execution of workflow applications on cloud typically involves a large number of virtual machines (VMs), which makes the cost as well as energy consumption a great concern. To alleviate this issue, more and more cloud service providers introduce new pricing policies considering Dynamic Voltage and Frequency Scaling (DVFS), where users are charged on the basis of allocated CPU frequencies together with various combinations of VM configurations and prices. However, the customizable CPU frequencies make resource provisioning and scheduling harder to achieve a cost-optimal solution. The things become even worse, since lowering CPU voltages of VMs will increase their chance of suffering soft errors, which results in a high rate of completion time failures of workflow applications. To address the above problem, this paper proposes a novel task scheduling method for the purpose of cost optimization based on the genetic algorithm. By introducing new genetic operators and frequency scaling scheme for DVFS-enabled cloud workflows, our approach can quickly figure out cost-optimal resource provisioning and task scheduling solutions by allocating tasks to appropriate VMs with specific operating frequencies under energy, reliability, makespan and memory constraints. Extensive experiments on various well-known scientific workflow benchmarks validate the effectiveness of the proposed method. Comparing with state-of-the-art methods, our approach can significantly reduce the overall cost and energy consumption without violating the given constraints.

Performance Interference of Virtual Machines: A Survey

The rapid development of cloud computing with virtualization technology has benefited both academia and industry. For any cloud data center at scale, one of the primary challenges is how to effectively orchestrate a large number of virtual machines (VMs) in a performance-aware and cost-effective manner. A key problem here is that the performance interference between VMs can significantly undermine the efficiency of cloud data centers, leading to performance degradation and additional operation cost. To address this issue, extensive studies have been conducted to investigate the problem from different aspects. In this survey, we make a comprehensive investigation into the causes of VM interference and provide an in-depth review of existing research and solutions in the literature. We first categorize existing studies on interference models according to their modeling objectives, metrics used, and modeling methods. Then we revisit interference-aware strategies for scheduling optimization as well as co-optimization-based approaches. Finally, the survey identifies open challenges with respect to VM interference in data centers and discusses possible research directions to provide insights for future research in the area.

Genetic-Based Virtual Machines Consolidation Strategy With Efficient Energy Consumption in Cloud Environment

In cloud computing environments, virtualization is used to share physical machine (PM) resources among multiple users by creating virtual machines (VMs). Running the PM consumes a large amount of energy. Additionally, the PM will be overloaded when the demand for resources exceeds the PM capacity. This overload on the PM leads to violations of Service Level Agreements (SLAs). Dynamic VM consolidation techniques use live migration of VMs to optimize resource utilization and minimize energy consumption. However, excessive migration of VMs impacts negatively the application performance due to the incurred overhead at the runtime. This paper presents a modified genetic-based VM consolidation (MGVMC) strategy that aims to replace VMs in an online manner taking into account energy consumption, SLA violations, and the number of VM migrations. The MGVMC strategy utilizes the genetic algorithm to migrate VMs to the appropriate PM in a way that minimizes the number of over-utilized and under-utilized physical machines (PMs) as low as possible. The performance of the MGVMC strategy was evaluated using the CloudSim Plus framework with a large number of VMs and workload traces from the PlanetLab platform. The experimental results revealed that the MGVMC strategy achieved a significant improvement in energy consumption, SLA violations, and the number of VM migrations compared to other recent approaches. These results demonstrate the effectiveness of the MGVMC strategy in optimizing VM consolidation in the cloud environment.

Mapping of Virtual Machines Using Machine Learning Algorithms for Detection of Faulty Nodes

A distributed system is characterized by a large number of nodes that are linked to a network and are mostly used for transaction processing. Large set of users are likely to communicate information over the network to the nodes, consistency and dependability remain a critical problem in the distributed environments. Independent failure of the component is one of the major problems in the distributed systems as it slowly impacts the performance of the other nodes in the system. The quality of service - QoS of a distributed network may be improved by a quick way of detecting problematic nodes. Sometime heavy nodes required high computation for transaction processing while idle nodes take low computation. In this paper, we proposed identification of straggler nodes in distributed environment with the help of hybrid machine learning algorithm. The work basically carried out to set up of large number of virtual machines and collect current log audits of each VM. According to the available parameters of audit files to each machine, algorithms decide that specific node is overheated or ideal condition. In expensive experimental analysis we demonstrate a accuracy of proposed hybrid machine learning algorithm. The proposed algorithm produces higher precision up to 4.5% than state-of-art methods. Key highlights of the VM mapping strategy were also investigated through a scrutiny of ongoing contracts. Main focus remains on machine learning (ML) to distinguish PM (Physical Machine) congestion, determining VMs from crowded PMs, and VM conditions as major exercises. This paper aims to review and characterize research on the planning and status of VMs that use ML using asset usage history. Energy productivity, VM migration, and quality of service were the main exhibition boundaries used to investigate cloud data center presentations.

An Effective Cloud Workflow Scheduling Approach Combining PSO and Idle Time Slot-Aware Rules

Workflow scheduling is a key issue and remains a challenging problem in cloud computing. Faced with the large number of virtual machine (VM) types offered by cloud providers, cloud users need to choose the most appropriate VM type for each task. Multiple task scheduling sequences exist in a workflow application. Different task scheduling sequences have a significant impact on the scheduling performance. It is not easy to determine the most appropriate set of VM types for tasks and the best task scheduling sequence. Besides, the idle time slots on VM instances should be used fully to increase resources' utilization and save the execution cost of a workflow. This paper considers these three aspects simultaneously and proposes a cloud workflow scheduling approach which combines particle swarm optimization (PSO) and idle time slot-aware rules, to minimize the execution cost of a workflow application under a deadline constraint. A new particle encoding is devised to represent the VM type required by each task and the scheduling sequence of tasks. An idle time slot-aware decoding procedure is proposed to decode a particle into a scheduling solution. To handle tasks' invalid priorities caused by the randomness of PSO, a repair method is used to repair those priorities to produce valid task scheduling sequences. The proposed approach is compared with state-of-the-art cloud workflow scheduling algorithms. Experiments show that the proposed approach outperforms the comparative algorithms in terms of both of the execution cost and the success rate in meeting the deadline.

Research on security situation assessment algorithm under virtual technology of cloud platform

With the increasing attacks and destructions against cloud platforms, the security guarantee mechanism of cloud platforms has also changed from traditional passive protectionto active defense. Situation assessment is a method that can actively analyze and evaluate thecurrent security risk status of the cloud platform, and is a key part of the whole process of situation awareness. This paper aims at a large number of virtual machines deployed in cloud platforms, on the basis of analysis and extraction of virtual layer security situation assessment elements, an improved adaptive genetic simulated annealing algorithm OAGSAA is proposed, and applied to BP neural network, which can effectively analyze and evaluate the security status of the virtual layer. Simulation experiment results show that the method has higher prediction accuracy and convergence speed, and can avoid falling into the local minimum.

Design of Energy Aware Scheduling Algorithm for Executing Scientific Workflows in Cloud

The usage of cloud computing and its resources for the execution of scientific workflow is a rapidly increasing demand. The Scientific applications are generally large in scale; even a single scientific workflow includes more number of complex tasks. Execution of these tasks can be made successful only by deploying it in the cloud virtual machines, because only cloud environment can only provide very large number of computing assets. In cloud, every processing resource is given as Virtual Machine. Any scientific workflow deployed in the cloud needs large number of virtual machines so; huge amount of computational energy is spent by the virtual machines to execute multifaceted scientific workflows. Hence there arises the need to utilize the cloud resources in an energy efficient way. Also, if the virtual machines are planned to schedule in an energy efficient manner there is an increase of makepsan of the workflow which is going to be an important parameter for completing the workflow within the deadline. So, the need for executing scientific workflows in energy efficient way with reduced makespan becomes a major issue among the researchers. It also becomes very challenging task to executing a scientific workflow in within the given deadline of a task in the given workflow. To address these issues, a new Energy Aware workflow scheduling algorithm is proposed and designed with improved makespan for the execution of different scientific applications in cloud environment.

A comparative study of multi-objective machine reassignment algorithms for data centres

At a high level, data centres are large IT facilities hosting physical machines (servers) that often run a large number of virtual machines (VMs)—but at a lower level, data centres are an intricate collection of interconnected and virtualised computers, connected services, complex service-level agreements. While data centre managers know that reassigning VMs to the servers that would best serve them and also minimise some cost for the company can potentially save a lot of money—the search space is large and constrained, and the decision complicated as they involve different dimensions. This paper consists of a comparative study of heuristics and exact algorithms for the multi-objective machine reassignment problem. Given the common intuition that the problem is too complicated for exact resolutions, all previous works have focused on various (meta)heuristics such as First-Fit, GRASP, NSGA-II or PLS. In this paper, we show that the state-of-art solution to the single objective formulation of the problem (CBLNS) and the classical multi-objective solutions fail to bridge the gap between the number, quality and variety of solutions. Hybrid metaheuristics, on the other hand, have proven to be more effective and efficient to address the problem—but as there has never been any study of an exact resolution, it was difficult to qualify their results. In this paper, we present the most relevant techniques used to address the problem, and we compare them to an exact resolution ($$\epsilon $$-Constraints). We show that the problem is indeed large and constrained (we ran our algorithm for 30 days on a powerful node of a supercomputer and did not get the final solution for most instances of our problem) but that a metaheuristic (GeNePi) obtains acceptable results: more (+ 188%) solutions than the exact resolution and a little more than half (52%) the hypervolume (measure of quality of the solution set).

Constraint-aware VM placement in heterogeneous computing clusters

Virtualized systems consist of a large number of machines that are configured with different hardware and software, and execute a large number of virtual machines (VMs) for diverse applications. There can be various constraint conditions of placing VMs in such systems due to the concerns on security, availability, performance, etc. However, VM placement constraints can limit the choice of hosts for VMs, affecting the performance of the systems negatively. In this paper, we study constraint-aware VM placement in heterogeneous computing clusters. We first present a model of VM placement constraints that supports all types of constraints between VMs, and between VMs and hosts. Second, we discuss six constraint-aware VM placement algorithms which optimize the performance for either energy saving or load balancing. Third, using simulations, we analyze the effects of different types of VM placement constraints on VM placement, and evaluate the performance of the algorithms over various settings. We also run experiments of the algorithms in a small cluster. Our extensive simulation results demonstrate that the effects of VM placement constraints vary, depending on the optimization goal, the types of the constraints, and the system configurations. For the constraint-aware algorithms, we show that an energy saving algorithm which attempts to place a new VM on one of active hosts by utilizing VM migrations, and a load balancing algorithm which attempts to migrate some VMs from a selected host for a new VM, i.e. a potential hotspot, to another host provide good performance.

DYVINE: Fitness-Based Dynamic Virtual Network Embedding in Cloud Computing

Virtual network embedding (VNE) is the process of embedding the set of interconnected virtual machines onto the set of interconnected physical servers (PSs) in the cloud computing environment. The level of complexity of VNE problem increases when a large number of virtual machines with a set of resource demand need to be embedded onto a network of thousands of PSs. The key challenge of VNE is the efficient mapping of virtual networks (VNs), which may have dynamic resource demands. Existing solutions mainly emphasize on the embedding of static VN resulting in poor resource utilization and very low acceptance rate. To tackle such level of complexity in VNE, a fitness-based dynamic virtual network embedding (DYVINE) algorithm is proposed with the goal to maximize the resource utilization by maximizing the acceptance rate. Local and global fitness values of the virtual machines and VN, respectively, are used to utilize the maximum amount of physical resources. The proposed VNE algorithm allows the VN to be dynamic, which indicates that the structure and resource demand can be changed during its execution time. Furthermore, in order to reduce the embedding time in each time slot, a set of PSs is selected to host the VN instead of considering thousands of PSs, which may significantly increase the embedding time. The proposed embedding mechanism is evaluated through extensive simulation and is compared with similar existing embedding algorithms, which outperforms over others.

Making Sensor Node Virtual Machines Work for Real-World Applications

A large number of virtual machines (VMs) have been developed for sensor nodes. Many are based on Java, although implementations based on Python and.Net also exist. Some aspects of the java virtual machine, such as the simple yet sufficiently expressive instruction set, work well on a sensor node. However, Java was developed for more powerful devices, which results in certain functionality not being ideal for resource constrained devices. While writing VM implementations and programmes to run on sensor node devices over the past decade, we encountered a number of situations where this mismatch between standard languages and the specifics of sensor node programming limited the practical use of current VMs. In this letter, we describe what we consider to be the most pressing issues and discuss how these may be resolved in future VMs.

A Provident Resource Defragmentation Framework for Mobile Cloud Computing

To facilitate mobile cloud computing, a cloud service provider must dynamically create and terminate a large number of virtual machines (VMs), causing fragmented resources that cannot be further utilized. To solve this problem proactively, most of the existing studies have been based on server consolidation, with the main objective of minimizing the number of active servers. Although this approach can minimize resource fragmentation at a particular time, it may be over aggressive at the price of too frequent VM migration and low system stability. To address this issue, we propose a novel provident resource defragmentation framework that is revenue-oriented with the goal to reduce unnecessary VM migration. Within the proposed framework, we formulate an optimization problem for resource defragmentation at a particular time epoch, with the consideration of the future impact of any VM migration. We then develop an efficient heuristic algorithm that can obtain near-optimal results. Extensive numerical results confirm that our framework can provide the highest profit and can significantly reduce the VM migration cost in practical scenarios.

Leveraging content similarity among VMI files to allocate virtual machines in cloud

To meet a myriad of customers’ demands, a large number of virtual machines (VMs) have to be provisioned simultaneously in cloud data centers. Provisioning is usually time consuming due to the large size of virtual machine image (VMI) file that needs to be transferred via networks. To address this issue, researchers attempt to leverage the content similarity among different VMI files to reduce the volume of transferred data. In the VM provisioning, the VM packing problem that minimizes the number of physical machines is another important issue. In this paper, our goal is to find a solution that tries to pack VMs to the minimum number of PMs as well as significantly reduces the total amount of transferred data. We formally define the problem of VM packing and minimizing the data transferring in the VM provisioning, named RTVD-VA. We first propose an approximation algorithm to minimize the amount of transferred VMI data when provisioning K VMs with the same size to a single physical machine. We then extend the algorithm to address the scenario of multiple PMs when using the minimum number of PMs. Based on the above two approximation algorithms, we propose a heuristic algorithm, namely Balance-Placement, to solve the problem in general cases. Our simulation results show that Balance-Placement outperforms existing solutions like PSO and Greedy-Cache and achieves the least amount of transferred data and the minimum number of used PMs in most scenarios.

Price forecasting for spot instances in Cloud computing

Big data applications usually need to rent a large number of virtual machines from Cloud computing providers. As a result of the policies employed by Cloud providers, the prices of spot virtual machine instances behavior stochastically. Spot prices (prices of spot instances) fluctuate greatly or have multiple regimes. Choosing virtual machines according to trends in prices is helpful in decreasing the resource rental cost. Existing price prediction methods are unable to accurately predict prices in these environments. As a result, a dynamic-ARIMA and two markov regime-switching autoregressive model based forecasting methods have been developed in this paper. Experimental results show that the proposals are better than the existing MonthAR in most scenarios.

Reservation schemes for IaaS cloud broker: a time‐multiplexing way for different rental time

SummaryInfrastructure‐as‐a‐Service cloud providers always charge users in 2 ways: on‐demand billing and reservation billing with different discounts. On‐demand billing is more flexible for short‐term use but with higher cost, while reservation billing is much cheaper for long renting period but may cause inefficiency because of underutilized capacity. As middle agent, cloud broker always tries to reserve a large number of virtual machine (VM) instances from cloud providers to meet the requirement of all users, while making high profit through reducing its cost by leveraging long‐term reservations and time‐multiplexing of instances. Different from existing work, we believe that the rental time of each user's request should be served consecutively in one VM instance, rather than partitioned into different VM instances by cloud broker to enhance time‐multiplexing ratio. In this paper, we first try to smooth users' unordered and time‐overlapping requests using demand graph, based on which we propose two reservation algorithms for cloud broker, one for off‐line and the other for online. We also consider cloud broker with multiple service providers, finding that much more cost can be saved by leveraging the various reservation discounts of those providers. Extensive simulations have been done on the traces from real cloud platforms. Experiment results show that the broker cost can be substantially saved using our off‐line and online algorithms.

Handling Boot Storms in Virtualized Data Centers—A Survey

Large-scale virtual machine (VM) deployment in virtualized data centers is a very slow process. This is primarily due to the resource bottlenecks that are created at the storage, network, and host physical machines when a large number of VMs are requested simultaneously. For companies that provide virtual desktops to their employees, it is common to encounter such requests each day, when their employees turn up for work. In addition, a large number of VMs are often required to be deployed instantly, in order to absorb a spike in the workload, at online e-commerce websites. In such scenarios, long deployment times are unacceptable, and reducing them is of paramount importance. In this article, we first abstract out the key techniques suggested in the literature to speed up this deployment process. We follow this with a classification of these techniques into a taxonomy and propose a framework that can be used to compare them. Finally, we identify problem areas that warrant further research and bring out the shortcomings of the current state-of-the-art solutions.

VirtMan: design and implementation of a fast booting system for homogeneous virtual machines in iVCE

Internet-based virtual computing environment (iVCE) has been proposed to combine data centers and other kinds of computing resources on the Internet to provide efficient and economical services. Virtual machines (VMs) have been widely used in iVCE to isolate different users/jobs and ensure trustworthiness, but traditionally VMs require a long period of time for booting, which cannot meet the requirement of iVCE’s large-scale and highly dynamic applications. To address this problem, in this paper we design and implement VirtMan, a fast booting system for a large number of virtual machines in iVCE. VirtMan uses the Linux Small Computer System Interface (SCSI) target to remotely mount to the source image in a scalable hierarchy, and leverages the homogeneity of a set of VMs to transfer only necessary image data at runtime. We have implemented VirtMan both as a standalone system and for OpenStack. In our 100-server testbed, VirtMan boots up 1000 VMs (with a 15 GB image of Windows Server 2008) on 100 physical servers in less than 120 s, which is three orders of magnitude lower than current public clouds.

Online anomaly detection for multi‐source VMware using a distributed streaming framework

SummaryAnomaly detection refers to the identification of patterns in a dataset that do not conform to expected patterns. Such non‐conformant patterns typically correspond to samples of interest and are assigned to different labels in different domains, such as outliers, anomalies, exceptions, and malware. A daunting challenge is to detect anomalies in rapid voluminous streams of data.This paper presents a novel, generic real‐time distributed anomaly detection framework for multi‐source stream data. As a case study, we investigate anomaly detection for a multi‐source VMware‐based cloud data center, which maintains a large number of virtual machines (VMs). This framework continuously monitors VMware performance stream data related to CPU statistics (e.g., load and usage). It collects data simultaneously from all of the VMs connected to the network and notifies the resource manager to reschedule its CPU resources dynamically when it identifies any abnormal behavior from its collected data. A semi‐supervised clustering technique is used to build a model from benign training data only. During testing, if a data instance deviates significantly from the model, then it is flagged as an anomaly.Effective anomaly detection in this case demands a distributed framework with high throughput and low latency. Distributed streaming frameworks like Apache Storm, Apache Spark, S4, and others are designed for a lower data processing time and a higher throughput than standard centralized frameworks. We have experimentally compared the average processing latency of a tuple during clustering and prediction in both Spark and Storm and demonstrated that Spark processes a tuple much quicker than storm on average. Copyright © 2016 John Wiley & Sons, Ltd.

가상 스위치 구현 및 비교에 관한 연구

ABSTRACT Nowadays, most Internet servers run on virtual a network which connects each virtual machine, to process large amounts of data on single physical computer. The virtual machines for their own purposes are interconnected to other virtual server systems with various types of operating systems. Each virtual server may share one physical computer resources and operates at a service seamlessly to each other. Sometimes even the virtual machines are implemented using a method that lease the physical IT resources without purchasing physical server. Therefore, a virtual switch is needed to improve the performance when a large number of virtual machines operate on physical server. In this study, the implementation and investigation was performed for the virtual switches, OpenvSwitch and DPDKvSwitch, in order to provide the necessary high-performance cloud services to creative business. 키워드 : 가상머신, 가상네트워크, 고속화 가상스위치, 클라우드 서비스 Key word : Virtual machine, virtual network, accelerated virtual switch, cloud serviceJournal of the Korea Institute of Information andCommunication Engineering