IaaS Cloud Research Articles

A Virtual Machine Instance Anomaly Detection System for IaaS Cloud Computing

Infrastructure as a Service (IaaS) is one of the three important fundamental service models provided by cloud computing. It provides users with computing resource and storage resource in terms of virtual machine instances. Because of the rapid development of cloud computing, more and more application systems have been deployed on the IaaS cloud computing platforms. Therefore, once anomalies incur in the IaaS cloud computing platforms, all the application systems cannot work normally. In order to enhance the dependability of IaaS cloud computing platform, a virtual machine instance anomaly detection system is proposed for IaaS cloud computing platform to detect virtual machine instances that exhibit abnormal behaviors. The proposed virtual machine instance system consists of four modules that are the data collection, the data transmission, the data storage, and the anomaly detection. In order to reduce the computing complexity and improve the detection precision, the anomaly detection module introduces the principal components analysis to reprocess the collected data and then adopts the Bayesian decision theory to detect the abnormal data. Experimental results show that the proposed virtual machine instance anomaly detection system is effective.

Shoot4U

Virtual Machine based approaches to workload consolidation, as seen in IaaS cloud as well as datacenter platforms, have long had to contend with performance degradation caused by synchronization primitives inside the guest environments. These primitives can be affected by virtual CPU preemptions by the host scheduler that can introduce delays that are orders of magnitude longer than those primitives were designed for. While a significant amount of work has focused on the behavior of spinlock primitives as a source of these performance issues, spinlocks do not represent the entirety of synchronization mechanisms that are susceptible to scheduling issues when running in a virtualized environment. In this paper we address the virtualized performance issues introduced by TLB shootdown operations. Our profiling study, based on the PARSEC benchmark suite, has shown that up to 64% of a VM's CPU time can be spent on TLB shootdown operations under certain workloads. In order to address this problem, we present a paravirtual TLB shootdown scheme named Shoot4U. Shoot4U completely eliminates TLB shootdown preemptions by invalidating guest TLB entries from the VMM and allowing guest TLB shootdown operations to complete without waiting for remote virtual CPUs to be scheduled. Our performance evaluation using the PARSEC benchmark suite demonstrates that Shoot4U can reduce benchmark runtime by up to 85% compared an unmodified Linux kernel, and up to 44% over a state-of-the-art paravirtual TLB shootdown scheme.

Cloud Crawler: a declarative performance evaluation environment for infrastructure‐as‐a‐service clouds

SummaryAs the number of infrastructure‐as‐a‐service (IaaS) cloud offers in the market increases, selecting an appropriate configuration of cloud resources for a given application becomes a non‐trivial task even for experienced developers. Because cloud resources are relatively cheap, usually charged by the hour, developers could systematically evaluate the performance of their application using different resource types from different cloud providers, thus allowing them to accurately identify the best providers and resource types for their application. However, conducting systematic performance tests in multiple IaaS clouds may require a significant amount of planning and configuration effort from application developers. This paper presents Cloud Crawler, a declarative environment for specifying and conducting application performance tests in IaaS clouds. The environment includes a novel declarative domain‐specific language, Crawl, by means of which cloud users can describe, at a high abstraction level, a large variety of performance evaluation scenarios for a given application, and a scenario execution engine, Crawler, which automatically configures, executes, and collects the results of the scenarios described in Crawl. The paper also reports on how Cloud Crawler has been successfully used to systematically test the performance of two open‐source web applications in public IaaS clouds. Copyright © 2016 John Wiley & Sons, Ltd.

Building the IBM Containers cloud service

Linux® containers have been employed for operating system-level virtualization since around 2008, but only recently they have started to gain tremendous popularity, fueled by open source projects such as Warden and Docker. Initially, container technologies have been used mainly for stateless web applications in platform-as-a-service (PaaS) clouds, but as the technology matures, they are being used for a larger variety of cloud workloads that are typically run on infrastructure-as-a-service (IaaS) clouds. For this reason, we developed the IBM Containers service, a cloud service that allows multiple developers to compose, deploy, and manage the lifecycle of complex application topologies with security and isolation on a shared infrastructure. The service provides developers with features of a PaaS cloud and flexibility of an IaaS cloud. In this paper, we discuss the challenges, architecture, security considerations, and capabilities of the service.

Standards for enabling heterogeneous IaaS cloud federations

Technology market is continuing a rapid growth phase where different resource providers and Cloud Management Frameworks are positioning to provide ad-hoc solutions–as management interfaces, information discovery or billing–trying to differentiate from competitors resulting in incompatibilites between them when addressing more complex scenarios like federated clouds.Therefore, grasping interoperability problems present in current infrastructures by studying how existing and emerging standards could enhance the cloud user experience.In this paper we will review the current open challenges in Infrastructure as a Service cloud interoperability and federation, as well as point to the potential standards that should alleviate these problems.

An Auction Mechanism for Cloud Spot Markets

Dynamic forms of resource pricing have recently been introduced by cloud providers that offer Infrastructure as a Service (IaaS) capabilities in order to maximize profits and balance resource supply and demand. The design of a mechanism that efficiently prices perishable cloud resources in line with a provider’s profit maximization goal remains an open research challenge, however. In this article, we propose the Online Extended Consensus Revenue Estimate mechanism in the setting of a recurrent, multiunit and single price auction for IaaS cloud resources. The mechanism is envy-free, has a high probability of being truthful, and generates a near optimal profit for the provider. We combine the proposed auction design with a scheme for dynamically calculating reserve prices based on data center Power Usage Effectiveness (PUE) and electricity costs. Our simulation-based evaluation of the mechanism demonstrates its effectiveness under a broad variety of market conditions. In particular, we show how it improves on the classical uniform price auction, and we investigate the value of prior knowledge on the execution time of virtual machines for maximizing profit. We also developed a system prototype and conducted a small-scale experimental study with a group of 10 users that confirms the truthfulness property of the mechanism in a real test environment.

Online mechanism with latest-reservation for dynamic VMs allocation in private cloud

Cloud computing provided as a utility has emerged in recent years. Resource allocation mechanisms play a critical role toward the success of cloud computing. Maximization of social welfare is the reasonable objective of private clouds. Cloud resources are expiring goods, and users come to cloud randomly. However, because of the dynamic behavior of cloud providers and users, existing allocation mechanisms in public clouds with static environments are unsuitable for private IaaS clouds in dynamic environments, given that they can neither maximize the sum of the users’ value, nor provide service guarantees. In order to mitigate this problem, we propose a novel online cloud resources allocation mechanism to capture the features of private clouds. In this paper, we propose a latest-reservation allocation mechanism to maximize social welfare. We also prove that our mechanism is truthful. We simulate the proposed online mechanism using data from RICC, thus showing that our mechanism leads to high social welfare and high percentage of served users compared with the MOVMPA mechanism.

Experience in Practical Implementation of Abstraction Interface for Integrated Cloud Resource Management on Multi-Clouds

Infrastructure-as-a-Service (IaaS) clouds provide infrastructure as a pool of virtual resources, and the public IaaS clouds, e.g. Amazon Web Service (AWS) and private IaaS cloud toolkits, e.g. OpenStack, CloudStack, etc. provide their own application programming interfaces (APIs) for managing the cloud resources they offer. The heterogeneity of the APIs, however, makes it difficult to access and use the multiple cloud services concurrently and collectively. In this paper, we explore previous efforts to solve this problem and present our own implementation of an integrated cloud API, which can make it possible to access and use multiple clouds collectively in a uniform way. The implemented API provides a RESTful access and hides underlying cloud infrastructures from users or applications. We show the implementation details of the integrated API and performance evaluation of it comparing the proprietary APIs based on our cloud testbed. From the evaluation results, we could conclude that the overhead imposed by our interface is negligibly small and can be successfully used for multi-cloud access.

Cost-aware DAG scheduling algorithms for minimizing execution cost on cloud resources

Directed acyclic graph (DAG) scheduling is a well-known problem, because a DAG can be used to describe a wide range of complex applications, including scientific applications and parallel computing jobs. Most DAG scheduling algorithms were proposed to minimize the job makespan (i.e., execution time) on a multiprocessor computer or cluster. However, as the cost-driven public cloud services have become an attractive and popular platform for providing computing resources, cost minimization emerges as a new critical issue. Therefore, the objective of this work is to propose and solve the cost optimization problem for scheduling DAGs on an IaaS cloud platform where task scheduling must cope with resource provisioning to achieve the optimal solution. In this paper, we proposed both optimal and heuristic scheduling algorithms, and we evaluated them across a variety of DAGs using the price model from EC2. Comparing to other cost-oblivious DAG schedules that aim to minimize makespan or resource usage, the results show that our cost-aware heuristic algorithm can reduce cost by 20---50 % and achieve a cost within x1.16 of the optimal one.

An Appraisal of Meta-Heuristic Resource Allocation Techniques for IaaS Cloud

Background/Objectives: This appraisal investigates the meta-heuristics resource allocation techniques for maximizing financial gains and minimizing the financial expenses of cloud users for IaaS in cloud computing environment. Methods/Statistical Analysis: Overall, a total of ninety-one studies from 1954 to 2015 have been reviewed in this paper. However, twenty-three studies are selected that focused on the meta-heuristic algorithms for their research. The selected papers are categorized into eight groups according to the optimization algorithms used. Findings: From the analytical study, we pointed out the various issues addressed (optimal and dynamically resource allocation, energy and QoS aware resource allocation, VM allocation and placement) through resource allocation meta-heuristics algorithms.Whereas, the improvement shows better performance concerns minimizing the execution and response time, energy consumption and cost while enhancing the efficiency and QoS in this environment. The comparison parameters (makespan 35%,execution time 13%, response time 26%, cost 22%, utilization21% and other 13% including energy, throughput etc) and also the experimental tools (CloudSim 43%, GridSim 5%, Simjava 9%, Matlab 9% and others 13%) used for evaluation of the various techniques for resource allocation in IaaS cloud computing. Applications/Improvements: The comprehensive review and systematic comparison of meta-heuristic resource allocation algorithms described in this appraisal will help researchers to analyze different techniques for future research directions.

Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds

Recently, we have witnessed workflows from science and other data-intensive applications emerging on Infrastructure-asa-Service (IaaS) clouds, and many workflow service providers offering workflow as a service (WaaS). The major concern of WaaS providers is to minimize the monetary cost of executing workflows in the IaaS cloud. While there have been previous studies on this concern, most of them assume static task execution time and static pricing scheme, and have the QoS notion of satisfying a deterministic deadline. However, cloud environment is dynamic, with performance dynamics caused by the interference from concurrent executions and price dynamics like spot prices offered by Amazon EC2. Therefore, we argue that WaaS providers should have the notion of offering probabilistic performance guarantees for individual workflows on IaaS clouds. We develop a probabilistic scheduling framework called Dyna to minimize the monetary cost while offering probabilistic deadline guarantees. The framework includes an A*-based instance configuration method for performance dynamics, and a hybrid instance configuration refinement for utilizing spot instances. Experimental results with three real-world scientific workflow applications on Amazon EC2 demonstrate (1) the accuracy of our framework on satisfying the probabilistic deadline guarantees required by the users; (2) the effectiveness of our framework on reducing monetary cost in comparison with the existing approaches.

Execution of scientific workflows on IaaS cloud by PBRR algorithm

Job scheduling of scientific workflow applications in IaaS cloud is a challenging task. Optimal resource mapping of jobs to virtual machines is calculated considering schedule constraints such as timeline and cost. Determining the required number of virtual machines to execute the jobs is key in finding the optimal schedule makespan with minimal cost. In this paper, VMPROV algorithm has been proposed to find the required virtual machines. Priority-based round robin (PBRR) algorithm is proposed for finding the job to resource mapping with minimal makespan and cost. Executions of four real-world scientific application jobs by PBRR algorithm are compared with MINMIN, MAXMIN, MCT, and round robin algorithms execution times. The results show that the proposed algorithm PBRR can predict the mapping of tasks to virtual machines in better way compared to the other classic algorithms.

Service provisioning of flexible advance reservation leases in IaaS clouds

Resource provisioning is a very important and challenging task in cloud computing. The techniques employed for resource provisioning needs to avoid under-provisioning and over-provisioning of resources. It also needs to ensure the cloud providers that the required SLAs are met. In this paper, an efficient resource provisioning and scheduling algorithm is proposed to schedule service requests in IaaS clouds. Service requests in the cloud are of four types: immediate, best-effort, advance reservation and deadline sensitive. In the proposed work, advance reservation requests with flexible start times (flexible advance reservation requests) are considered. The experiments are conducted in a cloud environment developed using OpenNebula. The proposed algorithm is compared with the existing algorithms of Haizea and SAB. The experimental results indicate that the proposed algorithm gives better request acceptance rate and reduced resource fragmentation than the existing ones.

Cloud Computing-System Implementation for Business Applications

Nowadays, companies are faced with the task of processing huge quantum of data. As the traditional database systems cannot handle this task in a cost-efficient manner, companies have built customized data processing frameworks. Cloud computing has emerged as a promising approach to rent a large IT infrastructure on a short-term pay-per-usage basis. This paper attempts to schedule tasks on compute nodes so that data sent from one node to the other has to traverse as few network switches as possible. The challenges and opportunities for efficient parallel data processing in cloud environments have been demonstrated and Nephele, the first data processing framework, has been presented to exploit the dynamic resource provisioning offered by the IaaS clouds. The overall utilisation of resources has been improved by assigning specific virtual machine types to specific tasks of a processing job and by automatically allocating or deallocating virtual machines in the course of a job execution. This has led to substantial reduction in the cost of parallel data processing.

Cloud Performance Modeling with Benchmark Evaluation of Elastic Scaling Strategies

In this paper, we present generic cloud performance models for evaluating Iaas, PaaS, SaaS, and mashup or hybrid clouds. We test clouds with real-life benchmark programs and propose some new performance metrics. Our benchmark experiments are conducted mainly on IaaS cloud platforms over scale-out and scale-up workloads. Cloud benchmarking results are analyzed with the efficiency, elasticity, QoS, productivity, and scalability of cloud performance. Five cloud benchmarks were tested on Amazon IaaS EC2 cloud: namely YCSB, CloudSuite, HiBench, BenchClouds, and TPC-W. To satisfy production services, the choice of scale-up or scale-out solutions should be made primarily by the workload patterns and resources utilization rates required. Scaling-out machine instances have much lower overhead than those experienced in scale-up experiments. However, scaling up is found more cost-effective in sustaining heavier workload. The cloud productivity is greatly attributed to system elasticity, efficiency, QoS and scalability. We find that auto-scaling is easy to implement but tends to over provision the resources. Lower resource utilization rate may result from auto-scaling, compared with using scale-out or scale-up strategies. We also demonstrate that the proposed cloud performance models are applicable to evaluate PaaS, SaaS and hybrid clouds as well.

CEAT: a cloud evidence acquisition tool for aiding forensic investigations in cloud systems

Owing to the rise in cyber crime in cloud computing systems, cloud forensics has gained a lot of interest in the research community. Cloud forensic investigations differ from traditional forensic investigations due to the variability in architecture, lack of standardisation, distributed nature of cloud artefacts, scattered evidences and so on. The existing digital forensic tools cannot address these issues directly. The major challenge in cloud forensics is the identification and acquisition of artefacts. We propose a tool for forensic acquisition of artefacts on cloud systems. Our tool considers various artefacts such as cloud infrastructure logs, snapshots and volumes in addition to virtual disk and memory images. In this paper, we discuss the functional and non-functional requirements considered by our cloud forensic acquisition tool. We have implemented our solution on Openstack private IaaS cloud test bed.

CEAT: a cloud evidence acquisition tool for aiding forensic investigations in cloud systems

Owing to the rise in cyber crime in cloud computing systems, cloud forensics has gained a lot of interest in the research community. Cloud forensic investigations differ from traditional forensic investigations due to the variability in architecture, lack of standardisation, distributed nature of cloud artefacts, scattered evidences and so on. The existing digital forensic tools cannot address these issues directly. The major challenge in cloud forensics is the identification and acquisition of artefacts. We propose a tool for forensic acquisition of artefacts on cloud systems. Our tool considers various artefacts such as cloud infrastructure logs, snapshots and volumes in addition to virtual disk and memory images. In this paper, we discuss the functional and non-functional requirements considered by our cloud forensic acquisition tool. We have implemented our solution on Openstack private IaaS cloud test bed.

Power and Fault Aware Reliable Resource Allocation for Cloud Infrastructure

Cloud computing is now trending and more popular in these days for the computation and adopted by many companies like google, amazon, Microsoft etc. As the cloud size increases with increase in number of data center power consumption over a data center increases. As number of request over the data center increase with increase in load and power consumption of the data center. So the requests need to be balanced in such manner which having more effective strategy for resources utilization, request failure and improved power consumption. Cloud computing made it more complicated with respective to requests type that may increase or decrease power consumption. A recent survey on cloud computation shows that the power consumption of a server, increasing in a linear way due to utilization of resource (processors) resulting in request failure at datacenters. Request balancing in such manner without having knowledge of load over server maximize resource utilization but also increasing power consumption at server. So to overcome these issues in cloud Infrastructure as a service (IaaS), we have proposing a fault and power aware scheduling algorithm to minimize the power consumption, request failure and cost over a data center. Proposed algorithm has proven to have better performance in term of load and power efficiency as compared to previously proposed load balancing algorithm for cloud IaaS.

Power & Load Aware Resource Allocation Policy for Hybrid Cloud

Cloud computing is now trending and more popular in these days for the computation and adopted by many companies like google, amazon, Microsoft etc., As the cloud size increases with increase in number of data center power consumption over a data center increases. As number of request over the data center increase with increase in load and power consumption of the data center. So the requests need to be balanced in such manner which having more effective strategy for utilization of resources, balance load over data center and reduction in power consumption. Hybrid cloud computing made it more complicated with respective to requests type that may increase or decrease power consumption. A recent survey on cloud computation shows that the power consumption of a server, increasing in a linear way due to utilization of resource (processors). Request balancing in such manner without having knowledge of load over data center maximize resource utilization but also increasing power consumption at server. So to overcome these issues in cloud Infrastructure as a service (IaaS), we have proposing a load balancing algorithm to minimize the power consumption and cost over a data center. Proposed algorithm has proven to have better performance in term of load and power efficiency as compared to previously proposed load balancing algorithm for cloud IaaS.

Toward Anomaly Detection in IaaS Cloud Computing Platforms

Toward Anomaly Detection in IaaS Cloud Computing Platforms