Cloud Bursting Research Articles

Redundancy Decision Method Based on Reliability Requirements for Cloud Bursting

Redundancy Decision Method Based on Reliability Requirements for Cloud Bursting

A DRL-based online VM scheduler for cost optimization in cloud brokers.

The virtual machine (VM) scheduling problem in cloud brokers that support cloud bursting is fraught with uncertainty due to the on-demand nature of Infrastructure as a Service (IaaS) VMs. Until a VM request is received, the scheduler does not know in advance when it will arrive or what configurations it demands. Even when a VM request is received, the scheduler does not know when the VM's lifecycle expires. Existing studies begin to use deep reinforcement learning (DRL) to solve such scheduling problems. However, they do not address how to guarantee the QoS of user requests. In this paper, we investigate a cost optimization problem for online VM scheduling in cloud brokers for cloud bursting to minimize the cost spent on public clouds while satisfying specified QoS restrictions. We propose DeepBS, a DRL-based online VM scheduler in a cloud broker which learns from experience to adaptively improve scheduling strategies in environments with non-smooth and uncertain user requests. We evaluate the performance of DeepBS under two request arrival patterns which are respectively based on Google and Alibaba cluster traces, and the experiments show that DeepBS has a significant advantage over other benchmark algorithms in terms of cost optimization.

Flood Risk Analysis of Upper Krishna Sub-Basin using GIS

Abstract: Floods are the most and repeatedly occurring destructive natural disaster due to an overflow of water submerges land which is basically dry. Generally, floods are occurring due to heavy rainfall or cloud bursting or manmade disturbance to nature, fast snowmelt, global warming or tropical cyclone or tsunami. In 2019, the Krishna River has faced very heavy rainfall and major floods which took the lives of approximately 500 people and nearly isolating 350 villages and leaving millions homeless. Here we studied the previous flood disaster at the basin of the Krishna River and processed the GIS environment using software tools. This facilitates exploring the data and methods that are mostly unexplored, and areas that have not lightened in the field of flood studies in Krishna basin. It is impossible to avoid floods and risk associated with flood, however it is possible to work on the flood reduction. Flood hazard mapping is to identify comparatively safe sites in high elevation with low risk is one of the powerful tools for this purpose. Flood hazard mapping flash flood will be beneficial for risk assessment. Management and emergency services during flood events. The objective of this paper is to generate flood hazard zonation maps of Upper Krishna sub-basin using GIS tools and satellite images. To do so, we use spatial data and SRTM DEMs with accuracy assessment is achieved by using check points, obtained by GPS observations. Runoff, surface slope, drainage density, distance to main channel and land use were considered causative factors. All used data are processed and integrated in an ArcMap and QGIS to prepare a final flood hazard map for Upper Krishna sub-basin. The areas in high risk flood zones are obtained by overlaying the flood hazard index map with the zone boundaries layer

Improving the hybrid cloud performance through disk activity-aware data access

Cloud computing has been making significant contributions to Internet of Things, Big Data, and many other cutting-edge research areas in recent years. To deal with the cloud bursting, on-premises private clouds often extend their service capacity with off-premises public clouds, which causes the migration of jobs and their corresponding data from private clouds to public clouds. For jobs executed in public clouds, promptly transferring data they need from private clouds to public clouds is essential for their quick completion as the volume of data is often large for cloud applications. The Internet connection between private clouds and public clouds is with limited bandwidth in most cases. Therefore, it will be valuable if the underlying operating system could expedite the course of reading data from hard drives to speed up the process of moving data from private clouds to public clouds. The Apache Hadoop is considered as one of the most widely used cloud platforms in the community of cloud computing. It keeps multiple replicas of data across its cluster nodes to improve data availability. We designed and implemented a new model enabling computing nodes in Hadoop to get data in request from a cluster node with the least amount of disk activity regardless of its location to hasten the course of accessing data. Experimental results show that jobs could reduce their execution time by up to 80.83% in our model. Accordingly, our model could help accelerate the completion of job execution in both private clouds and public clouds in the environment of hybrid clouds.

An Osmosis-Based Intelligent Agent Scheduling Framework for Cloud Bursting in a Hybrid Cloud

An intelligent system to efficiently provision resources in a hybrid cloud environment is necessary due to the high level of complexity. The semi-permeable agent for hybrid cloud scheduling (SPAH) is a bio-inspired agent that adapts the biological process of osmosis into cloud bursting. The primary objective of the agent is to minimize the makespan. The framework and algorithm for the two phases of SPAH, to recognize the state and decide on action are presented. A QoS (Quality of Service) deadline factor metric is proposed to study the indirect impact of SPAH in deadline satisfaction. SPAH shows significant improvement in deadline satisfaction of up to 85% as compared to other cloud bursting techniques. This is the result of a reduced makespan and a reduced cumulative waiting time. The analysis of SPAH shows that it works in quadratic time complexity.

An automated implementation of hybrid cloud for performance evaluation of distributed databases

A Hybrid cloud is an integration of resources between private and public clouds. It enables users to horizontally scale their on-premises infrastructure up to public clouds in order to improve performance and cut up-front investment cost. This model of applications deployment is called cloud bursting that allows data-intensive applications especially distributed database systems to have the benefit of both private and public clouds. In this work, we present an automated implementation of a hybrid cloud using (i) a robust and zero-cost Linux-based VPN to make a secure connection between private and public clouds, and (ii) Terraform as a software tool to deploy infrastructure resources based on the requirements of hybrid cloud. We also explore performance evaluation of cloud bursting for six modern and distributed database systems on the hybrid cloud spanning over local OpenStack and Microsoft Azure. Our results reveal that MongoDB and MySQL Cluster work efficient in terms of throughput and operations latency if they burst into a public cloud to supply their resources. In contrast, the performance of Cassandra, Riak, Redis, and Couchdb reduces if they significantly leverage their required resources via cloud bursting.

AN ARTIFICIAL NEURAL NETWORK BASED DEVELOPMENT OF CLOUD BURST FORECASTING MODEL BY USING TIME-SERIES DATA.

Over the last decades, weather forecasting models using the numerical data of various previous years have been improving steadily to provide a more predictable and accurate model. However, the atmosphere conditions are a highly chaotic system and always vary with time and weather forecasts are a major benefit for society and sustainable development. So, in this research we focus to develop a model using the optimized Artificial Neural Network (ANN) for prediction of cloud bursting in India and developed model in known as Cloud Burst Foresting (CBF) model for forecasting of rainfall or cloud burst based on the previous record of the bursting in different state. Here the concept of Particle Swarm Optimization (PSO) is used as an optimization technique in pre-processing step to separate the previous year rainfalls data into two categories such minimum and maximum rainfalls. Basically, PSO separate the cloud burst recorded data using a novel fitness function that help to train the CBF model accuracy and if training is better, then the prediction accuracy will be high. By utilizing the concept of optimized ANN, the prediction accuracy is high in terms of percentage of correct predication and with minimum percentage of incorrect prediction. At last, the performance of the model is calculated to validate the proposed CBF model and this shows that it is possible to use ANN as a machine learning technique in order to estimate future cloud burst forecast uncertainty from past forecasts data of bursting. The main constraint in the performance of our proposed CBF model seems to be the number of past forecasts available for training the ANN.

A Two-Stage Heuristic Hybrid Evolutionary Algorithm for Virtual Machine Migration and Placement in Cloud Bursting

The platform for cloud computing offers virtualization and a dynamic pool of resources to the consumers of the cloud. The acceptance and demand of cloud is growing on a regular basis. Cloud computing offers utility-based consumer services across the globe on a pay as you go strategy. Live Virtual Machine (VM) migration sets the basic foundation for cloud management. It hasakey role in reducing operating expenditure and revising quality of service without disruption of cloud services running in the VM. A lot of research has been done to yield better performance in live VM migration and has seen noteworthy development and accomplishment. However, some crucial problems require results and enhancement. With the growth of new cloud computing models like Mobile Edge Computing, certain problems related to optimization need to be addressed. The primary aim of this research work is to emphasize on optimum functioning of live migration. A migration algorithm to consolidate the computational resources, storage resources and network resources dynamically with a two-stage heuristic hybrid evolutionary algorithm is discussed. The resources are consolidated to cut down the energy and cost utilization depending upon the evolutionary Particle Swarm Optimization and the Ant Colony Optimization algorithms. These algorithms can rapidly identify the migrating virtual machines and locate their positions respectively.

Enhancing the Grid with Cloud Computing

Scientific computing has evolved considerably in recent years. Scientific applications have become more complex and require an increasing number of computing resources to perform on a large scale. Grid computing has become widely used and is the chosen infrastructure for many scientific calculations and projects, even though it demands a steep learning curve. The computing and storage resources in the Grid are limited, heterogeneous and often overloaded. This heterogeneity is not only present in the hardware setups, but also in the software composition, where configuration permissions are limited. It also has a negative effect on the portability of scientific applications. The use of Cloud resources could eliminate those constraints. In the Cloud, resources are provisioned on demand and can be scaled up and down, while scientists can easily customize their execution environments in the form of virtual machines. Extending the Grid with Cloud resources would improve the utilization of shared resources and would enable the use of additional resources when the Grid resources are overloaded – known as Cloud bursting. We propose an integration model of the Grid and the Cloud using the HTCondor batch system and the NorduGrid ARC middleware. This model enables batch job execution in any public or private Cloud by deploying a virtualized Grid cluster using the ARC middleware - PaaS model for running Grid applications. An evaluation of the virtual Grid cluster was made and compared with the physical one by running NAMD simulations.

Towards operational cost minimization for cloud bursting with deadline constraints in hybrid clouds

In hybrid clouds, there is a technique named cloud bursting which can allow companies to expand their capacity to meet the demands of peak workloads in a low-priced manner. In this work, a cost-aware job scheduling approach based on queueing theory in hybrid clouds is proposed. The job scheduling problem in the private cloud is modeled as a queueing model. A genetic algorithm is applied to achieve optimal queues for jobs to improve the utilization rate of the private cloud. Then, the task execution time is predicted by back propagation neural network. The max–min strategy is applied to schedule tasks according to the prediction results in hybrid clouds. Experiments show that our cost-aware job scheduling algorithm can reduce the average job waiting time and average job response time in the private cloud. In additional, our proposed job scheduling algorithm can improve the system throughput of the private cloud. It also can reduce the average task waiting time, average task response time and total costs in hybrid clouds.

Performance Model of MapReduce Iterative Applications for Hybrid Cloud Bursting

Hybrid cloud bursting (i.e., leasing temporary off-premise cloud resources to boost the overall capacity during peak utilization) can be a cost-effective way to deal with the increasing complexity of big data analytics, especially for iterative applications. However, the low throughput, high latency network link between the on-premise and off-premise resources (“weak link”) makes maintaining scalability difficult. While several data locality techniques have been designed for big data bursting on hybrid clouds, their effectiveness is difficult to estimate in advance. Yet such estimations are critical, because they help users decide whether the extra pay-as-you-go cost incurred by using the off-premise resources justifies the runtime speed-up. To this end, the current paper presents a performance model and methodology to estimate the runtime of iterative MapReduce applications in a hybrid cloud-bursting scenario. The paper focuses on the overhead incurred by the weak link at fine granularity, for both the map and the reduce phases. This approach enables high estimation accuracy, as demonstrated by extensive experiments at scale using a mix of real-world iterative MapReduce applications from standard big data benchmarking suites that cover a broad spectrum of data patterns. Not only are the produced estimations accurate in absolute terms compared with experimental results, but they are also up to an order of magnitude more accurate than applying state-of-art estimation approaches originally designed for single-site MapReduce deployments.

Multi-queue scheduling of heterogeneous jobs in hybrid geo-distributed cloud environment

In hybrid geo-distributed clouds, there is a technique named cloud bursting in which applications are handled in the private cloud with less expenses and burst into public clouds when the resources of the private cloud run out. However, how to deploy heterogeneous jobs in heterogeneous hybrid cloud environment is still a challenge. In this paper, a multi-queue scheduling approach of heterogeneous jobs for cloud bursting is proposed. In the private cloud, jobs are classified into I/O-intensive and CPU-intensive jobs, and nodes are divided into main I/O and CPU resource pools. Jobs are dispatched to corresponding resource pools to reduce the job execution time in heterogeneous cloud environment. A genetic algorithm is applied to schedule jobs to optimal job queues, which can reduce the job waiting time. Then, the execution time of each task is predicted by BP neural network. Jobs with high priority will be allocated to resources with the earliest finish time in the private cloud according to the prediction results. If the private cloud cannot meet the demand of users, public clouds with minimal costs will be applied. Experiments show that our proposed algorithm can reduce the average job response time and improve the throughput of the private cloud. It also can reduce the average task waiting time, average task execution time and average task response time significantly. Moreover, the costs of the hybrid clouds are reduced.

Elastic extension of a local analysis facility on external clouds for the LHC experiments

The computing infrastructures serving the LHC experiments have been designed to cope at most with the average amount of data recorded. The usage peaks, as already observed in Run-I, may however originate large backlogs, thus delaying the completion of the data reconstruction and ultimately the data availability for physics analysis. In order to cope with the production peaks, the LHC experiments are exploring the opportunity to access Cloud resources provided by external partners or commercial providers. In this work we present the proof of concept of the elastic extension of a local analysis facility, specifically the Bologna Tier-3 Grid site, for the LHC experiments hosted at the site, on an external OpenStack infrastructure. We focus on the Cloud Bursting of the Grid site using DynFarm, a newly designed tool that allows the dynamic registration of new worker nodes to LSF. In this approach, the dynamically added worker nodes instantiated on an OpenStack infrastructure are transparently accessed by the LHC Grid tools and at the same time they serve as an extension of the farm for the local usage.

Fuzzy logic-based call admission control in 5G cloud radio access networks with preemption

Fifth generation (5G) cellular networks will be comprised of millions of connected devices like wearable devices, Androids, iPhones, tablets, and the Internet of Things (IoT) with a plethora of applications generating requests to the network. The 5G cellular networks need to cope with such sky-rocketing traffic requests from these devices to avoid network congestion. As such, cloud radio access networks (C-RAN) has been considered as a paradigm shift for 5G in which requests from mobile devices are processed in the cloud with shared baseband processing. Despite call admission control (CAC) being one of radio resource management techniques to avoid the network congestion, it has recently been overlooked by the community. The CAC technique in 5G C-RAN has a direct impact on the quality of service (QoS) for individual connections and overall system efficiency. In this paper, a novel fuzzy logic-based CAC scheme with preemption in C-RAN is proposed. In this scheme, cloud bursting technique is proposed to be used during congestion, where some delay tolerant low-priority connections are preempted and outsourced to a public cloud with a penalty charge. Simulation results show that the proposed scheme has low blocking probability below 5%, high throughput, low energy consumption, and up to 95% of return on revenue.

Problem Identification and Climate Change Perception of Bakerwal Pastoralists of Jammu & Kashmir

The present study was conducted in JK North-Western hilly state of India during 2013 to 2015 to study the problems experienced by transhumant Bakerwal pastoralists in different phases of migration and their climate change perception. The sample of 120 respondents was selected randomly for problem identification while as 39 respondents were purposively selected for examining their climate change perception. The study revealed that lack of basic facilities (viz. shelter for animals and humans, clean drinking water, medical facilities, roads connectivity, marketing, education for children, electricity and means of communication) was the top ranking problem experienced by the respondents in summer pastures and en-route migration. This was followed by lack of veterinary health services and disease occurrence which were ranked second most important problems respectively. Reduced pastures availability and water scarcity for animals during later part of the winter were the top most problems experienced by the respondents at winter pastures. Further it was reported that respondents perceived a drastic change in predictability of weather conditions and incidences of torrential rains, cloud bursting, hail storms etc. in the study area. The respondents also expressed a moderate change in average seasonal temperature, amount of rainfall, amount of snowfall and severity of draught at pastures. The study recommends the immediate policy intervention on part of government for provision of various basic facilities to the Bakerwals in different phases of migration besides promotion of income diversification within pastoralism through skill development and need based training programmes for securing the livelihood of Bakerwals.

A Reference Model for Virtual Machine Launching Overhead

Cloud bursting is one of the key research topics in the cloud computing communities. A well designed cloud bursting module enables private clouds to automatically launch virtual machines (VMs) to public clouds when more resources are needed. One of the main challenges in developing a cloud bursting module is to decide when and where to launch a VM so that all resources are most effectively and efficiently utilized and the system performance is optimized. However, based on system operational data obtained from FermiCloud, a private cloud developed by the Fermi National Accelerator Laboratory for scientific workflows, the VM launching overhead is not a constant. It varies with physical resource utilization, such as CPU and I/O device utilizations, at the time when a VM is launched. Hence, to make judicious decisions as to when and where a VM should be launched, a VM launching overhead reference model is needed. In this paper, we first develop a VM launching overhead reference model based on operational data we have obtained on FermiCloud. Second, we apply the developed reference model on FermiCloud and compare calculated VM launching overhead values based on the model with measured overhead values on FermiCloud. Our empirical results on FermiCloud indicate that the developed reference model is accurate. We believe, with the guidance of the developed reference model, efficient resource allocation algorithms can be developed for cloud bursting process to minimize the operational cost and resource waste.

Cloud Bursting, Brokerage and Aggregation with Load Balancing

Cloud Bursting, Brokerage and Aggregation with Load Balancing

Is Cloud Bursting/Hybrid Cloud Advantageous to a Firm Who Has On-Site Capacity?

Cloud bursting is a hybrid computing resource model in which a firm firstly uses its own internal computing resources and then bursts to a public cloud for extra resources as needed. This paper develops quantitative models to examine whether a firm who has on-site capacity can be benefited from cloud bursting. If a firm adopts a hybrid cloud, while it benefits from immediate, extra computing resources, and scalability, but it assumes the risks of security threats and service outages from this external resource. If a firm uses only a private cloud, while security threats and service outage risks may be far lower than with a hybrid cloud, it may lose the opportunity to profit due at peak times due to limited resource capacity. Overall, we show that firms can benefit from the hybrid clouds if risk is considerably low while firms would benefit from using the private clouds if risk is considerably high. Interestingly, firms may display counterintuitive cloud deployments when facing risk of outage that one firm deploys a hybrid cloud and another deploys a private cloud when risk of outage is considerably low and both firms prefer the hybrid clouds when risk of outage is considerably high.

Pareto-Optimal Cloud Bursting

Large-scale Bag-of-Tasks (BoT) applications are characterized by their massively parallel, yet independent operations. The use of resources in public clouds to dynamically expand the capacity of a private computer system might be an appealing alternative to cope with such massive parallelism. To fully realize the benefit of this `cloud bursting', the performance to cost ratio (or cost efficiency) must be thoroughly studied and incorporated into scheduling and resource allocation strategies. In this paper, we present PANDA, a framework for static scheduling BoT applications across resources in both private and public clouds. The framework at the core incorporates a fully polynomial-time approximation scheme (FPTAS) as a novel scheduling algorithm, which generates schedules with the best trade-off point between cost and performance; hence Pareto-optimality. We have theoretically discussed the complexity and correctness of our algorithms, and experimentally verified their efficacy and practicality using ISOMAP-a widely-used nonlinear manifold method as a real-world BoT application. Our evaluation conducted in a 'multi-cloud' environment of our 40-core private system and Amazon EC2 public cloud demonstrates the scheduling quality of PANDA is guaranteed to be within a measurable distance from the optimal solution. Results obtained from our experiments show such quality is 8 percent or less from the optimum. We also show the sensitivity and robustness of our scheduling solutions against performance errors in both resources and applications.

Cost-Aware Cloud Bursting for Enterprise Applications

The high cost of provisioning resources to meet peak application demands has led to the widespread adoption of pay-as-you-go cloud computing services to handle workload fluctuations. Some enterprises with existing IT infrastructure employ a hybrid cloud model where the enterprise uses its own private resources for the majority of its computing, but then “bursts” into the cloud when local resources are insufficient. However, current commercial tools rely heavily on the system administrator’s knowledge to answer key questions such as when a cloud burst is needed and which applications must be moved to the cloud. In this article, we describe Seagull, a system designed to facilitate cloud bursting by determining which applications should be transitioned into the cloud and automating the movement process at the proper time. Seagull optimizes the bursting of applications using an optimization algorithm as well as a more efficient but approximate greedy heuristic. Seagull also optimizes the overhead of deploying applications into the cloud using an intelligent precopying mechanism that proactively replicates virtualized applications, lowering the bursting time from hours to minutes. Our evaluation shows over 100% improvement compared to naïve solutions but produces more expensive solutions compared to ILP. However, the scalability of our greedy algorithm is dramatically better as the number of VMs increase. Our evaluation illustrates scenarios where our prototype can reduce cloud costs by more than 45% when bursting to the cloud, and that the incremental cost added by precopying applications is offset by a burst time reduction of nearly 95%.