An adaptive framework for utility-based optimization of scientific applications in the cloud

Martin Koehler

doi:10.1186/2192-113x-3-4

Abstract

Cloud computing plays an increasingly important role in realizing scientific applications by offering virtualized compute and storage infrastructures that can scale on demand. This paper presents a self-configuring adaptive framework optimizing resource utilization for scientific applications on top of Cloud technologies. The proposed approach relies on the concept of utility, i.e., measuring the usefulness, and leverages the well-established principle from autonomic computing, namely the MAPE-K loop, in order to adaptively configure scientific applications. Therein, the process of maximizing the utility of specific configurations takes into account the Cloud stack: the application layer, the execution environment layer, and the resource layer, which is supported by the defined Cloud stack configuration model. The proposed framework self-configures the layers by evaluating monitored resources, analyzing their state, and generating an execution plan on a per job basis. Evaluating configurations is based on historical data and a utility function that ranks them according to the costs incurred. The proposed adaptive framework has been integrated into the Vienna Cloud Environment (VCE) and the evaluation by means of a data-intensive application is presented herein.

Highlights

Executing scientific applications in a Cloud-based environment requires dynamic allocation of computing resources, provisioning of the underlying programming environments and the applications themselves
This paper presents a Cloud stack configuration model tackling this challenge by supporting the specification of layer-specific parameters to be taken into account during the utility optimization process
The adaptive framework itself acts as autonomic manager of the different layers of the Cloud stack and the planning component relies on the utility function

Summary

Introduction

Executing scientific applications in a Cloud-based environment requires dynamic allocation of computing resources, provisioning of the underlying programming environments and the applications themselves. These applications are often Cloud-enabled by following the Software as a Service approach. Cloud computing [1,2] offers researchers the illusion of virtually infinite resources that can be allocated on demand and are accessible via the Internet. A shared goal of service providers and clients is the optimization of resource utilization while keeping costs and runtime of potentially time-consuming applications low. Researchers want to obtain results in a given period of time and they want to spend as little money as possible on compute resources.

Methods

Conclusion