On distributed computation rate optimization for deploying cloud computing programming frameworks

Abstract

With the rapidly growing challenges of big data analytics, the need for efficient and distributed algorithms to optimize cloud computing performances is unprecedentedly high. In this paper, we consider how to optimally deploy a cloud computing programming framework (e.g., MapReduce and Dryad) over a given underlying network hardware infrastructure to maximize the end-to-end computation rate and minimize the overall computation and communication costs. The main contributions in this paper are three-fold: i) we develop a new network flow model with a generalized flowconservation law to enable a systematic design of distributed algorithms for computation rate utility maximization problems (CRUM) in cloud computing; ii) based on the network flow model, we reveal key separable properties of the dual functions of Problem CRUM, which further lead to a distributed algorithm design; and iii) we offer important networking insights and meaningful economic interpretations for the proposed algorithm and point out their connections to and distinctions from distributed algorithms design in traditional data communications networks. This paper serves as an important first step towards the development of a theoretical foundation for distributed computation analytics in cloud computing.