A Scalable Master-Worker Architecture for PaaS Clouds

Abstract

Clouds provide an attractive infrastructural option to deploy highly-scalable distributed applications. Platform as a Service (PaaS) clouds offer basic software stack and services along with the execution containers to simplify the hosting of user applications. However, traditional many task computing architectures cannot be hosted as-is on current PaaS platforms due to certain limitations. This paper describes a novel modified architecture for master-worker, a well-known many task computing paradigm, to take advantage of the fast scalability provided by PaaS. The architecture is transformed into a multiagent system where the distributed agents use a message broker for communication and to store the computation progress. The agents are capable of dynamically shifting between a master and a worker role based on the information available with a durable message broker. This state-less feature of the agents makes them amenable for a PaaS platform and adds fault-tolerance to the system. The experiments illustrate the promising potential of the architecture to efficiently scale computationally intensive tasks on PaaS.