Extensible architecture for high-throughput task processing based on hybrid cloud infrastructure

Abstract

Cloud computing has been growing in popularity because of the demand of scalability and extensibility for infrastructure resources. Recent work has shown that dynamic increasing resource capacity by using public cloud infrastructure services such as Amazon's EC2 can achieve higher peak throughput with lower cost. On the other hand, dedicated server infrastructure is still suitable to frontend requests and processing average workload rather than pure cloud infrastructure. In this paper, we propose a novel extensible architecture for high throughput task processing to self-adapt peak workload on the hybrid infrastructure combining fixed-size system with public cloud. Usually, public cloud only can loosely couple with fixed size system without the access of local resources. Hence, bridged queues and event based instant storage are proposed to cascade the cloud queue and local task queue and to decouple the dependency of database access during task processing. Furthermore, an implementation based on the approach of bridged queues and event based instant storage is presented.