On QoS-aware scheduling of data stream applications over fog computing infrastructures

Abstract

Fog computing is rapidly changing the distributed computing landscape by extending the Cloud computing paradigm to include wide-spread resources located at the network edges. This diffused infrastructure is well suited for the implementation of data stream processing (DSP) applications, by possibly exploiting local computing resources. Storm is an open source, scalable, and fault-tolerant DSP system designed for locally distributed clusters. We made it suitable to operate in a geographically distributed and highly variable environment; to this end, we extended Storm with new components that allow to execute a distributed QoS-aware scheduler and give self-adaptation capabilities to the system. In this paper we provide a thorough experimental evaluation of the proposed solution using two sets of DSP applications: the former is characterized by a simple topology with different requirements; the latter comprises some well known applications (i.e., Word Count, Log Processing). The results show that the distributed QoS-aware scheduler outperforms the centralized default one, improving the application performance and enhancing the system with runtime adaptation capabilities. However, complex topologies involving many operators may cause some instability that can decrease the DSP application availability.