Aging-related performance anomalies in the apache storm stream processing system

Abstract

Event stream processing has recently emerged as a popular paradigm for implementing high-volume distributed (near-)real time data processing applications. Several open source systems are today available, supporting the development of such applications, many of which developed with the technologies of the Apache Software Foundation. These so called stream processors are long-running complex software systems which may be affected by software aging, a well-known phenomenon among operation engineers, consisting of a progressive increase in the failure rate or in performance degradation of a software system over time.We address the problem of identifying symptoms and sources of software aging in the Apache Storm event stream processing system; this helps to identify proper strategies to prevent or mitigate anomalous behaviors in production environments. To this aim, we present an experimental study investigating aging manifestations in a popular system, namely Apache Storm. Results show that Storm presents anomalous behaviors in long runs, which prevent some topologies from working continuously. These can be attributed to software aging, due to Storm internal resource management mechanisms influenced by the garbage collector and the memory assigned to worker processes. We discuss the aging-related Apache Storm behaviors, and we experiment rejuvenation actions, showing that they are actually able to remove them.