Evaluating the impact of a coordinated checkpointing in distributed data streams processing systems using discrete event simulation

Abstract

Data Streams Processing systems process continuous flows of data under Quality of Service requirements. Data streams often contain critical information which requires real-time processing. To guarantee systems' dependability and avoid information loss, one must use a fault-tolerance strategy. However, there are several strategies available, and the proper evaluation of which mechanism is better for each system architecture is challenging, especially in large-scale distributed systems. In this paper, we propose a discrete simulation model for investigating the impacts of the Coordinated Checkpoint fault tolerance strategy imposes on Data Stream Processing Systems. Results show that this strategy critically affects stream processing in failure-prone situations due to an increase in latency up to 120% and information loss, reaching 95% of the processing window in the worst case.