SinkFlow: Fast and traceable root-cause localization for multidimensional anomaly events

Abstract

With the development of various artificial intelligence (AI)–based applications, detecting anomalies and analyzing the root causes from massive data are critical to increasing the usability of AI. Fast, accurate root-cause analysis (RCA) that finds the main reason for an anomaly, as well as reasonable explanations, helps in solving problems effectively. Thus, RCA plays an important role in troubleshooting and fault diagnosis, making its application in data analysis crucial. Previous root-cause-localization approaches for multidimensional anomaly events encompass various techniques to reduce search space and have improved the localization performance. However, they do not effectively balance the requirements in terms of performance, compatibility, and interpretability. To solve these problems, we propose a new root-cause-localization method called SinkFlow. It provides a unified framework event-aggregation Graph (EAG) to describe the constraints of event aggregation and relations between events, so it can be easily generalized to various domains. SinkFlow introduces an applicable measure evaluation method for both fundamental and derived measures to quantify the impact of events. Also, it utilizes an optimal search strategy to reduce the search space based on the anomaly behavioral consistency and deviation significance. Our experimental results on semisynthetic datasets show that SinkFlow achieved better performance than other baselines and ran much faster, achieving a 1.88% increase of the F1-score and only 25% of the time cost of the second best localization method. In addition, SinkFlow offered clear, visible explanations of the localization results to answer the questions of why they are root causes and how the anomaly is formed.