Discovery of Anomalous Windows through a Robust Nonparametric Multivariate Scan Statistic (RMSS)

Abstract

This paper studies unusual phenomena by discovering anomalous windows in multivariate spatial data. Such an anomalous window is a group of contiguous spatial objects indicating the occurrence of unusual phenomenon in terms of multiple variables. The paper presents a novel Robust non-parametric Multivariate Scan Statistic (RMSS). In contrast to the existing work, the authors’ approach is designed to deal with anomalous window discovery in multivariate data. They propose their multivariate scan statistic that employs the robust Mahalanobis distance which enables taking into account multiple behavioral attributes at the same time and their correlations for the discovery of significant anomalous windows. The proposed multivariate scan statistic is non-parametric such that it does not rely on any prior assumption about the data distribution. It is robust such that it can handle data with large amount of outliers, up to 50% of the overall data size. It is also affine equivariant such that affine transformation such as stretch or rotation of the data would not affect the results. The authors evaluate their approach with (a) real-world multivariate climate data for discovering natural disasters and climate changes, (b) real-world multivariate traffic accident data for identifying accident hubs, which are route segments with underlying accident-prone issues, and (c) synthetic data of both continuous and discrete multivariate distribution for identifying clusters of known outliers under different outlier percentage in data. They compare their results to state of the art multivariate scan statistic method (Kulldorff et al., 2007). The evaluation shows the detection power of the authors’ method, and the significant improvement over the existing methods.