Monotone Split and Conquer for Anomaly Detection in IoT Sensory Data

Abstract

Anomaly detection is essential to guarantee the correctness of sensory data collected from Internet of Things. The latest detection approaches only operate under one of the two general forms of short-term or long-term anomaly. In this article, we propose monotone split and conquer (MSC) scheme to tackle both anomaly forms. The proposed scheme exploits the spatial–temporal correlation between neighboring sensors to detect abnormal sensory data. MSC splits the collected data into monotonic subtrends in the training phase to establish the trend-based normal profiles for the online detection phase. To eradicate the overfitting phenomenon, we further develop a general formulation to estimate the square prediction error (SPE) control limit. Both monotone split and general formulation contribute to the advancement of MSC in terms of accuracy (ACC) and false positive rate (FPR) in the online detection phase. To evaluate the performance of MSC, we design a general anomaly model to generate artificial short-term and long-term anomalies. Numerical experiments with the Intel Berkeley Research Lab (IBRL) data set demonstrate that MSC obtains about 8% higher ACC and 5% lower FPR on average when compared to existing schemes. Remarkably, MSC requires only a few observations for anomaly detection as it is applicable to real-time systems.