IPMOD: An efficient outlier detection model for high-dimensional medical data streams

Abstract

Outlier detection in high-dimensional medical data streams in real-time is critical and challenging research, which is of great help to disease prevention and source analysis. Although academia has done a lot of research on outlier detection of time series data streams, these methods have the following two shortcomings: (1) Insufficient detection accuracy on high-dimensional data streams; (2) Insufficient accuracy in dynamic data streams scenarios low. To this end, we propose a sliding window model based on efficient pruning and information entropy, namely IPMOD(Information Entropy-Pruning Multi-dimensional Outlier Detection). In IPMOD, we first designed a new index weight measurement method combined with information entropy to quantify the weight of different indexes in multi-dimensional data, to determine the influence of different attributes on the prediction results. Then we designed a new sliding window and sub-sequence measurement mechanism to judge whether the data is abnormal based on the distance between the target sequence and the non-self-match. After that, we designed a pruning strategy to further reduce the computational complexity of the algorithm. The final comprehensive experiment shows that our proposed scheme not only has higher detection accuracy than many current schemes on multiple sets of real data-sets but also can quickly detect outliers in different medical data streams in real-time.