Abstract

Multivariate time-series data with a contextual spatial attribute have extensive use for finding anomalous patterns in a wide variety of application domains such as earth science, hurricane tracking, fraud, and disease outbreak detection. In most settings, spatial context is often expressed in terms of ZIP code or region coordinates such as latitude and longitude. However, traditional anomaly detection techniques cannot handle more than one contextual attribute in a unified way. In this paper, a new hybrid approach based on deep learning is proposed to solve the anomaly detection problem in multivariate spatio-temporal dataset. It works under the assumption that no prior knowledge about the dataset and anomalies are available. The architecture of the proposed hybrid framework is based on an autoencoder scheme, and it is more efficient in extracting features from the spatio-temporal multivariate datasets compared to the traditional spatio-temporal anomaly detection techniques. We conducted extensive experiments using buoy data of 2005 from National Data Buoy Center and Hurricane Katrina as ground truth. Experiments demonstrate that the proposed model achieves more than 10% improvement in accuracy over the methods used in the comparison where our model jointly processes the spatial and temporal dimensions of the contextual data to extract features for anomaly detection.

Highlights

  • An anomaly is an observation whose properties are significantly different from the majority of other observations under consideration which are called the normal data

  • The proposed hybrid framework consists of two major components: a multi-channel convolutional neural network-based encoder (MC-CNN-Encoder) and a Long Short-Term Memory-based decoder (LSTM-Decoder)

  • An LSTM-Decoder is designed to represent those temporal dependencies. It decodes the MC-CNN-Encoder output and reconstructs the sequence data using hidden states. It is composed of an LSTM block and a fully connected neural network (FCNN) layer

Read more

Summary

Introduction

An anomaly is an observation whose properties are significantly different from the majority of other observations under consideration which are called the normal data. Collecting data from various spatial locations and at regular time intervals are very important for some problem settings In such settings, detection of spatio-temporal outliers can lead to the discovery of interesting and generally unexpected patterns such as local instability [2]. IsolationForest [10,11] is a powerful approach for anomaly detection in multivariate data without relying on any distance or density measure It is an unsupervised, tree-based ensemble method that applies the novel concept of isolation to anomaly detection. The definition of distance between data points in multivariate spatio-temporal data with mixed attributes is often challenging This difficulty may have an adverse effect on outlier detection performance of distance-based clustering algorithms. To address these challenges, we propose a hybrid deep autoencoder framework.

Related Work
Autoencoders
Anomaly Detection with Autoencoders
Spatio-Temporal Data Pre-Processing
Anomaly Detection Procedure
Proposed Hybrid Framework
MC-CNN-Encoder
LSTM-Decoder
Experiments
Python
Data Preparation
Algorithms Used for Comparison
Framework Tuning
Anomaly Detection Results
Quantitative Evaluation of Models
Quantitative Evaluation of Framework Variants
Background
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call