SVAD: End-to-End Sensory Data Analysis for IoBT-Driven Platforms

Abstract

The rapid advancement of IoT technologies has led to its flexible adoption in battle field networks, known as Internet of Battlefield Things (IoBT) networks. One important application of IoBT networks is the weather sensory network characterized with a variety of weather, land and environmental sensors. This data contains hidden trends and correlations, needed to provide situational awareness to soldiers and commanders. To interpret the incoming data in real-time, machine learning algorithms are required to automate strategic decision-making. Existing solutions are not well-equipped to provide the fine-grained feedback to military personnel and cannot facilitate a scalable, end-to-end platform for fast unlabeled data collection, cleaning, querying, analysis and threats identification. In this work, we present a scalable end-to-end IoBT data driven platform for SVAD (Storage, Visualization, Anomaly Detection) analysis of heterogeneous weather sensor data. Our SVAD platform includes extensive data cleaning techniques to denoise efficiently data to differentiate data from anomalies and noise data instances. We perform comparative analysis of unsupervised machine learning algorithms for multi-variant data analysis and experimental evaluation of different data ingestion pipelines to show the ability of the SVAD platform for (near) real-time processing. Our results indicate impending turbulent weather conditions that can be detected by early anomaly identification and detection techniques.