A Hierarchical Storage System for Industrial Time-Series Data

Abstract

The increasing interest among manufacturers in monitoring and analyzing industrial systems is generating a problem related to the considerable costs associated with the storage of captured data. This paper presents a three-level hierarchical architecture for time-series data storage on cloud environments that helps to decrease those costs. In the first level, new raw time-series data is stored for a short-period of time (e.g., one day) on electronic non-volatile storage such as solid-state drives (SSDs) that provide fast access for real time visualization of the latest data. In the second level, recent time series are stored for a medium-period of time (e.g., one week) on magnetic hard disk drives (HDDs) that are lower-cost devices with slower data transfer speed. In the third level, a reduced representation of the time series obtained by applying time-series reduction techniques are stored in HDDs, for a longer period of time (e.g., one year). Dealing with those reduced representations, data storage and transmission costs can be decreased, without limiting the future use of the data in different processes.The architecture has been implemented by using the top Database Management System from three different categories: Wide column store, Time series DBMS and Graph DBMS. It has been tested using industrial time series coming from a real manufacturing environment, and with three different types of queries proposed by domain experts. The performance results regarding storage space, storage costs and query time processing are shown on the paper.