Monitoring Data Management Services on the Edge Using Enhanced TSDBs

Abstract

Many IoT systems are data intensive and are for the purpose of monitoring of critical systems. In these monitoring systems, a large volume of data steadily flow out of a large number of sensors which monitor the physical systems and environments. Thus, first of all, we need to consider how to store and manage these IoT data. Also, data sharing can greatly enhance the quality of data analytics and help with cold start of similar systems. Thus, the data storage and management solutions should consider how to help discover useful data in order to facilitate data sharing. Time series databases (TSDBs) have been developed in recent years for storing IoT data, but they have some deficiencies. One problem is that they are not very effective in supporting data sharing due to the lack of a good semantic model for proper data specifications, which is critical in data discovery. To resolve this problem, we develop a monitoring data annotation (MDA) model to guide the systematic specification of monitoring data streams. To support the realization of the MDA model, we also develop an external tool suite, which stores the additional MDA-based specifications for the data streams and interfaces with queries to perform preliminary processing to allow effective monitoring data discovery based on the MDA specifications. Another problem with current TSDBs is their focus on storing time series data that arrive at a fixed rate, but not on storing and retrieval of event data, which may come sporadically with irregular timing patterns. When storing such event data in existing TSDBs, the retrieval may have performance problems. Also, existing TSDBs do not have specific query language defined for event analysis. We develop a model for event specifications and use it to specify abnormal system states to be captured to allow timely mitigation. The event model is integrated into the TSDB by translating them to continuous queries defined in some TSDBs. Also, we develop an event storage scheme and incorporate it in TSDBs to facilitate efficient event retrieval. Experimental results show that our event solution for the TSDB is effective and efficient.