On the Evaluation, Management and Improvement of Data Quality in Streaming Time Series

Abstract

Listen

Translate article

The Internet of things (IoT) technologies plays a key role in the Fourth Industrial Revolution (Industry 4.0). It means the digitisation of the industry and its services to improve the productiveness. To achieve the necessary information throughout the different processes, useful data streams are obtained to provide the Artificial Intelligence and Big data algorithms. However, strategic decision-making based on these algorithms may not be successful if they have been developed based on inadequate low-quality data. This research work proposes a set of metrics to measure Data Quality (DQ) in streaming time series and implements and validates a set of techniques and tools that allow monitoring and improving the quality of the information. These techniques allow the early detection of problems that arise in relation to the quality of the data collected; and, in addition, they provide some mechanisms to solve these problems. Later, also as part of the work, a use case related to industrial field is presented, where these techniques and tools have been deployed into the data management, monitoring and data analysis platform. This integration provides additional functionality to the platform, a Decision Support System (DSS) named <i>DQ-REMAIN</i> (<i>Data Quality REport MAnagement and ImprovemeNt</i>), to decision-making regarding the quality of data obtained from streaming time series.