A rule based data quality assessment architecture and application for electrical data

Abstract

Data quality assessment plays an important role in electricity consumption big data. It can help business people master the overall data situation, which can provide a strong guarantee for subsequent data improvement, analysis and decision. According to the electrical data quality issues, we design a rule-based data quality assessment architecture for electrical big data. It includes six types of data quality assessment indexes (such as comprehensiveness, accuracy, completeness), and the related data quality rules (such as non-empty rule and range rule), which can be used to guide the electrical data quality inspection. Meanwhile, for the accuracy, we propose an outlier detection method based on time time-relevant k-means, which is used to detect the voltage, curve and power data issues in electricity data. The experimental and simulation results show that the proposed architecture and method can work well for the comprehensive data quality assessment of electrical data.