A divide-and-conquer method for compression and reconstruction of smart meter data

Abstract

As smart grid sensors, smart meters generate abundant valuable data, laying the foundation for data-driven applications. However, the data collection brings huge communication pressure to electric utilities. In this context, considering that different types of devices have different power consumption patterns, and different types of data compression methods have their own applicable scenarios, we propose a divide-and-conquer method for compression and reconstruction of smart meter data. First, based on algorithm of voice activity detection (VAD), a load power fluctuation segment location method is proposed, which is combined with load event detection method to divide the load data into the event segments, fluctuation segments, and steady-state segments. Then, for the fluctuation segments, a cloud-device collaboration adaptive strategy based on the compressive sensing (CS) theory is designed, in which the sparse basis and measurement matrix are updated accordingly to ensure the high reconstruction accuracy in different scenarios. For the steady-state segments, a data compression method based on the improved symbolic aggregation approximation (SAX) is established, in which the dividing rectangle (DIRECT) algorithm and the irregular time partitioning method are combined to reduce the data volume for transmission without losing important information. For the event segments, the original data values are retained since the event power curves are relatively more complex and short duration. Finally, the received compressed data are reconstructed into the original power time series data in the master station on cloud to support advanced data analytics. Comparative experiments are conducted on the private and public datasets of 12 households in North America and China. The results show that our method has higher data reconstruction accuracy and compression efficiency compared to the existing methods.