Cocv: A compression algorithm for time-series data with continuous constant values in IoT-based monitoring systems

Abstract

Sensor-generated time-series data now constitutes a significant and growing portion of the world’s data due to the rapid proliferation of the Internet of Things (IoT). The transmission and storage of such voluminous data have emerged as enormous challenges. Data compression and reduction strategies have been instrumental in mitigating these challenges to some extent. However, they have exhibited limitations when applied to real-time IoT-based monitoring systems. This stems from their failure to adequately consider the stringent requirements of real-time data transmission and the continuous constant-value redundancy within periodic monitoring data. Consequently, we introduce a dedicated compression algorithm tailored specifically for time-series data within periodic IoT-based monitoring systems, namely Cocv. It takes advantage of the continuous constant-value repetition of the time-series data to compress data by discarding redundant data points. It can not only compress static batches of data but also dynamically compress data streams to improve system performance in real-time IoT-based monitoring systems. The offline Cocv outperforms traditional compressors on gas-leak monitoring data with a compression ratio of 98.5%, maintaining a decent speed for both compression and decompression. In an actual IoT-based gas-leak monitoring system, the online Cocv improves handling capacity by 255%, reading speed by 728%, reduces bandwidth consumption by 94%, and storage space consumption by 98% compared to the original scheme.