Real-time surrogate-assisted preprocessing of streaming sensor data

Abstract

A new dynamic programming based algorithm for real-time simultaneous segmentation, compression, and C2-smoothing of potentially infinite data streams – studied in the context of streaming sensor data – is shown to be both effective and (energy) efficient. Its three key elements – the cubic splinelet based adaptive search space, the objective function (combining two measures of approximation errors), and the two-phase search space reduction technique (combining segment length based pruning and surrogate based pruning) – are presented in detail. The output quality is measured in terms of the signal approximation accuracy and the corresponding compression ratio. The numerical results show that the new algorithm outperforms both reference algorithms for all test streams, in some cases—significantly (up to 32% smaller approximation errors and up to 41% higher compression ratios). Due to the effective search space auto-adaptation and special pruning techniques, in typical cases, this is obtained with significantly lower computational cost. The proposed algorithm can be applied to various domains including online compression and smoothing of streaming data coming from IoT devices, sensor networks, and sensors located in autonomous vehicles (cars, drones) and robots. The possible application areas also include real-time IoT analytics and embedded time-series databases.