A gradient-based algorithm for trend and outlier prediction in dynamic data streams

Abstract

Trend and outlier are frequently used to derive early warning predictive signal to decision maker in order to achieve ultimate quality decision outcome in domain specific (e.g. commercial, scientific, biomedical and engineering, just to name a few) applications. We develop a gradient-based algorithm using sample entropy gradient(SEG) for trend and outlier prediction in high frequency time series data streams. L2 similarity measure (Euclidean distance between two linearized gradient curves is then computed and used to quantify the degree of similarity and compared with a threshold L2 value to judge the extend of dissimilarity that would be classified as outlier. SEG algorithm which circumvents the need to pre-specify tolerance parameter in those cross sample entropy (CSE)-based algorithms that invariably involve real domain expert to set the tolerance threshold. We conduct real data experiments on SEG algorithm to two application areas: dynamic wind speed data stream; and financial time series data. Our experiments demonstrated that SEG algorithm can be feasibly used in online implementation to derive predictive early warning signals to domain-specific decision maker.