Streaming Time Series Research Articles

On-line Elastic Similarity Measures for time series

The way similarity is measured among time series is of paramount importance in many data mining and machine learning tasks. For instance, Elastic Similarity Measures are widely used to determine whether two time series are similar to each other. Indeed, in off-line time series mining, these measures have been shown to be very effective due to their ability to handle time distortions and mitigate their effect on the resulting distance. In the on-line setting, where available data increase continuously over time and not necessary in a stationary manner, stream mining approaches are required to be fast with limited memory consumption and capable of adapting to different stationary intervals. In this sense, the computational complexity of Elastic Similarity Measures and their lack of flexibility to accommodate different stationary intervals, make these similarity measures incompatible with the requirements mentioned. To overcome these issues, this paper adapts the family of Elastic Similarity Measures – which includes Dynamic Time Warping, Edit Distance, Edit Distance for Real Sequences and Edit Distance with Real Penalty – to the on-line setting. The proposed adaptation is based on two main ideas: a forgetting mechanism and the incremental computation. The former makes the similarity consistent with streaming time series characteristics by giving more importance to recent observations, whereas the latter reduces the computational complexity by avoiding unnecessary computations. In order to assess the behavior of the proposed similarity measure in on-line settings, two different experiments have been carried out. The first aims at showing the efficiency of the proposed adaptation, to do so we calculate and compare the computation time for the elastic measures and their on-line adaptation. By analyzing the results drawn from a distance-based streaming machine learning model, the second experiment intends to show the effect of the forgetting mechanism on the resulting similarity value. The experimentation shows, for the aforementioned Elastic Similarity Measures, that the proposed adaptation meets the memory, computational complexity and flexibility constraints imposed by streaming data.

Interpretable multiple data streams clustering with clipped streams representation for the improvement of electricity consumption forecasting

This paper presents a new interpretable approach for multiple data streams clustering in a smart grid used for the improvement of forecasting accuracy of aggregated electricity consumption and grid analysis named ClipStream. Consumers time series streams are compressed and represented by interpretable features extracted from the clipped representation. The proposed representation has low computational complexity and is incremental in the sense of the windowing method. From the extracted features, outlier consumers can be simply and quickly detected. The clustering phase consists of three parts: clustering non-outlier representations, the aggregation of consumption within clusters, and unsupervised change detection procedure on aggregated time series streams windows. ClipStream behaviour and its forecasting accuracy improvement were evaluated on four different real datasets containing variable patterns of electricity consumption. The clustering accuracy with the proposed feature extraction method from the clipped representation was evaluated on 85 time series datasets from a large public repository. The results of experiments proved the stability of the proposed ClipStream in the sense of improving forecasting accuracy and showed the suitability of the proposed representation in many tested applications.

Freshwater pearl mussels as a stream water stable isotope recorder

AbstractFor several decades, stable isotopes have been a commonly used and effective tool for flow path analysis, stream water source apportionment, and transit time analysis. The Global Network of Isotopes in Precipitation repository now has monthly precipitation isotope time series extending over several years and even decades in some settings. However, stream water isotope composition time series remain rather short with only very few data sets spanning over more than a few years. A critical challenge in this respect is the collection of stream water isotope data sets across a wide variety of headwater streams and for long durations. We rely on a new approach for stream signal reconstruction based on freshwater mussels, specifically the freshwater pearl mussel Margaritifera margaritifera. We use secondary ion mass spectrometry (SIMS) to quantify oxygen isotope ratios in pearl mussel shell growth bands. In our study area, the observed seasonal variability in precipitation δ18O values ranges between −15‰ and −3‰. This input signal is strongly damped in stream water, where observed values of δ18O range between −10‰ and −6.5‰. These values are consistent with our measured average shell‐derived stream water δ18O of −7.19‰. Along successive growth bands, SIMS‐based stream water δ18Ow values varied within a seasonal range of −9‰ to −5‰. The proposed SIMS‐based shell analysis technique is obviously well suited for analysing isotopic signatures of O in shell material—especially from the perspective of reconstructing historical series of in‐stream isotope signatures.

Hierarchical symbolic dynamic filtering of streaming non-stationary time series data

Abstract This paper proposes a hierarchical feature extractor for non-stationary streaming time series based on the concept of switching observable Markov chain models. The slow time-scale non-stationary behaviors are considered to be a mixture of quasi-stationary fast time-scale segments that are exhibited by complex dynamical systems. The key idea is to model each unique stationary characteristics without apriori knowledge (e.g., number of possible unique characteristics) at a lower logical level, and capture the transitions from one low-level model to another at a higher level. In this context, the concepts in the recently developed Symbolic Dynamic Filtering is extended, to build an online algorithm suited for handling quasi-stationary data at a lower level and a non-stationary behavior at a higher level without apriori knowledge. A key observation made in this study is that the rate of change of data likelihood seems to be a better indicator of change in data characteristics compared to the traditional methods that mostly consider data likelihood for change detection. Thus, an adaptive Chinese Restaurant Process (CRP) distribution is formulated to accommodate the rate of change of data likelihood in the learning algorithm. The algorithm also aims to minimize model complexity while capturing data characteristics (likelihood). Efficacy demonstration and comparative evaluation of the proposed algorithm are performed using time series data both from a simulated system that exhibit nonlinear dynamics as well as a real residential electrical energy disaggregation case study using the publicly available reference energy disaggregation dataset (REDD). We discuss results that show that the proposed hierarchical symbolic dynamic filtering (HSDF) algorithm can identify underlying features with significantly high degree of accuracy, even under very noisy conditions. Based on our validation experiments, we demonstrate better performance of the proposed fast algorithm compared to the baseline Hierarchical Dirichlet Process-Hidden Markov Models (HDP-HMM). The proposed algorithm’s low computational complexity also makes it suitable for on-board, real time operations.

Fast multi-subsequence monitoring on streaming time-series based on Forward-propagation

Streaming time-series has drawn unprecedented interests from the computer science researchers. It requires faster execution time and less memory space than traditional approaches in processing historical time-series. Given the real-time constraint in the analysis over streaming time-series, a proper pre-processing step may not even be applicable. Subsequence monitoring is one of the main functions used in a wide range of time series related applications, e.g. quantitative trading in the stock market. In this paper, we propose a novel approach for multi- subsequence monitoring on streaming time-series. The proposed Forward-propagation NSPRING (FPNS) approach is inspired by the forward propagation mechanism in Artificial Neural Networks (ANN). In our proposed approach the concept of forward propagation is adopted to by-pass the unnecessary calculations as in NSPRING where the whole matrix is computed for the final result. FPNS computes a small part of the matrix by indexing only the necessary calculations with the aid of the forward propagation mechanism. As a result, FPNS can effectively reduce the execution time. In the experiments, we compared the scalability, execution time and memory requirement of FPNS, NSPRING, and UCR-DTW using synthetic and real datasets. The experimental results show that on average, FPNS is about three times faster than NSPRING and one order of magnitude faster than UCR-DTW. In addition, FPNS preserves the same accuracy with NSPRING while FPNS runs much faster than NSPRING.

A Novel Multi-resolution Representation for Streaming Time Series

Along with the coming of IoT (Internet of things) era, massive numbers of instruments and applications in various fields are continuously producing oceans of time series stream data, which could be characterized by its large amount, high dimensionality and continuity nature. In order to carry out different kinds of data mining tasks (similarity search, classification, clustering, prediction etc.) based on streaming time series efficiently and effectively, segmentation and representation which segment a streaming time series into several subsequences and provide more compact representation for the raw data, should be done as the first step. With the virtue of solid theoretical foundations, piecewise linear representation (PLR) has been gained success in yielding more compact representation and fewer segments, however, the current state of art PLR methods have their own flaws. In this paper, we propose a novel online time series segmentation algorithm called continuous segmentation and multi-resolution representation algorithm based on turning points (CSMR_TP), which partitions the streaming time series by a set of temporal feature points and represents the time series flexibly. Our method can not only generate more accurate approximation than the state-of-the-art of PLR algorithm, but also represent the streaming time series in a more flexible way to meet the diverse needs of users. Extensive experiments on different kinds of typical time series datasets have been conducted to demonstrate the superiorities of our method.

Efficient search for top-k discords in streaming time series

The problem of anomaly detection in streaming time series has received much attention recently. The problem addresses finding the most anomalous subsequence (discord) over a time-series stream, which might arrive at high speed. The fact that finding top-k discords is more useful than finding the most unusual subsequence since users might make a choice among the top-k discords instead of choosing only one. Hence, an efficient method of search for top-k discords in streaming time series is proposed in the paper. The method uses a lower bound threshold, a lower bounding technique on a common dimensionality reduction transform, and a state-of-the-art technique of the distance computation between two time-series subsequences to prune off unnecessary distance calculations. The three techniques are arranged in a cascading fashion to speed up the performance of the method. Furthermore, the proposed method can return a set of top-k discords on the fly. The experimental results show that the proposed method can acquire quality discords nearly identical to those obtained by HOT SAX, a well-known method of anomaly detection. Remarkably, our proposed method demonstrates a fast response in handling time-series streams at high speed.

Low Flow Trends and Frequency Analysis in the Blue Nile Basin, Ethiopia

Low flow analysis provides crucial information for the planning and design water resource development, risk assessment and environmental flow management. Understanding the low flow regimes and evaluating the magnitudes for incorporating in water resources management is vital for the countries like Ethiopia where demand for water is increasing. However, there were hardly enough studies in understanding the trends of low flow and frequency analysis. Therefore, this study focuses on evaluation of the trends in low flows and regional low flow analysis in the Blue Nile Basin, Ethiopia. In order to carry out the study, 15 river sub-basins in the Blue Nile Basin were selected based on the long term data availability and presence of quality of data. The 3-day sustained low flow (3d-slf), the 7-day sustained low flow (7d-slf) and the 14-day sustained low flow (14d-slf) models were used to extract the data from the daily time series stream data obtained from MoWIE. Trends in low flow were analyzed separately by using Mann-Kendall (MK) trend test. Low flow frequency analysis was used to estimate the long term low flow quantiles. In addition, regional analysis for estimating the quantiles for ungaged catchments was also developed based on the regional growth curve and catchment characteristic of drainage basins. The results indicated that 3d-slf, 7d-slf and 14d-slf models of low flow series indicated no significant difference for each station at 95% CI. Out of the 15 selected stations, 12 of stations have indicated decreasing; two stations indicated increasing and remaining one station with no trend. Mainly decreasing trend was associated with the land cover and climate change which results in increasing runoff and evapotranspiration respectively. Weibull distribution—GEV and LGN was found best fit based on the L-Moment Ratio Diagram (L-MRD). Hence quantile estimations have indicated diminishing magnitudes of low flow quintiles for 2 - 500 years return periods. Regional low frequency analysis has provided a very good relationship between discharge and catchment characteristics with an R2 of 0.72. Where area (A) and rainfall (R) followed by slope were found sensitive to compute in developing the regional region equations between mean low flows and the physiographic data. This study indicated that there needs to be a new water management scenario and adaptation mechanism of climate change and land use land cover dynamics for utilizing water resource in the Blue Nile Basin.

Deconstructing the Effects of Flow on DOC, Nitrate, and Major Ion Interactions Using a High‐Frequency Aquatic Sensor Network

AbstractStreams provide a physical linkage between land and downstream river networks, delivering solutes derived from multiple catchment sources. We analyzed high‐frequency time series of stream solutes to characterize the timing and magnitude of major ion, nutrient, and organic matter transport over event, seasonal, and annual timescales as well as to assess whether nitrate ( ) and dissolved organic carbon (DOC) transport are coupled in catchments, which would be expected if they are subject to similar biogeochemical controls throughout the watershed. Our data set includes in situ observations of , fluorescent dissolved organic matter (DOC proxy), and specific conductance spanning 2–4 years in 10 streams and rivers across New Hampshire, including observations of nearly 700 individual hydrologic events. We found a positive response of and DOC to flow in forested streams, but watershed development led to a negative relationship between and discharge, and thus a decoupling of the overall and DOC responses to flow. On event and seasonal timescales, and DOC consistently displayed different behaviors. For example, in several streams, FDOM yield was greatest during summer storms while yield was greatest during winter storms. Most streams had generalizable storm and DOC responses, but differences in the timing of and DOC transport suggest different catchment sources. Further, certain events, including rain‐on‐snow and summer storms following dry antecedent conditions, yielded disproportionate responses. High‐frequency data allow for increased understanding of the processes controlling solute variability and will help reveal their responses to changing climatic regimes.

Fast wavelet transform assisted predictors of streaming time series

We explore the shift variance of the decimated, convolutional Discrete Wavelet Transform, also known as Fast Wavelet Transform. We prove a novel theorem improving the FWT algorithm and implement a new prediction method suitable to the multiresolution analysis of streaming univariate datasets using compactly supported Daubechies Wavelets. An effective real value forecast is obtained synthesizing the one step ahead crystal and performing its inverse DWT, using an integrated group of estimating machines. We call Wa.R.P. (Wavelet transform Reduced Predictor) the new prediction method. A case study, testing a cryptocurrency exchange price series, shows that the proposed system can outperform the benchmark methods in terms of forecasting accuracy achieved. This result is confirmed by further tests performed on other time series. Developed in C++, Standard 2014 conformant, the code implementing the FWT and the novel Shift Variance Theorem is available to research purposes and to build efficient industrial applications.

ASAP

Time series visualization of streaming telemetry (i.e., charting of key metrics such as server load over time) is increasingly prevalent in modern data platforms and applications. However, many existing systems simply plot the raw data streams as they arrive, often obscuring large-scale trends due to small-scale noise. We propose an alternative: to better prioritize end users' attention, smooth time series visualizations as much as possible to remove noise, while retaining large-scale structure to highlight significant deviations. We develop a new analytics operator called ASAP that automatically smooths streaming time series by adaptively optimizing the trade-off between noise reduction (i.e., variance) and trend retention (i.e., kurtosis). We introduce metrics to quantitatively assess the quality of smoothed plots and provide an efficient search strategy for optimizing these metrics that combines techniques from stream processing, user interface design, and signal processing via autocorrelation-based pruning, pixel-aware preaggregation, and on-demand refresh. We demonstrate that ASAP can improve users' accuracy in identifying long-term deviations in time series by up to 38.4% while reducing response times by up to 44.3%. Moreover, ASAP delivers these results several orders of magnitude faster than alternative search strategies.

How are you feeling?: A personalized methodology for predicting mental states from temporally observable physical and behavioral information

It is believed that anomalous mental states such as stress and anxiety not only cause suffering for the individuals, but also lead to tragedies in some extreme cases. The ability to predict the mental state of an individual at both current and future time periods could prove critical to healthcare practitioners. Currently, the practical way to predict an individual’s mental state is through mental examinations that involve psychological experts performing the evaluations. However, such methods can be time and resource consuming, mitigating their broad applicability to a wide population. Furthermore, some individuals may also be unaware of their mental states or may feel uncomfortable to express themselves during the evaluations. Hence, their anomalous mental states could remain undetected for a prolonged period of time. The objective of this work is to demonstrate the ability of using advanced machine learning based approaches to generate mathematical models that predict current and future mental states of an individual. The problem of mental state prediction is transformed into the time series forecasting problem, where an individual is represented as a multivariate time series stream of monitored physical and behavioral attributes. A personalized mathematical model is then automatically generated to capture the dependencies among these attributes, which is used for prediction of mental states for each individual. In particular, we first illustrate the drawbacks of traditional multivariate time series forecasting methodologies such as vector autoregression. Then, we show that such issues could be mitigated by using machine learning regression techniques which are modified for capturing temporal dependencies in time series data. A case study using the data from 150 human participants illustrates that the proposed machine learning based forecasting methods are more suitable for high-dimensional psychological data than the traditional vector autoregressive model in terms of both magnitude of error and directional accuracy. These results not only present a successful usage of machine learning techniques in psychological studies, but also serve as a building block for multiple medical applications that could rely on an automated system to gauge individuals’ mental states.

Adaptive Anomaly Detection in Performance Metric Streams

Continuous detection of performance anomalies such as service degradations has become critical in cloud and Internet services due to impact on quality of service and end-user experience. However, the volume and fast changing behavior of metric streams have rendered it a challenging task. Many diagnosis frameworks often rely on thresholding with stationarity or normality assumption, or on complex models requiring extensive offline training. Such techniques are known to be prone to spurious false-alarms in online settings as metric streams undergo rapid contextual changes from known baselines. Hence, we propose two unsupervised incremental techniques following a two-step strategy. First, we estimate an underlying temporal property of the stream via adaptive learning and, then we apply statistically robust control charts to recognize deviations. We evaluated our techniques by replaying over 40 time-series streams from the Yahoo! Webscope S5 datasets as well as four other traces of real Web service QoS and ISP traffic measurements. Our methods achieve high detection accuracy and few false-alarms, and better performance in general compared to an open-source package for time-series anomaly detection.

Self CNN‐based time series stream forecasting

Self-learning convolutional neural network (self-CNN) for time series stream forecasting is proposed. First, the proposed self-CNN model was trained using the different types of the time series data. With the lapse of the time series stream the self-CNN model was self-trained again and again, which was using the previously predicted correct data as the input. Finally, the model was used to forecast the new time series data. The performance evaluation using the self-CNN method forecast and generate from the financial time series stream shows that the proposed self-CNN method performs better than the traditional Bollinger bands method.

Similarity search for numerous patterns over multiple time series streams under dynamic time warping which supports data normalization

A huge challenge in nowadays' data mining is similarity search in streaming time series under Dynamic Time Warping (DTW). In the similarity search, data normalization is a must to obtain accurate results. However, data normalization on the fly and the DTW calculation cost a great deal of computational time and memory space. In the paper, we present two methods, SUCR-DTW and ESUCR-DTW, which conduct similarity search for numerous prespecified patterns over multiple time-series streams under DTW supporting data normalization. These two methods utilize a combination of techniques to mitigate the aforementioned costs. The efficient methods inherit the cascading lower bounds introduced in UCR-DTW, a state-of-the-art method of similarity search in the static time series, to admissibly prune off unpromising subsequences. To be adaptive in the streaming setting, SUCR-DTW performs incremental updates on the envelopes of new-coming time-series subsequences and incremental data normalization on time-series data. However, like UCR-DTW, SUCR-DTW retrieves only similar subsequences that have the same length as the patterns. ESUCR-DTW, an extension of SUCR-DTW, can find similar subsequences whose lengths are different from those of the patterns. Furthermore, our proposed methods exploit multi-threading to have a fast response to high-speed time-series streams. The experimental results show that SUCR-DTW obtains the same precision as UCR-DTW and has lower wall clock time. Besides, the experimental results of SUCR-DTW and ESUCR-DTW reveal that the extended method has higher accuracy in spite of longer wall clock time. Also, the paper evaluates the influence of incremental z-score normalization and incremental min---max normalization on the obtained results.

Particle swarm optimization for time series motif discovery

Efficiently finding similar segments or motifs in time series data is a fundamental task that, due to the ubiquity of these data, is present in a wide range of domains and situations. Because of this, countless solutions have been devised but, to date, none of them seems to be fully satisfactory and flexible. In this article, we propose an innovative standpoint and present a solution coming from it: an anytime multimodal optimization algorithm for time series motif discovery based on particle swarms. By considering data from a variety of domains, we show that this solution is extremely competitive when compared to the state-of-the-art, obtaining comparable motifs in considerably less time using minimal memory. In addition, we show that it is robust to different implementation choices and see that it offers an unprecedented degree of flexibility with regard to the task. All these qualities make the presented solution stand out as one of the most prominent candidates for motif discovery in long time series streams. Besides, we believe the proposed standpoint can be exploited in further time series analysis and mining tasks, widening the scope of research and potentially yielding novel effective solutions.

NSPRING: the SPRING extension for subsequence matching of time series supporting normalization

Mining sequences and patterns in time series data streams is fast becoming a common practice in today's world. The rapid progress of data collection and web technologies yields tremendous growth of flowing data in various complex forms that need to be analyzed in real time. Traditional data mining methods that typically require the process data to be scanned repeatedly are not feasible for stream data applications. However, new techniques like SPRING attempt to address these challenges by identifying sequences of patterns on time series streams, thus reducing the complexity to be linear in both time and space. Unfortunately, SPRING does not support data normalization, which renders it to be not applicable for most data sets. In this paper, we are proposing an approach called NSPRING based on SPRING that extends the advantages of SPRING, e.g., low in time and space complexity, while it can support normalization. Furthermore, NSPRING retains similar mining accuracy to SPRING.

A Survey on Effective Pattern Matching in Uncertain Time Series Stream Data

A Survey on Effective Pattern Matching in Uncertain Time Series Stream Data

Classification of streaming time series under more realistic assumptions

Much of the vast literature on time series classification makes several assumptions about data and the algorithm's eventual deployment that are almost certainly unwarranted. For example, many resea...

A general framework for never-ending learning from time series streams

Time series classification has been an active area of research in the data mining community for over a decade, and significant progress has been made in the tractability and accuracy of learning. However, virtually all work assumes a one-time training session in which labeled examples of all the concepts to be learned are provided. This assumption may be valid in a handful of situations, but it does not hold in most medical and scientific applications where we initially may have only the vaguest understanding of what concepts can be learned. Based on this observation, we propose a never-ending learning framework for time series in which an agent examines an unbounded stream of data and occasionally asks a teacher (which may be a human or an algorithm) for a label. We demonstrate the utility of our ideas with experiments that consider real-world problems in domains as diverse as medicine, entomology, wildlife monitoring, and human behavior analyses.