Real-world Time Series Research Articles

Taxi Demand Forecasting Utilizing Neighbourhood Influence Based on Ensemble Learning

Abstract: This Time series forecasting (TSF) assists in making better strategic decisions under uncertain circumstances so that financial crisis can be avoided, wise investments can be made, under/over contracting of utility can be avoided, staffs can be scheduled appropriately, service providers can provide better service, mankind can get prepared for natural disasters and many more. However, the accuracy in forecasting plays a vital role and achieving such is a challenging task owing to the vagueness and nonlinearity associated with most of the real world time series. Therefore, improving the forecasting accuracy has become a keen area of interest among the forecasters from different domains of science and engineering. In this work, a time series forecasting model for Chicgo taxi trip data is proposed. The proposed model utilizes neighbourhood influence in terms of taxi demand to predict taxi demand. Keywords: Time series forecasting, Regularization, Regression

Fast Summarization of Long Time Series with Graphics Processor

Summarization of a long time series often occurs in analytical applications related to decision-making, modeling, planning, and so on. Informally, summarization aims at discovering a small-sized set of typical patterns (subsequences) to briefly represent the long time series. Apparent approaches to summarization like motifs, shapelets, cluster centroids, and so on, either require training data or do not provide an analyst with information regarding the fraction of the time series that a typical subsequence found corresponds to. Recently introduced, the time series snippet concept overcomes the above-mentioned limitations. A snippet is a subsequence that is similar to many other subsequences of the time series with respect to a specially defined similarity measure based on the Euclidean distance. However, the original Snippet-Finder algorithm has cubic time complexity concerning the lengths of the time series and the snippet. In this article, we propose the PSF (Parallel Snippet-Finder) algorithm that accelerates the original snippet discovery schema with GPU and ensures acceptable performance over very long time series. As opposed to the original algorithm, PSF splits the calculation of the similarity of all the time series subsequences to a snippet into several steps, each of which is performed in parallel. Experimental evaluation over real-world time series shows that PSF outruns both the original algorithm and a straightforward parallelization.

TSFuse: automated feature construction for multiple time series data

A central paradigm for building predictive models from time series data is to convert the data into a feature vector representation and then apply standard inductive learners. Typically, the conversion is done by manually defining features, which is an extremely time-consuming and error-prone process. This has motivated the development of algorithms that automatically construct features from time series. However, these systems are typically designed for univariate time series data. In contrast, many real-world applications require analyzing time series consisting of data collected by multiple sensors. In this context, it is often useful to derive new series by fusing the collected data both within a sensor and across multiple different sensors. Unfortunately, this poses additional challenges for automated construction as exponentially more operations are possible than in the univariate case. This paper proposes an automated feature construction system called TSFuse, which supports fusion and explores the search space in a computationally efficient way. We perform an empirical evaluation on real-world time series classification datasets and show that our system is able to find a better feature representation compared to existing feature construction systems for univariate time series data.

Traffic forecasting with missing data via low rank dynamic mode decomposition of tensor

Traffic forecasting is an important part in realising intelligent traffic management, which helps traffic controllers and travellers make effective decisions. However, traffic forecasting accuracy is often affected by missing traffic data due to hardware and software failure. Therefore, accurate prediction based on incomplete traffic data is an important problem as well as a challenge. Though many approaches recover the missing values before prediction, the errors from the data-filling step are likely to cause additional bias to the prediction result. Besides, this tactic is difficult to guarantee the timeliness and may impede real-time prediction. In this case, a traffic forecasting model is proposed to directly predict traffic data with missing values. This model develops tensor formed dynamic mode decomposition, recording the dynamic information of traffic data into a state transition tensor. In addition, the model takes low rank property of the dynamic tensor and the similarity of temporal variation trend into consideration. In order to verify the effectiveness and the robustness of the proposed model, experiments were performed on two real-world time series datasets. The results demonstrate that the model achieves better performance on forecasting than other baseline approaches under the impact of missing data.

Time Series Forecasting via Learning Convolutionally Low-Rank Models

Recently, Liu and Zhang studied the rather challenging problem of time series forecasting from the perspective of compressed sensing. They proposed a no-learning method, named Convolution Nuclear Norm Minimization (CNNM), and proved that CNNM can exactly recover the future part of a series from its observed part, provided that the series is convolutionally low-rank. While impressive, the convolutional low-rankness condition may not be satisfied whenever the series is far from being seasonal, and is in fact brittle to the presence of trends and dynamics. This paper tries to approach the issues by integrating a learnable, orthonormal transformation into CNNM, with the purpose for converting the series of involute structures into regular signals of convolutionally low-rank. We prove that the resultant model, termed Learning-Based CNNM (LbCNNM), strictly succeeds in identifying the future part of a series, as long as the transform of the series is convolutionally low-rank. To learn proper transformations that may meet the required success conditions, we devise an interpretable method based on Principal Component Pursuit (PCP). Equipped with this learning method and some elaborate data argumentation skills, LbCNNM not only can handle well the major components of time series (including trends, seasonality and dynamics), but also can make use of the forecasts provided by some other forecasting methods; this means LbCNNM can be used as a general tool for model combination. Extensive experiments on 100,452 real-world time series from Time Series Data Library (TSDL) and M4 Competition (M4) demonstrate the superior performance of LbCNNM.

AvgNet: Adaptive Visibility Graph Neural Network and Its Application in Modulation Classification

Our digital world is full of time series and graphs which capture the various aspects of many complex systems. Traditionally, there are respective methods in processing these two different types of data, e.g., Recurrent Neural Network (RNN) and Graph Neural Network (GNN), while in recent years, time series could be mapped to graphs by using the techniques such as Visibility Graph (VG), which simultaneously captures relevant aspects of both local and global dynamics in an easy way, so that researchers can use graph algorithms to mine the knowledge in time series and gain special latent graph representation features. Such mapping methods establish a bridge between time series and graphs, and have high potential to facilitate the analysis of various real-world time series. However, the VG method and its variants are just based on fixed rules and thus lack of flexibility, largely limiting their application in reality. In this paper, we propose an Adaptive Visibility Graph (AVG) algorithm that can adaptively map time series into graphs, based on which we further establish an end-to-end classification framework AvgNet, by utilizing GNN model DiffPool as the classifier. We then adopt AvgNet for radio signal modulation classification which is an important task in the field of wireless communication. The simulations validate that AvgNet outperforms a series of advanced deep learning methods, achieving the state-of-the-art performance in this task.

A Survey on Time Series Forecasting Approaches and Applications

Abstract: This Time series forecasting (TSF) assists in making better strategic decisions under uncertain circumstances so that financial crisis can be avoided, wise investments can be made, under/over contracting of utility can be avoided, staffs can be scheduled appropriately, service providers can provide better service, mankind can get prepared for natural disasters and many more.However, the accuracy in forecasting plays a vital role and achieving such is a challenging task owing to the vagueness and nonlinearity associated with most of the real world time series. Therefore, improving the forecasting accuracy has become a keen area of interest among the forecasters from different domains of science and engineering. In this work a survey on time series forecasting approaches on various applications has been performed. Keywords: Time series forecasting, Regularization, Regression

Missing Data Imputation in GNSS Monitoring Time Series Using Temporal and Spatial Hankel Matrix Factorization

GNSS time series for static reference stations record the deformation of monitored targets. However, missing data are very common in GNSS monitoring time series because of receiver crashes, power failures, etc. In this paper, we propose a Temporal and Spatial Hankel Matrix Factorization (TSHMF) method that can simultaneously consider the temporal correlation of a single time series and the spatial correlation among different stations. Moreover, the method is verified using real-world regional 10-year period monitoring GNSS coordinate time series. The Mean Absolute Error (MAE) and Root-Mean-Square Error (RMSE) are calculated to compare the performance of TSHMF with benchmark methods, which include the time-mean, station-mean, K-nearest neighbor, and singular value decomposition methods. The results show that the TSHMF method can reduce the MAE range from 32.03% to 12.98% and the RMSE range from 21.58% to 10.36%, proving the effectiveness of the proposed method.

Echo state network with logistic mapping and bias dropout for time series prediction

An echo state network (ESN) is a special structure of a recurrent neural network in which the recurrent neurons are randomly connected. ESN models that have achieved high accuracy on time series prediction tasks can be utilized as time series prediction models in many fields. Nevertheless, in most ESN models, the input weights are irregularly generated and the reservoir layer units are generally redundant, which cannot guarantee that the ESN models will always be optimal for a given task. In this paper, a novel ESN model that combines logistic mapping (LM) and bias dropout (BD) algorithms is proposed to optimize the irregular input weight matrix and generate a superior and simpler reservoir. Initially, the initial input weight matrix of ESN is replaced by an LM input weight matrix that is generated by the recurrent LM algorithm. Meanwhile, the reservoir, which can convert the input space into a high-dimensional feature space, is formed by the input signals and the LM input weight matrix. Then, the units with low activation values that are determined by a reservoir dropout probability are discarded through the BD algorithm. The dropout probability is determined by the contributions of the reservoir units to the training performance. Three multivariable benchmark tasks and four univariate real-world time series tasks indicate that the proposed LM-BD-ESN model is effective in reducing the testing time, reservoir size, and model complexity while improving the performance of the traditional ESN.

OnlineSTL

Decomposing a complex time series into trend, seasonality, and remainder components is an important primitive that facilitates time series anomaly detection, change point detection, and forecasting. Although numerous batch algorithms are known for time series decomposition, none operate well in an online scalable setting where high throughput and real-time response are paramount. In this paper, we propose OnlineSTL, a novel online algorithm for time series decomposition which is highly scalable and is deployed for real-time metrics monitoring on high-resolution, high-ingest rate data. Experiments on different synthetic and real world time series datasets demonstrate that OnlineSTL achieves orders of magnitude speedups (100x) for large seasonalities while maintaining quality of decomposition.

Labeling Self-Tracked Menstrual Health Records With Hidden Semi-Markov Models.

Globally, millions of women track their menstrual cycle and fertility via smartphone-based health apps, generating multivariate time series with frequent missing data. To leverage this type of data for studies of fertility or studies of the effect of the menstrual cycle on symptoms and diseases, it is critical to have methods for identifying reproductive events, such as ovulation, pregnancy losses or births. Here, we present a hierarchical approach relying on hidden semi-Markov models that adapts to changes in tracking behavior, explicitly captures variable- and state- dependent missingness, allows for variables of different type, and quantifies uncertainty. The accuracy on simulated data reaches 98% with no missing data and 90% with realistic missingness. On our partially labeled real-world time series, the accuracy reaches 93%. Our method also accurately predicts cycle length by learning user characteristics. Its implementation is publicly available (HiddenSemiMarkov R package) and transferable to any health time series, including self-reported symptoms and occasional tests.

Gibbs sampling for Bayesian estimation of triple seasonal autoregressive models

In last few decades some researchers have extended existing time series models to fit high frequency time series characterized by exhibiting multiple seasonalities. In the Bayesian framework, most of the work in this direction is only at the level of double seasonality. Therefore, in this paper we have the objective to fill part of this gap by extending autoregressive models to fit time series that have three layers of seasonality, i.e., triple seasonal autoregressive (TSAR) models, and introducing the Bayesian estimation for these models using the Gibbs sampling algorithm. In order to achieve this objective, we first assume the model errors are normally distributed and employ the normal-gamma and g priors for the model parameters. We then derive the full conditional posterior distributions to be multivariate normal for the model coefficients and inverse gamma for the model variance. Using these closed-form conditional posterior distributions, we propose the Gibbs sampling algorithm to approximate empirically the joint posterior of the TSAR model parameters, enabling us easily to carry out the Bayesian estimation of TSAR models. By employing the g prior for the model parameters, we execute an extensive simulation study to evaluate the efficiency of the proposed Bayesian estimation of TSAR models, and we then apply our work to real-world hourly electricity load time series datasets in six European countries.

Soft Load Shedding Based Demand Control of Residential Consumers

Power generation and consumption is an instantaneous process and maintaining the balance between demand and supply is crucial since the demand and supply mismatch leads to various risks like over-investment, over-generation, under-generation, and the collapse of the power system. Therefore, the reduction in demand and supply mismatch is critical to ensure the safety and reliability of power system operation and economics. A typical and common approach, called full load shedding (FLS), is practiced in cases where electric power demand exceeds the available generation. FLS operation alleviates the power demand by cutting down the load for an entire area or region, which results in several challenges and problems for the utilities and consumers. In this study, a demand-side management (DSM) technique, called Soft-load shedding (SLS), is proposed, which uses data analytics and software-based architecture, and utilizes the real-world time-series energy consumption data available at one-minute granularity for a diversified group of residential consumers. The procedure is based on pattern identification extracted from the dataset and allocates a certain quota of power to be distributed on selected consumers such that the excessive demand is reduced, thereby minimizing the demand and supply mismatch. The results show that the proposed strategy obtains a significant reduction in the demand and supply mismatch such that the mismatch remains in the range of 10–15%, especially during the period where demand exceeds generation, operating within the utility constraints, and under the available generation, to avoid power system failure without affecting any lifeline consumer, with a minimum impact on the consumer’s comfort.

A New Hierarchical Temporal Memory Algorithm Based on Activation Intensity.

As a human-cortex-inspired computing model, hierarchical temporal memory (HTM) has shown great promise in sequence learning and has been applied to various time-series applications. HTM uses the combination of columns and neurons to learn the temporal patterns within the sequence. However, the conventional HTM model compacts the input into two naive column states—active and nonactive, and uses a fixed learning strategy. This simplicity limits the representation capability of HTM and ignores the impacts of active columns on learning the temporal context. To address these issues, we propose a new HTM algorithm based on activation intensity. By introducing the column activation intensity, more useful and fine-grained information from the input is retained for sequence learning. Furthermore, a self-adaptive nonlinear learning strategy is proposed where the synaptic connections are dynamically adjusted according to the activation intensity of columns. Extensive experiments are carried out on two real-world time-series datasets. Compared to the conventional HTM and LSTM model, our method achieved higher accuracy and less time overhead.

Interpretable cognitive learning with spatial attention for high-volatility time series prediction

In reality, some kinds of time series have the characteristics of high volatility, where fluctuation patterns of sequential data contain rich semantic knowledge and represent the spatial features from two-dimensional perspective. Especially, some non-trivial fluctuations may provide key information for the forecasting of time series. However, how to appropriately represent the fluctuation patterns and achieve the causal reasoning among them remains open. In this work, by learning the causal knowledge of fluctuation patterns, a novel spatial attention fuzzy cognitive map with high-order structure is proposed for the interpretable prediction of time series with high volatility. Firstly, a kind of extended polar fuzzy information granules is utilized to convert time series into granule sequences with interpretable fluctuation features, based on which fuzzy cognitive maps can be constructed by using full data-driven way. Secondly, in order to capture the key fluctuation patterns, the attention mechanism is first introduced into fuzzy cognitive map, where the spatial features of the focused patterns can be taken fully utilized. Thirdly, the high-order structure is involved into the proposed model for the learning of the temporal knowledge existing in the pattern sequences. Finally, real-world financial time series with strong noises and high volatility are empirically utilized to verify the promising performance of the proposed method.

Prediction of chaotic time series based on Nyström Cauchy kernel conjugate gradient algorithm

Chaotic time series can well reflect the nonlinearity and non-stationarity of real environment changes. The traditional kernel adaptive filter (KAF) with second-order statistical characteristics suffers performance degeneration dramatically for predicting chaotic time series containing noises and outliers. In order to improve the robustness of adaptive filters in the presence of impulsive noise, a nonlinear similarity measure named Cauchy kernel loss (CKL) is proposed, and the global convexity of CKL is guaranteed by the half-quadratic (HQ) method. To improve the convergence rate of stochastic gradient descent and avoid a local optimum simultaneously, the conjugate gradient (CG) method is used to optimize CKL. Furthermore, to address the issue of kernel matrix network growth, the Nyström sparse strategy is adopted to approximate the kernel matrix and then the probability density rank-based quantization (PRQ) is used to improve the approximation accuracy. To this end, a novel Nyström Cauchy kernel conjugate gradient with PRQ (NCKCG-PRQ) algorithm is proposed for the prediction of chaotic time series in this paper. Simulations on prediction of synthetic and real-world chaotic time series validate the advantages of the proposed algorithm in terms of filtering accuracy, robustness, and computational storage complexity.

MSTL: A Seasonal-Trend Decomposition Algorithm for Time Series with Multiple Seasonal Patterns

The decomposition of time series into components is an important task that helps to understand time series and can enable better forecasting. Nowadays, with high sampling rates leading to high-frequency data (such as daily, hourly, or minutely data), many real-world datasets contain time series data that can exhibit multiple seasonal patterns. Although several methods have been proposed to decompose time series better under these circumstances, they are often computationally inefficient or inaccurate. In this study, we propose Multiple Seasonal-Trend decomposition using Loess (MSTL), an extension to the traditional Seasonal-Trend decomposition using Loess (STL) procedure, allowing the decomposition of time series with multiple seasonal patterns. In our evaluation on synthetic and a perturbed real-world time series dataset, compared to other decomposition benchmarks, MSTL demonstrates competitive results with lower computational cost. The implementation of MSTL is available in the R package forecast.

Embedded Point Iteration Based Recursive Algorithm for Online Identification of Nonlinear Regression Models

This paper presents a novel online identification algorithm for nonlinear regression models. The online identification problem is challenging due to the presence of nonlinear structure in the models. Previous works usually ignore the special structure of nonlinear regression models, in which the parameters can be partitioned into a linear part and a nonlinear part. In this paper, we develop an efficient recursive algorithm for nonlinear regression models based on analyzing the equivalent form of variable projection (VP) algorithm. By introducing the embedded point iteration (EPI) step, the proposed recursive algorithm can properly exploit the coupling relationship of linear parameters and nonlinear parameters. In addition, we theoretically prove that the proposed algorithm is mean-square bounded. Numerical experiments on synthetic data and real-world time series verify the high efficiency and robustness of the proposed algorithm.

CLPVG: Circular limited penetrable visibility graph as a new network model for time series.

A visibility graph transforms time series into graphs, facilitating signal processing by advanced graph data mining algorithms. In this paper, based on the classic limited penetrable visibility graph method, we propose a novel mapping method named circular limited penetrable visibility graph, which replaces the linear visibility line in limited penetrable visibility graph with nonlinear visibility arc for pursuing more flexible and reasonable mapping of time series. Tests on degree distribution and some common network features of the generated graphs from typical time series demonstrate that our circular limited penetrable visibility graph can effectively capture the important features of time series and show higher robust classification performance than the traditional limited penetrable visibility graph in the presence of noise. The experiments on real-world time-series datasets of radio and electroencephalogram signals also suggest that the structural features provided by a circular limited penetrable visibility graph, rather than a limited penetrable visibility graph, are more useful for time-series classification, leading to higher accuracy. This classification performance can be further enhanced through structural feature expansion by adopting subgraph networks. All of these results demonstrate the effectiveness of our circular limited penetrable visibility graph model.

On the Validity of Detrended Fluctuation Analysis at Short Scales.

Detrended Fluctuation Analysis (DFA) has become a standard method to quantify the correlations and scaling properties of real-world complex time series. For a given scale ℓ of observation, DFA provides the function , which quantifies the fluctuations of the time series around the local trend, which is substracted (detrended). If the time series exhibits scaling properties, then asymptotically, and the scaling exponent is typically estimated as the slope of a linear fitting in the vs. plot. In this way, measures the strength of the correlations and characterizes the underlying dynamical system. However, in many cases, and especially in a physiological time series, the scaling behavior is different at short and long scales, resulting in vs. plots with two different slopes, at short scales and at large scales of observation. These two exponents are usually associated with the existence of different mechanisms that work at distinct time scales acting on the underlying dynamical system. Here, however, and since the power-law behavior of is asymptotic, we question the use of to characterize the correlations at short scales. To this end, we show first that, even for artificial time series with perfect scaling, i.e., with a single exponent valid for all scales, DFA provides an value that systematically overestimates the true exponent . In addition, second, when artificial time series with two different scaling exponents at short and large scales are considered, the value provided by DFA not only can severely underestimate or overestimate the true short-scale exponent, but also depends on the value of the large scale exponent. This behavior should prevent the use of to describe the scaling properties at short scales: if DFA is used in two time series with the same scaling behavior at short scales but very different scaling properties at large scales, very different values of will be obtained, although the short scale properties are identical. These artifacts may lead to wrong interpretations when analyzing real-world time series: on the one hand, for time series with truly perfect scaling, the spurious value of could lead to wrongly thinking that there exists some specific mechanism acting only at short time scales in the dynamical system. On the other hand, for time series with true different scaling at short and large scales, the incorrect value would not characterize properly the short scale behavior of the dynamical system.