Time Series Problems Research Articles

Modeling time series data with deep Fourier neural networks

We present a method for training a deep neural network containing sinusoidal activation functions to fit to time-series data. Weights are initialized using a fast Fourier transform, then trained with regularization to improve generalization. A simple dynamic parameter tuning method is employed to adjust both the learning rate and the regularization term, such that both stability and efficient training are achieved. We show how deeper layers can be utilized to model the observed sequence using a sparser set of sinusoid units, and how non-uniform regularization can improve generalization by promoting the shifting of weight toward simpler units. The method is demonstrated with time-series problems to show that it leads to effective extrapolation of nonlinear trends.

An Evolving Interval Type-2 Neurofuzzy Inference System and Its Metacognitive Sequential Learning Algorithm

In this paper, we propose an evolving interval type-2 neurofuzzy inference system (IT2FIS) and its fully sequential learning algorithm. IT2FIS employs interval type-2 fuzzy sets in the antecedent part of each rule and the consequent realizes Takagi–Sugeno–Kang fuzzy inference mechanism. In order to render the inference fast and accurate, we propose a data-driven interval-reduction approach to convert interval type-1 fuzzy set in antecedent to type-1 fuzzy number in the consequent. During learning, the sequential algorithm learns a sample one-by-one and only once. The IT2FIS structure evolves automatically and adapts its network parameters using metacognitive learning mechanism concurrently. The metacognitive learning regulates the learning process by appropriate selection of learning strategies and helps the proposed IT2FIS to approximate the input–output relationship efficiently. An evolving IT2FIS employing a metacognitive learning algorithm is referred to as McTI2FIS. Performance of metacognitive interval type-2 neurofuzzy inference system (McIT2FIS) is evaluated using a set of benchmark time-series problems and is compared with existing type-2 and type-1 fuzzy inference systems. Finally, the performance of the proposed McIT2FIS has been evaluated using a practical stock price-tracking problem. The results clearly highlight that McIT2FIS performs better than other existing results in the literature.

Optimization of type-2 fuzzy weights in backpropagation learning for neural networks using GAs and PSO

In this paper the optimization of type-2 fuzzy inference systems using genetic algorithms (GAs) and particle swarm optimization (PSO) is presented. The optimized type-2 fuzzy inference systems are used to estimate the type-2 fuzzy weights of backpropagation neural networks. Simulation results and a comparative study among neural networks with type-2 fuzzy weights without optimization of the type-2 fuzzy inference systems, neural networks with optimized type-2 fuzzy weights using genetic algorithms, and neural networks with optimized type-2 fuzzy weights using particle swarm optimization are presented to illustrate the advantages of the bio-inspired methods. The comparative study is based on a benchmark case of prediction, which is the Mackey-Glass time series (for τ=17) problem.

Minimum message length analysis of multiple short time series

This paper applies the Bayesian minimum message length principle to the multiple short time series problem, yielding satisfactory estimates for all model parameters as well as a test for autocorrelation. Connections with the method of conditional likelihood are also discussed.

E-commerce business model mining and prediction

We study the problem of business model mining and prediction in the e-commerce context. Unlike most existing approaches where this is typically formulated as a regression problem or a time-series prediction problem, we take a different formulation to this problem by noting that these existing approaches fail to consider the potential relationships both among the consumers (consumer influence) and among the shops (competitions or collaborations). Taking this observation into consideration, we propose a new method for e-commerce business model mining and prediction, called EBMM, which combines regression with community analysis. The challenge is that the links in the network are typically not directly observed, which is addressed by applying information diffusion theory through the consumer-shop network. Extensive evaluations using Alibaba Group e-commerce data demonstrate the promise and superiority of EBMM to the state-of-the-art methods in terms of business model mining and prediction.

Clustering-Based Behavioural Analysis of Biological Objects

The article examines the problem of processing short time series for bioinformatics tasks using data mining methods in the field of pharmacology. The experiments were conducted using heart contraction (contraction and relaxation) power data that were obtained in experiments with laboratory animals with the goal of registering the power changes of heart contractions in different stages of experiment in a given period of time. The selected data were treated using data preprocessing technologies. The short time series were compared using various time-point similarity search methods using agglomerative hierarchical clustering, k- means clustering, modified k-means clustering and expectation-maximization clustering algorithms. Based on the clustering result evaluation the most suitable algorithm was chosen and the optimal number of clusters was determined for the least clustering error. The acquired clusters were used for to create cluster prototypes that aggregate the groups of similar heart contraction power objects. The article offers an examination of the errors produced by algorithms and methods as well as a discussion of the obtained clustering results using different evaluation methodologies. It also gives conclusions about the application of data mining methods in solving bioinformatics tasks and outlines further research directions.

Individual privacy constraints on time-series data

Many recent applications depend on time series of data containing personal information. For example, the smart grid collects and distributes time series of energy-consumption data from households. Our concern is information hiding in such data according to individual privacy constraints, considering several constraints at a time. The existing information-hiding approaches we are aware of make limiting assumptions regarding the nature of such constraints. Our approach in turn lets the individuals concerned specify information that must be hidden arbitrarily, and it also lets the data receivers specify characteristics of the data needed to perform a certain task. We use these constraints to formulate an optimization problem that generates perturbed time series that fulfill the constraints of the data receivers and do not contain more sensitive information than allowed. Next, we propose a complexity-reduction approach that speeds up solving this optimization problem for time series by orders of magnitude. Three case studies on real-world data confirm that our approach is applicable to a wide range of application domains, and that it provides more protection against well-known privacy attacks such as re-identification, reconstruction and disaggregation. In addition, we provide a Java implementation of our approach and supplementary material on our web page.11http://www.ipd.kit.edu/laforetf/inpact

A Potential Method for Determining Nonlinearity in Wind Data

This paper investigates a basic problem in modeling time series of wind data: whether there exists detectable correlation or nonlinearity in the observed wind time series. At present, a variety of linear and nonlinear time series models have been applied to predict the wind data. The first question that should be answered before building a model, however, is whether the data studied are correlated or carry nonlinearity. It would be futile to model the relationships if the pertaining wind data cannot be distinguished from the white noise. Advanced nonlinear prediction models are also not necessary if there are no nonlinear structures in the data. In this paper, we test by the surrogate data method: 1) whether the differenced wind speed time series (taking the first difference of the time series) is white noise, and 2) the presence of nonlinearity in the original wind speed time series. Nine data sets, including 10 min and hourly wind speed data, are examined. The results show that all of the differenced wind speed time series are correlated, and three out of the nine original wind speed time series satisfy the hypothesis of a linear stochastic generating process. It is concluded that for a specific wind speed time series, the nonlinearity is data-dependent from the perspective of practical time series analysis.

A novel hybrid approach based on self-organizing maps, support vector regression and particle swarm optimization to forecast solar irradiance

We forecast hourly solar irradiance time series using a novel hybrid model based on SOM (self-organizing maps), SVR (support vector regression) and PSO (particle swarm optimization). In order to solve the noise and stationarity problems in the statistical time series forecasting modelling process, SOM is applied to partition the whole input space into several disjointed regions with different characteristic information on the correlation between the input and the output. Then SVR is used to model each disjointed regions to identify the characteristic correlation. In order to reduce the performance volatility of SVM (support vector machine) with different parameters, PSO is implemented to automatically perform the parameter selection in SVR modelling. This hybrid model has been used to forecast hourly solar irradiance in Colorado, USA and Singapore. The technique is found to outperform traditional forecasting models.

Vessel traffic flow forecasting by RSVR with chaotic cloud simulated annealing genetic algorithm and KPCA

The prediction of vessel traffic flow is complicated, its accuracy is influenced by uncertain socio-economic factors, especially by the singular points existed in the statistical data. Recently, the robust v-support vector regression model (RSVR) has been successfully employed to solve non-linear regression and time-series problems with the singular points. This paper will firstly propose a novel hybrid algorithm, namely chaotic cloud simulated annealing genetic algorithm (CcatCSAGA) for optimizing the parameters of RSVR, to improve the performance of vessel traffic flow prediction. In which, the proposed CcatCSAGA employs cat mapping to carefully expand variable searching space, to overcome premature local optimum, and uses cloud model efficiently to search a better solution in a small neighborhood of the current optimal solution, to improve the search efficiency. Secondly, the kernel principal component analysis (KPCA) algorithm is adopted to determine the final input vectors from the candidate input variables. Finally, a numerical example of vessel traffic flow and its influence factors data from Tianjin are employed to test the forecasting performance of the proposed KRSVR-CcatCSAGA model.

Least square support vector machines as an alternative method in seasonal time series forecasting

The least square support vector machines (LSSSVM) model is a novel forecasting approach and has been successfully used to solve time series problems. However, the applications of LSSVM model in a seasonal time series forecasting has not been widely investigated. This study aims at developing a LSSVM model to forecast seasonal time series data. To assess the effectiveness of this model, the airline passenger series exhibits nonlinear behaviour and shows multiplicative seasonal behaviour was applied. In order to obtain the optimal model parameters of the LSSVM, a grid search algorithm and cross-validation method were employed. In this study, seasonal autoregressive integrated moving average (SARIMA) and artificial neural network (ANN) models are employed for forecasting the same data sets. Empirical results indicate that the LSSVM yields well forecasting performances. Thus, the LSSVM model provides a promising alternative for seasonal time series forecasting.

Application of Nonlinear-Autoregressive-Exogenous model to predict the hysteretic behaviour of passive control systems

This paper proposes to use the nonlinear-autogressive models with exogenous input (NARX) model to predict the hysteretic behaviour of passive control systems. Although existing analytical hysteresis models such as the generalized Bouc–Wen (BW) model and the Bouc–Wen–Baber–Noori (BWBN) model can be used to model the hysteretic behaviour of passive control systems, the generalized BW model fails to account the pinching or stiffness degradation of hysteretic systems and the BWBN model requires to tune considerable parameters before its application. Therefore, we propose this alternative approach to predict the hysteresis response of passive control systems. The NARX model is the branch of artificial intelligence which is a promising tool for the forecasting of time series problems. We adopted the NARX model to predict the hysteretic behaviour with experimental results conducted on yielding shear panel device (YSPD) and steel slit damper (SSD), respectively. A good agreement between the experimental results on both YSPD and SSD and the prediction results was achieved. We also combined the NARX model and the general regression neural network (GRNN) as a hybrid model to predict hysteretic behaviour of the SSD of which the damper design was hidden from the model training process. The performance of using the hybrid model to predict the hysteretic behaviour of SSD is reasonably well. Finally, the applicability of the hybrid model has been successfully demonstrated through the optimisation of the geometrical parameters of the SSD. We concluded that the proposed NARX model is capable to predict the hysteretic behaviour of passive control systems.

A multiple filter test for the detection of rate changes in renewal processes with varying variance

Nonstationarity of the event rate is a persistent problem in modeling time series of events, such as neuronal spike trains. Motivated by a variety of patterns in neurophysiological spike train recordings, we define a general class of renewal processes. This class is used to test the null hypothesis of stationary rate versus a wide alternative of renewal processes with finitely many rate changes (change points). Our test extends ideas from the filtered derivative approach by using multiple moving windows simultaneously. To adjust the rejection threshold of the test, we use a Gaussian process, which emerges as the limit of the filtered derivative process. We also develop a multiple filter algorithm, which can be used when the null hypothesis is rejected in order to estimate the number and location of change points. We analyze the benefits of multiple filtering and its increased detection probability as compared to a single window approach. Application to spike trains recorded from dopamine midbrain neurons in anesthetized mice illustrates the relevance of the proposed techniques as preprocessing steps for methods that assume rate stationarity. In over 70% of all analyzed spike trains classified as rate nonstationary, different change points were detected by different window sizes.

New approach to the segmentation problem for time series of arbitrary nature

We consider the problem of splitting time series of arbitrary nature (stochastic, deterministic, or mixed) into segments generated by the same mechanism. We introduce a new concept of ∈-complexity of continuous functions and give a characterization of this quantity for Holder continuous functions. On the basis of the ∈-complexity parameters, we propose a new technique for the segmentation of time series that does not require any a priori knowledge of how these series were generated.

Imbalanced Learning for Air Pollution by Meta-Cognitive Online Sequential Extreme Learning Machine

Many time series problems such as air pollution index forecast require online sequential learning rather than batch learning. One of the major obstacles for air pollution index forecast is the data imbalance problem so that forecast model biases to the majority class. This paper proposes a new method called meta-cognitive online sequential extreme learning machine (MCOS-ELM) that aims to alleviate data imbalance problem and sequential learning at the same time. Under a real application of air pollution index forecast, the proposed MCOS-ELM was compared with retrained ELM and online sequential extreme learning machine in terms of accuracy and computational time. Experimental results show that MCOS-ELM has the highest efficiency and best accuracy for predicting the minority class (i.e., the most important but with fewest training samples) of air pollution level.

Typical decision problems in the first-order autoregressive time series

Applied statistical decision theory has wide applications in decision-making fields of studies, such as economic, business management and industrial managements. In this work, following Pratt et al.’s [Introduction to statistical decision theory. 3rd ed. Cambridge, MA: The MIT Press; 2001] approach, we provide theoretical and practical formulations for the calculations of the key decision-making indices expected value of perfect information and expected value of sample information, whenever the unknown state appears to be the first-order autoregressive (AR) time series parameter that assumes a normal prior distribution. A practical procedure is furnished for calculating the decision-making indices. We treat the finite and infinite state spaces for the linear value functions and the quadratic opportunity losses. Interestingly our investigations on the distribution of the mean of the posterior distribution lead us to a general form for the corresponding statistic and its distribution, discussed by Reeves [The distribution of the maximum likelihood estimator of the parameter in the first-order AR series. Biometrika. 1972;59:387–394], Moschopoulos and Canada [The distribution function of a linear combination of chi-squares. Comput Math Appl. 1984;10:383–386], and Roychowdhury and Bhattacharya [On the performance of estimators of parameter in AR model of order one and optimal prediction under asymmetric loss. Model Assist Stat Appl. 2008;3:225–232].

The modeling and prediction of time series based on synergy of high-order fuzzy cognitive map and fuzzy c-means clustering

The time series prediction models based on fuzzy set theory have been widely applied to diverse fields such as enrollments, stocks, weather and etc., as they can handle prediction problem under uncertain circumstances in which data are incomplete or vague. Researchers have presented diverse approaches to support the development of fuzzy time series prediction models. While the existing approaches exhibit two evident shortcomings: one is that they have low efficiency of development, which is hardly applicable in the prediction problem involving large-scale time series, and the other is that fuzzy logical relationships mined in an ad hoc way cannot uncover the global characteristics of time series, which reduces accuracy of the resulting model. In this paper, a novel modeling and prediction approach of time series based on synergy of high-order fuzzy cognitive map (HFCM) and fuzzy c-means clustering is proposed, in which fuzzy c-means clustering algorithm is used to construct information granules, transform original time series into granular time series and generate a structure of HFCM prediction model in an automatic fashion. Subsequently depending on historical data of time series, the HFCM prediction model of time series is completely formed by exploiting PSO algorithm to learn all parameters of one. Finally, the developed HFCM prediction model can realize numeric prediction by performing inference in the granular space. Four benchmark time series data sets with different statistical characteristics coming from different areas are applied to validate the feasibility and effectiveness of the proposed modeling approach. The obtained results clearly show the effectiveness of the approach. The developed HFCM prediction models depend on historical data of time series and is emerged in the form of map, which is simpler, legible and have high-level interpretability. Additionally, the proposed approach also exhibits a clear ability to handle the prediction problem of large-scale time series.

Time series forecasting by neural networks: A knee point-based multiobjective evolutionary algorithm approach

In this paper, we investigate the problem of time series forecasting using single hidden layer feedforward neural networks (SLFNs), which is optimized via multiobjective evolutionary algorithms. By utilizing the adaptive differential evolution (JADE) and the knee point strategy, a nondominated sorting adaptive differential evolution (NSJADE) and its improved version knee point-based NSJADE (KP-NSJADE) are developed for optimizing SLFNs. JADE aiming at refining the search area is introduced in nondominated sorting genetic algorithm II (NSGA-II). The presented NSJADE shows superiority on multimodal problems when compared with NSGA-II. Then NSJADE is applied to train SLFNs for time series forecasting. It is revealed that individuals with better forecasting performance in the whole population gather around the knee point. Therefore, KP-NSJADE is proposed to explore the neighborhood of the knee point in the objective space. And the simulation results of eight popular time series databases illustrate the effectiveness of our proposed algorithm in comparison with several popular algorithms.

Robust design modeling and optimization of a multi-response time series for a pharmaceutical process

Robust design (RD) methods, which are based upon the concept of building quality into products or processes, are increasingly popular in the science and engineering research communities. One particular area of RD research that has not received considerable attention is in working with multiple time series responses, observed frequently within the field of pharmaceutical science. In order to determine the optimal pharmaceutical formulation, or input factor settings, suitable robust experimental design and analysis methods must be performed. To achieve this objective, the primary aim of this paper is to propose a new methodology that specifically addresses the multi-response time series problem for a pharmaceutical formulation process. First, an experimental format and framework for testing drug release kinetics is proposed by implementing a mixture experimental design and time series response modeling. Second, an alternative robust design model is developed to identify the optimal pharmaceutical formulation, based upon the time series target profiles for drug release kinetics. Finally, a case study associated with a drug development process is performed to validate the proposed model. The results of this case study indicate that the optimal drug release kinetics is significantly similar to the target profile.

Firefly Algorithm with Artificial Neural Network for Time Series Problems

Time series classification is a supervised learning method maps the input to the output using historical data. The primary objective is to discover interesting patterns hidden in the data. For the purpose of solving time series classification problems used the multi-layered perceptrons Artificial Neural Networks (ANN). The weights in the ANN are modified to provide the output values of the net, which are much closer to the values of the preferred output. For this reason, several algorithms had been proposed to train the parameters of the neural network for time series classification problems. This study attempts to hybrid the Firefly Algorithm (FA) with the ANN in order to minimize the error rate of classification (coded as FA-ANN). The FA is employed to optimize the weights of the ANN model based on the processes. The proposed FA-ANN algorithm was tested on 6 benchmark UCR time series data sets. The experimental results have revealed that the proposed FA-ANN can effectively solve time series classification problems.