Time Series Prediction Problems Research Articles

Cascade-Correlation Algorithm with Trainable Activation Functions

According to the characteristic that higher order derivatives of some base functions can be expressed by primitive functions and lower order derivatives, cascade-correlation algorithm with tunable activation functions is proposed in this paper. The base functions and its higher order derivatives are used to construct the tunable activation functions in cascade-correlation algorithm. The parallel and series constructing schemes of the activation functions are introduced. The model can simply the neural network architecture, speed up the convergence rate and improve its generalization. The efficiency is demonstrated with the two-spiral classification and Mackay-Glass time series prediction problem.

A hybrid algorithm to optimize RBF network architecture and parameters for nonlinear time series prediction

This paper proposes a novel hybrid algorithm for automatic selection of the proper input variables, the number of hidden nodes of the radial basis function (RBF) network, and optimizing network parameters (weights, centers and widths) simultaneously. In the proposed algorithm, the inputs and the number of hidden nodes of the RBF network are represented by binary-coded strings and evolved by a genetic algorithm (GA). Simultaneously, for each chromosome with fixed inputs and number of hidden nodes, the corresponding parameters of the network are real-coded and optimized by a gradient-based fast-converging parameter estimation method. Performance of the presented hybrid approach is evaluated by several benchmark time series modeling and prediction problems. Experimental results show that the proposed approach produces parsimonious RBF networks, and obtains better modeling accuracy than some other algorithms.

Robust Initialization of a Jordan Network With Recurrent Constrained Learning

In this paper, we propose a robust initialization of a Jordan network with a recurrent constrained learning (RIJNRCL) algorithm for multilayered recurrent neural networks (RNNs). This novel algorithm is based on the constrained learning concept of the Jordan network with a recurrent sensitivity and weight convergence analysis, which is used to obtain a tradeoff between the training and testing errors. In addition to using classical techniques for the adaptive learning rate and the adaptive dead zone, RIJNRCL employs a recurrent constrained parameter matrix to switch off excessive contributions from the hidden layer neurons based on weight convergence and stability conditions of the multilayered RNNs. It is well known that a good response from the hidden layer neurons and proper initialization play a dominant role in avoiding local minima in multilayered RNNs. The new RIJNRCL algorithm solves the twin problems of weight initialization and selection of the hidden layer neurons via a novel recurrent sensitivity ratio analysis. We provide the detailed steps for using RIJNRCL in a few benchmark time-series prediction problems and show that the proposed algorithm achieves superior generalization performance.

Weighted Fuzzy Rule Interpolation Based on GA-Based Weight-Learning Techniques

In this paper, we propose a weighted fuzzy interpolative reasoning method for sparse fuzzy rule-based systems. It is based on a genetic algorithm (GA)-based weight-learning technique. The proposed method can deal with fuzzy rule interpolation with weighted antecedent variables. It also can deal with fuzzy rule interpolation based on polygonal membership functions and bell-shaped membership functions. We also propose a GA-based weight-learning algorithm to automatically learn the optimal weights of the antecedent variables of the fuzzy rules. Furthermore, we apply the proposed weighted fuzzy interpolative reasoning method and the proposed GA-based weight-learning algorithm to deal with the truck backer-upper control problem, the computer activity prediction problem, multivariate regression problems, and time series prediction problems. Based on statistical analysis techniques, the experimental results show that the proposed weighted fuzzy interpolative reasoning method by the use of the optimally learned weights that were obtained by the proposed GA-based weight-learning algorithm has statistically significantly smaller error rates than the existing methods.

Time series prediction with performance guarantee

Time series prediction has many important real applications such as network resource management and quality-of-service assurance. Many different techniques have been developed to deal with time series predictions, for example, the Box–Jenkins approach and machine learning. In this study, the authors focus on the problem of time series prediction with performance guarantees and describe two machine-learning techniques, namely prediction with expert advice and conformal predictors. The authors investigate the application of these techniques to network traffic demand and propose a novel way of combining these two techniques to provide performance guarantee on predictions. The method is generic and the authors demonstrate this approach by carrying out extensive experiments on both artificially generated data and publicly available network traffic demand datasets. Empirical results show that the proposed method can increase the performance of the prediction system.

A Generalized Ellipsoidal Basis Function Based Online Self-constructing Fuzzy Neural Network

In this paper, we propose a Generalized ellipsoidal basis function based online self-constructing fuzzy neural network (GEBF-OSFNN) which extends the ellipsoidal basis function (EBF)-based fuzzy neural networks (FNNs) by permitting input variables to be modeled by dissymmetrical Gaussian functions (DGFs). Due to the flexibility and dissymmetry of left and right widths of the DGF, the partitioning made by DGFs in the input space is more flexible and more interpretable, and therefore results in a parsimonious FNN with high performance under the online learning algorithm. The geometric growing criteria and the error reduction ratio (ERR) method are used as growing and pruning strategies respectively to realize the structure learning algorithm which implements an optimal and compact network structure. The GEBF-OSFNN starts with no hidden neurons and does not need to partition the input space a priori. In addition, all free parameters in premises and consequents are adjusted online based on the ?-completeness of fuzzy rules and the linear least square (LLS) approach, respectively. The performance of the GEBF-OSFNN paradigm is compared with other well-known algorithms like RAN, RANEKF, MRAN, ANFIS, OLS, RBF-AFS, DFNN, GDFNN GGAP-RBF, OS-ELM, SOFNN and FAOS-PFNN, etc., on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world benchmark problems. Simulation results demonstrate that the proposed GEBF-OSFNN approach can facilitate a more powerful and more parsimonious FNN with better performance of approximation and generalization.

Efficiency of local models ensembles for time series prediction

Many current technological challenges require the capacity of forecasting future measurements of a phenomenon. This, in most cases, leads directly to solve a time series prediction problem. Statistical models are the classical approaches for tackling this problem. More recently, neural approaches such as Backpropagation, Radial Basis Functions and recurrent networks have been proposed as an alternative. Most neural-based predictors have chosen a global modelling approach, which tries to approximate a goal function adjusting a unique model. This philosophy of design could present problems when data is extracted from a phenomenon that continuously changes its operational regime or represents distinct operational regimes in a unbalanced manner. In this paper, two alternative neural-based local modelling approaches are proposed. Both follow the divide and conquer principle, splitting the original prediction problem into several subproblems, adjusting a local model for each one. In order to check their adequacy, these methods are compared with other global and local modelling classical approaches using three benchmark time series and different sizes (medium and high) of training data sets. As it is shown, both models demonstrate to be useful pragmatic paradigms to improve forecasting accuracy, with the advantages of a relatively low computational time and scalability to data set size.

Chaotic Time Series Prediction Using Immune Optimization Theory

To solve chaotic time series prediction problem, a novel Prediction approach for chaotic time series based on Immune Optimization Theory (PIOT) is proposed. In PIOT, the concepts and formal definitions of antigen, antibody and affinity being used for time series prediction are given, and the mathematical models of immune optimization operators being used for establishing time series prediction model are exhibited. Chaotic time series is analyzed and corresponding sample space is reconstructed by phase space reconstruction method; then, the prediction model of chaotic time series is constructed by immune optimization theory; finally, using this prediction model to forecast chaotic time series. To demonstrate the effectiveness of PIOT, the three typical chaotic nonlinear time series are generated by nonlinear dynamics systems that are Lorenz, Mackey–Glass and Henon, respectively, and are used for simulating prediction. The simulation results show that PIOT is a feasible and effective prediction method, and meanwhile provides a novel prediction approach for chaotic time series.

A deterministic model selection scheme for incremental RBFNN construction in time series forecasting

This paper presents a fast and new deterministic model selection methodology for incremental radial basis function neural network (RBFNN) construction in time series prediction problems. The development of such special designed methodology is motivated by the problems that arise when using a K-fold cross-validation-based model selection methodology for this paradigm: its random nature and the subjective decision for a proper value of K, resulting in large bias for low values and high variance and computational cost for high values. Taking into account these drawbacks, the proposed model selection approach is a combined algorithm that takes advantage of two balanced and representative training and validation sets for their use in RBFNN initialization, optimization and network model evaluation. This way, the model prediction accuracy is improved, getting small variance and bias, reducing the computation time spent in selecting the model and avoiding random and computationally expensive model selection methodologies based on K-fold cross-validation procedures.

A Prediction Method for Nonlinear Correlative Time Series

A nonlinear correlative time series prediction method is presented in this paper.It is based on the mutual information of time series and orthogonal polynomial basis neural network. Inputs of network are selected by mutual information, and orthogonal polynomial basis is used as active function.The network is trained by an error iterative learning algorithm.This proposed method for nonlinear time series is tested using two well known time series prediction problems:Gas furnace data time series and Mackey-Glass time series.

Synthesis of neural tree models by improved breeder genetic programming

Neural tree model has been successfully applied to solving a variety of interesting problems. In most previous studies, optimization of the neural tree model was divided into two steps: first structure optimization, then parameter optimization. One major problem in the evolution of structure without parameter information was noisy fitness evaluation. In this paper, an improved breeder genetic programming algorithm is proposed to the synthesis of neural tree model. The effectiveness and performance of the method are evaluated on time series prediction problems and compared with those of related methods. Simulation results show that the proposed algorithm is a potential method with better performance and effectiveness.

Storage and recall capabilities of fuzzy morphological associative memories with adjunction-based learning

We recently employed concepts of mathematical morphology to introduce fuzzy morphological associative memories (FMAMs), a broad class of fuzzy associative memories (FAMs). We observed that many well-known FAM models can be classified as belonging to the class of FMAMs. Moreover, we developed a general learning strategy for FMAMs using the concept of adjunction of mathematical morphology. In this paper, we describe the properties of FMAMs with adjunction-based learning. In particular, we characterize the recall phase of these models. Furthermore, we prove several theorems concerning the storage capacity, noise tolerance, fixed points, and convergence of auto-associative FMAMs. These theorems are corroborated by experimental results concerning the reconstruction of noisy images. Finally, we successfully employ FMAMs with adjunction-based learning in order to implement fuzzy rule-based systems in an application to a time-series prediction problem in industry.

Türkiye de Enflasyonun İleri ve Geri Beslemeli Yapay Sinir Ağlarının Melez Yaklaşımı ile Öngörüsü

Obtaining the inflation prediction is an important problem. Having this prediction accurately will lead to more accurate decisions. Various time series techniques have been used in the literature for inflation prediction. Recently, Artificial Neural Network (ANN) is being preferred in the time series prediction problem due to its flexible modeling capacity. Artificial neural network can be applied easily to any time series since it does not require prior conditions such as a linear or curved specific model pattern, stationary and normal distribution. In this study, the predictions have been obtained using the feed forward and recurrent artificial neural network for the Consumer Price Index (CPI). A new combined forecast has been proposed based on ANN in which the ANN model predictions employed in analysis were used as data.

Chaotic Time Series Prediction Using Immune Optimization Theory

To solve chaotic time series prediction problem, a novel Prediction approach for chaotic time series based on Immune Optimization Theory (PIOT) is proposed. In PIOT, the concepts and formal definitions of antigen, antibody and affinity being used for time series prediction are given, and the mathematical models of immune optimization operators being used for establishing time series prediction model are exhibited. Chaotic time series is analyzed and corresponding sample space is reconstructed by phase space reconstruction method; then, the prediction model of chaotic time series is constructed by immune optimization theory; finally, using this prediction model to forecast chaotic time series. To demonstrate the effectiveness of PIOT, the three typical chaotic nonlinear time series are generated by nonlinear dynamics systems that are Lorenz, Mackey–Glass and Henon, respectively, and are used for simulating prediction. The simulation results show that PIOT is a feasible and effective prediction method, and ...

FASTER TRAINING USING FUSION OF ACTIVATION FUNCTIONS FOR FEED FORWARD NEURAL NETWORKS

Multilayer feed-forward neural networks are widely used based on minimization of an error function. Back propagation (BP) is a famous training method used in the multilayer networks but it often suffers from the drawback of slow convergence. To make the learning faster, we propose 'Fusion of Activation Functions' (FAF) in which different conventional activation functions (AFs) are combined to compute final activation. This has not been studied extensively yet. One of the sub goals of the paper is to check the role of linear AFs in combination. We investigate whether FAF can enable the learning to be faster. Validity of the proposed method is examined by performing simulations on challenging nine real benchmark classification and time series prediction problems. The FAF has been applied to 2-bit, 3-bit and 4-bit parity, the breast cancer, Diabetes, Heart disease, Iris, wine, Glass and Soybean classification problems. The algorithm is also tested with Mackey-Glass chaotic time series prediction problem. The algorithm is shown to work better than other AFs used independently in BP such as sigmoid (SIG), arctangent (ATAN), logarithmic (LOG).

Autoregressive time series prediction by means of fuzzy inference systems using nonparametric residual variance estimation

We propose an automatic methodology framework for short- and long-term prediction of time series by means of fuzzy inference systems. In this methodology, fuzzy techniques and statistical techniques for nonparametric residual variance estimation are combined in order to build autoregressive predictive models implemented as fuzzy inference systems. Nonparametric residual variance estimation plays a key role in driving the identification and learning procedures. Concrete criteria and procedures within the proposed methodology framework are applied to a number of time series prediction problems. The learn from examples method introduced by Wang and Mendel (W&M) is used for identification. The Levenberg–Marquardt (L–M) optimization method is then applied for tuning. The W&M method produces compact and potentially accurate inference systems when applied after a proper variable selection stage. The L–M method yields the best compromise between accuracy and interpretability of results, among a set of alternatives. Delta test based residual variance estimations are used in order to select the best subset of inputs to the fuzzy inference systems as well as the number of linguistic labels for the inputs. Experiments on a diverse set of time series prediction benchmarks are compared against least-squares support vector machines (LS-SVM), optimally pruned extreme learning machine (OP-ELM), and k-NN based autoregressors. The advantages of the proposed methodology are shown in terms of linguistic interpretability, generalization capability and computational cost. Furthermore, fuzzy models are shown to be consistently more accurate for prediction in the case of time series coming from real-world applications.

Structure optimization of BiLinear Recurrent Neural Networks and its application to Ethernet network traffic prediction

The BiLinear Recurrent Neural Network (BLRNN) based on the bilinear polynomial has been successfully applied to several nonlinear time series prediction problems. In order to obtain faster convergence in training the BLRNN, two procedures are applied to the network: (i) a structure simplification procedure whereby the multiplications for bilinear components in the BLRNN are reduced and (ii) a pruning procedure entailing application of a genetic algorithm. A structurally optimized BLRNN is thereby obtained. The computational complexity of the optimized BLRNN is reduced by about half in terms of the number of weights while preserving its generalization ability. Experiments on application of the structurally optimized BLRNN to a network traffic prediction problem are conducted on a real-world Ethernet network traffic data set. Results show that the optimized BLRNN-based prediction scheme can reduce the training time by roughly half when compared with the original BLRNN while preserving the prediction accuracy of the latter in terms of Normalized Mean Square Error (NMSE).

A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks

In this paper, we present a fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks (FAOS-PFNN), where a novel structure learning algorithm incorporating a pruning strategy into new growth criteria is developed. The proposed growing procedure without pruning not only speeds up the online learning process but also facilitates a more parsimonious fuzzy neural network while achieving comparable performance and accuracy by virtue of the growing and pruning strategy. The FAOS-PFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In the parameter learning phase, all the free parameters of hidden units, regardless of whether they are newly created or originally existing, are updated by the extended Kalman filter (EKF) method. The effectiveness and superiority of the FAOS-PFNN paradigm is compared with other popular approaches like resource allocation network (RAN), RAN via the extended Kalman filter (RANEKF), minimal resource allocation network (MRAN), adaptive-network-based fuzzy inference system (ANFIS), orthogonal least squares (OLS), RBF-AFS, dynamic fuzzy neural networks (DFNN), generalized DFNN (GDFNN), generalized GAP-RBF (GGAP-RBF), online sequential extreme learning machine (OS-ELM) and self-organizing fuzzy neural network (SOFNN) on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world regression problems. Simulation results demonstrate that the proposed FAOS-PFNN algorithm can achieve faster learning speed and more compact network structure with comparably high accuracy of approximation and generalization.

Online Prediction of Time Series Data With Kernels

Kernel-based algorithms have been a topic of considerable interest in the machine learning community over the last ten years. Their attractiveness resides in their elegant treatment of nonlinear problems. They have been successfully applied to pattern recognition, regression and density estimation. A common characteristic of kernel-based methods is that they deal with kernel expansions whose number of terms equals the number of input data, making them unsuitable for online applications. Recently, several solutions have been proposed to circumvent this computational burden in time series prediction problems. Nevertheless, most of them require excessively elaborate and costly operations. In this paper, we investigate a new model reduction criterion that makes computationally demanding sparsification procedures unnecessary. The increase in the number of variables is controlled by the coherence parameter, a fundamental quantity that characterizes the behavior of dictionaries in sparse approximation problems. We incorporate the coherence criterion into a new kernel-based affine projection algorithm for time series prediction. We also derive the kernel-based normalized LMS algorithm as a particular case. Finally, experiments are conducted to compare our approach to existing methods.

Linguistic modelling and information coarsening based on prototype theory and label semantics

The theory of prototypes provides a new semantic interpretation of vague concepts. In particular, the calculus derived from this interpretation results in the same calculus as label semantics proposed by Lawry. In the theory of prototypes, each basic linguistic label L has the form ‘ about P’, where P is a set of prototypes of L and the neighborhood size of the underlying concept is described by the word ‘ about’ which represents a probability density function δ on [ 0 , + ∞ ) . In this paper we propose an approach to vague information coarsening based on the theory of prototypes. Moreover, we propose a framework for linguistic modelling within the theory of prototypes, in which the rules are concise and transparent. We then present a linguistic rule induction method from training data based on information coarsening and data clustering. Finally, we apply this linguistic modelling method to some benchmark time series prediction problems, which show that our linguistic modelling and information coarsening methods are potentially powerful tools for linguistic modelling and uncertain reasoning.