Mackey-Glass Chaotic Time Series Research Articles

A modified fractional least mean square algorithm for chaotic and nonstationary time series prediction

A method of modifying the architecture of fractional least mean square (FLMS) algorithm is presented to work with nonlinear time series prediction. Here we incorporate an adjustable gain parameter in the weight adaptation equation of the original FLMS algorithm and absorb the gamma function in the fractional step size parameter. This approach provides an interesting achievement in the performance of the filter in terms of handling the nonlinear problems with less computational burden by avoiding the evaluation of complex gamma function. We call this new algorithm as the modified fractional least mean square (MFLMS) algorithm. The predictive performance for the nonlinear Mackey glass chaotic time series is observed and evaluated using the classical LMS, FLMS, kernel LMS, and proposed MFLMS adaptive filters. The simulation results for the time series with and without noise confirm the superiority and improvement in the prediction capability of the proposed MFLMS predictor over its counterparts.

Levenberg-Marquardt Algorithm for Mackey-Glass Chaotic Time Series Prediction

For decades, Mackey-Glass chaotic time series prediction has attracted more and more attention. When the multilayer perceptron is used to predict the Mackey-Glass chaotic time series, what we should do is to minimize the loss function. As is well known, the convergence speed of the loss function is rapid in the beginning of the learning process, while the convergence speed is very slow when the parameter is near to the minimum point. In order to overcome these problems, we introduce the Levenberg-Marquardt algorithm (LMA). Firstly, a rough introduction is given to the multilayer perceptron, including the structure and the model approximation method. Secondly, we introduce the LMA and discuss how to implement the LMA. Lastly, an illustrative example is carried out to show the prediction efficiency of the LMA. Simulations show that the LMA can give more accurate prediction than the gradient descent method.

Bifurcating fuzzy sets: Theory and application

To overcome the complexities in Type-2 Fuzzy Sets (T2FSs) and to present a general model for chaotic FSs, this paper introduces new FSs called Bifurcating FSs (BFSs). The BFS is inspired from human brain. Mathematically, it is equivalent to the coupled chaotic map in which one of the variables is a fuzzy membership function. Biologically, it is equivalent to the oscillatory neuron in which a fuzzy membership function is an activation function for the input neuron. Graphically, it is represented by an input-as-a-bifurcation-parameter diagram. This diagram demonstrates that a BFS can exhibit a wide range of FSs such as convex or non-convex, T1 or T2 and chaotic or non-chaotic FSs, and also make information processing fast. Using a BFS as an activation function in the neurons of the membership layer of a Fuzzy Neural Network (FNN), a new network called a Bifurcating FNN (BFNN) is presented. The proposed network has a similar design to an Interval T2 FNN (IT2FNN). The BFNN is applied to the problem of forecasting Mackey–Glass chaotic time series under different SNRs and different chaotic degrees. In comparison with T1FNNs and IT2FNNs, results show that BFNNs provide more accurate and robust predictions and handle more uncertainties.

Universal Approximation of a Class of Interval Type-2 Fuzzy Neural Networks in Nonlinear Identification

Neural networks (NNs), type-1 fuzzy logic systems (T1FLSs), and interval type-2 fuzzy logic systems (IT2FLSs) have been shown to be universal approximators, which means that they can approximate any nonlinear continuous function. Recent research shows that embedding an IT2FLS on an NN can be very effective for a wide number of nonlinear complex systems, especially when handling imperfect or incomplete information. In this paper we show, based on the Stone-Weierstrass theorem, that an interval type-2 fuzzy neural network (IT2FNN) is a universal approximator, which uses a set of rules and interval type-2 membership functions (IT2MFs) for this purpose. Simulation results of nonlinear function identification using the IT2FNN for one and three variables and for the Mackey-Glass chaotic time series prediction are presented to illustrate the concept of universal approximation.

RBF Network Based on Improved Niche Hybrid Hierarchy Genetic Algorithm

The improved niche hybrid hierarchy genetic algorithm is presented to overcome the premature convergence which happens in genetic algorithm constructing RBF network. The niche with poor fitness of every individual is eliminated to save system resource and raise operation speed. The simulation results demonstrate the better predicted performance on the Mackey-Glass chaotic time series than other algorithms.

Online Learning Algorithms of Direct Support Vector Machine

With the development of support vector machine, online learning became important in many areas.-Direct support vector machine (DSVM) was a new deformation algorithm for standard support vector-machine(SVM) which had fewer computation complexity for training and prediction. Based on the-mathematical mode of DSVM for classification and regression, as well as using the property of block-matrix and kernel function matrix, we deduced online learning algorithms of DSVM according to time-window theoretically. By using Mackey-Glass chaotic time series for simulation, the results show that-the algorithms are workable and predictive values are in good accordance with original values. Our-work favours SVM's applications where online learning is necessary.

Penerapan Logika Samar dalam Peramalan Data Runtun Waktu

Recently, there are so many soft computing methods been used in time series analysis. One of these methods is fuzzy logic system. In this paper, we will try to implement fuzzy logic system to predict a non-stationary time series data. The data we use here is Mackey-Glass chaotic time series. We also use MATLAB software to predict the time series data, which have been divided into four groups of input-output pairs. These groups then will be used as the input variables of the fuzzy logic system. There are two scenarios been used in this paper, first is by using seven fuzzy sets, and second is by using fifteen fuzzy sets. The result shows that the fuzzy system with fifteen fuzzy sets give a better forecasting result than the fuzzy system with seven fuzzy sets.
 Index Terms—forecasting, fuzzy logic, Mackey-Glass chaotic, MATLAB, time series analysis

An extended SHESN with leaky integrator neuron and inhibitory connection for Mackey-Glass prediction

Echo state network (ESN) proposed by Jaeger in 2001 has remarkable capabilities of approximating dynamics for complex systems, such as Mackey-Glass problem. Compared to that of ESN, the scale-free highly-clustered ESN, i.e., SHESN, which state reservoir has both small-world phenomenon and scale-free feature, exhibits even stronger approximation capabilities of dynamics and better echo state property. In this paper, we extend the state reservoir of SHESN using leaky integrator neurons and inhibitory connections, inspired from the advances in neurophysiology. We apply the extended SHESN, called e-SHESN, to the Mackey-Glass prediction problem. The experimental results show that the e-SHESN considerably outperforms the SHESN in prediction capabilities of the Mackey-Glass chaotic time-series. Meanwhile, the interesting complex network characteristic in the state reservoir, including the small-world property and the scale-free feature, remains unchanged. In addition, we unveil that the original SHESN may be unstable in some cases. However, the proposed e-SHESN model is shown to be able to address the flaw through the enhancement of the network stability. Specifically, by using the ridge regression instead of the linear regression, the stability of e-SHESN could be much more largely improved.

Extended and Unscented Kalman filtering based feedforward neural networks for time series prediction

With the ability to deal with high non-linearity, artificial neural networks (ANNs) and support vector machines (SVMs) have been widely studied and successfully applied to time series prediction. However, good fitting results of ANNs and SVMs to nonlinear models do not guarantee an equally good prediction performance. One main reason is that their dynamics and properties are changing with time, and another key problem is the inherent noise of the fitting data. Nonlinear filtering methods have some advantages such as handling additive noises and following the movement of a system when the underlying model is evolving through time. The present paper investigates time series prediction algorithms by using a combination of nonlinear filtering approaches and the feedforward neural network (FNN). The nonlinear filtering model is established by using the FNN’s weights to present state equation and the FNN’s output to present the observation equation, and the input vector to the FNN is composed of the predicted signal with given length, then the extended Kalman filtering (EKF) and Unscented Kalman filtering (UKF) are used to online train the FNN. Time series prediction results are presented by the predicted observation value of nonlinear filtering approaches. To evaluate the proposed methods, the developed techniques are applied to the predictions of one simulated Mackey–Glass chaotic time series and one real monthly mean water levels time series. Generally, the prediction accuracy of the UKF-based FNN is better than the EKF-based FNN when the model is highly nonlinear. However, comparing from prediction accuracy and computational effort based on the prediction model proposed in our study, we draw the conclusion that the EKF-based FNN is superior to the UKF-based FNN for the theoretical Mackey–Glass time series prediction and the real monthly mean water levels time series prediction.

A Study on the Prediction of the Nonlinear Chaotic Time Series Using Genetic Algorithm based Fuzzy Neural Network

In this paper we present an approach to the structure identification based on genetic algorithm and to the parameter identification by hybrid learning method in neuro-fuzzy-genetic hybrid system in order to predicate the Mackey-Glass Chaotic time series. In this scheme the basic idea consists of two steps. One is the construction of a fuzzy rule base for the partitioned input space via genetic algorithm, the other is the corresponding parameters of the fuzzy control rules adapted by the backpropagation algorithm. In an attempt to test the performance the proposed system, three patterns, x(t-3), x(t-6) and x(t-9), was prepared according to time interval. It was through lots of simulation proved that the initial small error of learning owed to the good structural identification via genetic algorithm. The performance was showed in Table 2.

Fuzzy model tuning using simulated annealing

This paper presents the use of simulated annealing metaheuristic for tuning Mamdani type fuzzy models. Structure of the Mamdani fuzzy model is learned from input–output data pairs using Wang and Mendel’s method and fuzzy c-means clustering algorithm. Then, parameters of the fuzzy system are tuned through simulated annealing. In this paper, we perform experiments to examine effects of (a) initial solution generated by Wang and Mendel’s method and fuzzy c-means clustering method, (b) membership function update procedure, (c) probability parameter for the calculation of the initial temperature, (d) temperature update coefficient used for cooling schedule, and (e) randomness level in the disturbance mechanism used in simulated annealing algorithm on the tuning of Mamdani type fuzzy models. Experiments are performed with Mackey–Glass chaotic time series. The results indicate that Wang and Mendel’s method provides better starting configuration for simulated annealing compared to fuzzy c-means clustering method, and for the membership function update parameter, MFChangeRate ∈ (0, 1], and the probability parameter for the calculation of the initial temperature, P 0 ∈ (0, 1), values close to zero produced better results.

Fuzzy transforms method in prediction data analysis

We present a prediction method based on Fuzzy transforms for forecasting problems and we compare it with the well known Wang–Mendel's one. Another comparison is made on the Local Linear Wavelet Neural Network method via forecasting time series. We apply these concepts to a Mackey–Glass chaotic time series and to the monthly NAO climatic index time series.

Multi groups cooperation based symbiotic evolution for TSK-type neuro-fuzzy systems design

In this paper, a TSK-type neuro-fuzzy system with multi groups cooperation based symbiotic evolution method (TNFS-MGCSE) is proposed. The TNFS-MGCSE is developed from symbiotic evolution. The symbiotic evolution is different from traditional GAs (genetic algorithms) that each chromosome in symbiotic evolution represents a rule of fuzzy model. The MGCSE is different from the traditional symbiotic evolution; with a population in MGCSE is divided into several groups. Each group formed by a set of chromosomes represents a fuzzy rule and cooperate with other groups to generate the better chromosomes by using the proposed cooperation based crossover strategy (CCS). In this paper, the proposed TNFS-MGCSE is used to evaluate by numerical examples (Mackey-Glass chaotic time series and sunspot number forecasting). The performance of the TNFS-MGCSE achieves excellently with other existing models in the simulations.

Design of Radial Basis Function Network Using Adaptive Particle Swarm Optimization and Orthogonal Least Squares

This paper presents a two-level learning method for designing an optimal Radial Basis Function Network (RBFN) using Adaptive Velocity Update Relaxation Particle Swarm Optimization algorithm (AVURPSO) and Orthogonal Least Squares algorithm (OLS) called as OLS-AVURPSO method. The novelty is to develop an AVURPSO algorithm to form the hybrid OLS-AVURPSO method for designing an optimal RBFN. The proposed method at the upper level finds the global optimum of the spread factor parameter using AVURPSO while at the lower level automatically constructs the RBFN using OLS algorithm. Simulation results confirm that the RBFN is superior to Multilayered Perceptron Network (MLPN) in terms of network size and computing time. To demonstrate the effectiveness of proposed OLS-AVURPSO in the design of RBFN, the Mackey-Glass Chaotic Time-Series as an example is modeled by both MLPN and RBFN.

FASTER TRAINING USING FUSION OF ACTIVATION FUNCTIONS FOR FEED FORWARD NEURAL NETWORKS

Multilayer feed-forward neural networks are widely used based on minimization of an error function. Back propagation (BP) is a famous training method used in the multilayer networks but it often suffers from the drawback of slow convergence. To make the learning faster, we propose 'Fusion of Activation Functions' (FAF) in which different conventional activation functions (AFs) are combined to compute final activation. This has not been studied extensively yet. One of the sub goals of the paper is to check the role of linear AFs in combination. We investigate whether FAF can enable the learning to be faster. Validity of the proposed method is examined by performing simulations on challenging nine real benchmark classification and time series prediction problems. The FAF has been applied to 2-bit, 3-bit and 4-bit parity, the breast cancer, Diabetes, Heart disease, Iris, wine, Glass and Soybean classification problems. The algorithm is also tested with Mackey-Glass chaotic time series prediction problem. The algorithm is shown to work better than other AFs used independently in BP such as sigmoid (SIG), arctangent (ATAN), logarithmic (LOG).

Evolutionary artificial neural networks for hydrological systems forecasting

The conventional ways of constructing artificial neural network (ANN) for a problem generally presume a specific architecture and do not automatically discover network modules appropriate for specific training data. Evolutionary algorithms are used to automatically adapt the network architecture and connection weights according to the problem environment without substantial human intervention. To improve on the drawbacks of the conventional optimal process, this study presents a novel evolutionary artificial neural network (EANN) for time series forecasting. The EANN has a hybrid procedure, including the genetic algorithm and the scaled conjugate gradient algorithm, where the feedforward ANN architecture and its connection weights of neurons are simultaneously identified and optimized. We first explored the performance of the proposed EANN for the Mackey–Glass chaotic time series. The performance of the different networks was evaluated. The excellent performance in forecasting of the chaotic series shows that the proposed algorithm concurrently possesses efficiency, effectiveness, and robustness. We further explored the applicability and reliability of the EANN in a real hydrological time series. Again, the results indicate the EANN can effectively and efficiently construct a viable forecast module for the 10-day reservoir inflow, and its accuracy is superior to that of the AR and ARMAX models.

Novel FTLRNN with Gamma Memory for Short-Term and Long-Term Predictions of Chaotic Time Series

Multistep ahead prediction of a chaotic time series is a difficult task that has attracted increasing interest in the recent years. The interest in this work is the development of nonlinear neural network models for the purpose of building multistep chaotic time series prediction. In the literature there is a wide range of different approaches but their success depends on the predicting performance of the individual methods. Also the most popular neural models are based on the statistical and traditional feed forward neural networks. But it is seen that this kind of neural model may present some disadvantages when long-term prediction is required. In this paper focused time-lagged recurrent neural network (FTLRNN) model with gamma memory is developed for different prediction horizons. It is observed that this predictor performs remarkably well for short-term predictions as well as medium-term predictions. For coupled partial differential equations generated chaotic time series such as Mackey Glass and Duffing, FTLRNN-based predictor performs consistently well for different depths of predictions ranging from short term to long term, with only slight deterioration after k is increased beyond 50. For real-world highly complex and nonstationary time series like Sunspots and Laser, though the proposed predictor does perform reasonably for short term and medium-term predictions, its prediction ability drops for long term ahead prediction. However, still this is the best possible prediction results considering the facts that these are nonstationary time series. As a matter of fact, no other NN configuration can match the performance of FTLRNN model. The authors experimented the performance of this FTLRNN model on predicting the dynamic behavior of typical Chaotic Mackey-Glass time series, Duffing time series, and two real-time chaotic time series such as monthly sunspots and laser. Static multi layer perceptron (MLP) model is also attempted and compared against the proposed model on the performance measures like mean squared error (MSE), Normalized mean squared error (NMSE), and Correlation Coefficient (r). The standard back-propagation algorithm with momentum term has been used for both the models.

On entropy, financial markets and minority games

The paper builds upon an earlier statistical analysis of financial time series with Shannon information entropy, published in [L. Molgedey, W. Ebeling, Local order, entropy and predictability of financial time series, European Physical Journal B—Condensed Matter and Complex Systems 15/4 (2000) 733–737]. A novel generic procedure is proposed for making multistep-ahead predictions of time series by building a statistical model of entropy. The approach is first demonstrated on the chaotic Mackey–Glass time series and later applied to Japanese Yen/US dollar intraday currency data. The paper also reinterprets Minority Games [E. Moro, The minority game: An introductory guide, Advances in Condensed Matter and Statistical Physics (2004)] within the context of physical entropy, and uses models derived from minority game theory as a tool for measuring the entropy of a model in response to time series. This entropy conditional upon a model is subsequently used in place of information-theoretic entropy in the proposed multistep prediction algorithm.

A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks

In real life, information about the world is uncertain and imprecise. The cause of this uncertainty is due to: deficiencies on given information, the fuzzy nature of our perception of events and objects, and on the limitations of the models we use to explain the world. The development of new methods for dealing with information with uncertainty is crucial for solving real life problems. In this paper three interval type-2 fuzzy neural network (IT2FNN) architectures are proposed, with hybrid learning algorithm techniques (gradient descent backpropagation and gradient descent with adaptive learning rate backpropagation). At the antecedents layer, a interval type-2 fuzzy neuron (IT2FN) model is used, and in case of the consequents layer an interval type-1 fuzzy neuron model (IT1FN), in order to fuzzify the rule’s antecedents and consequents of an interval type-2 Takagi–Sugeno–Kang fuzzy inference system (IT2-TSK-FIS). IT2-TSK-FIS is integrated in an adaptive neural network, in order to take advantage the best of both models. This provides a high order intuitive mechanism for representing imperfect information by means of use of fuzzy If–Then rules, in addition to handling uncertainty and imprecision. On the other hand, neural networks are highly adaptable, with learning and generalization capabilities. Experimental results are divided in two kinds: in the first one a non-linear identification problem for control systems is simulated, here a comparative analysis of learning architectures IT2FNN and ANFIS is done. For the second kind, a non-linear Mackey–Glass chaotic time series prediction problem with uncertainty sources is studied. Finally, IT2FNN proved to be more efficient mechanism for modeling real-world problems.

Chaotic Time Series Prediction Using a Neuro-Fuzzy System with Time-Delay Coordinates

This paper presents an investigation into the use of the delay coordinate embedding technique in the multi-input- multioutput-adaptive-network-based fuzzy inference system (MANFIS) for chaotic time series prediction. The inputs to the MANFIS are embedded-phase-space (EPS) vectors preprocessed from the time series under test, while the output time series is extracted from the output EPS vectors from the MANFIS. A moving root-mean-square error is used to monitor the error over the prediction horizon and to tune the membership functions in the MANFIS. With the inclusion of the EPS preprocessing step, the prediction performance of the MANFIS is improved significantly. The proposed method has been tested with one periodic function and two chaotic functions including Mackey-Glass chaotic time series and Duffing forced-oscillation system. The prediction performances with and without EPS preprocessing are statistically compared by using the t-test method. The results show that EPS preprocessing can help improve the prediction performance of a MANFIS significantly.