Mackey-Glass Chaotic Time Series Research Articles

Complex-Valued Ordinary Differential Equation Modeling for Time Series Identification

Time series identification is one of the key approaches to dealing with time series data and discovering the change rules. Therefore, time series forecasting can be treated as one of the most challenging issues in this field. In order to improve the forecasting performance, we propose a novel time series prediction model based on a complex-valued ordinary differential equation (CVODE) to predict time series. A multi expression programming (MEP) algorithm is utilized to optimize the structure of the CVODE model. So as to achieve the optimal complex-valued coefficients, a novel optimization algorithm based on a complex-valued crow search algorithm (CVCSA) is proposed. The chaotic Mackey-Glass time series, small-time scale traffic measurements, Nasdaq-100 index, and Shanghai stock exchange composite index are utilized to evaluate the performance of our method. The results prove that our proposed method could predict more accurately than state-of-the-art real-valued neural networks and an ordinary differential equation (ODE). The CVCSA has faster convergence speed and stronger optimization ability than the crow search algorithm (CSA) and particle swarm optimization (PSO).

Accurate fuzzy predictive models through complexity reduction based on decision of needed fuzzy rules

Abstract Fuzzy predictive models (FPMs) are employed in a wide variety of domains for their high prediction accuracy and good interpretability. For a FPM, the fuzzy rules from data or expert experience are always firstly extracted to establish a relatively complete fuzzy rule base. However, it is found that a large amount of extracted fuzzy rules which are always not used for prediction increase the complexity of the FPM. Moreover, inaccurate fuzzy rules may be used for prediction to degrade the accuracy of the FPM. Therefore, we proposed a method to establish an accurate FPM through complexity reduction based on decision of needed fuzzy rules. To avoid extractions of the rules which are not needed for prediction, reduce the complexity of the FPM, and suppress the influence of some inaccurate fuzzy rules, we add the decision of fuzzy rules needed for prediction based on membership values to predictive modeling before the extraction of fuzzy rules. And then an improved WM method based on optimization of centers of output fuzzy subsets for fuzzy rules (COiWM), which was proven to possess high completeness, robustness, and accuracy in our previous work, is used to generate the needed fuzzy rules directly from historical data. The rules which can not to be generated are extrapolated based on these data-generated fuzzy rules. At last, two fuzzy rule bases are built and used together to fuzzy predictive modeling. Experimental results of a case study on short term daily maximum electric load forecasting and Mackey–Glass chaotic time-series prediction show that our proposed method reduces the complexity of the FPMs and enhances the prediction accuracy.

A Beta basis function Interval Type-2 Fuzzy Neural Network for time series applications

The huge complexity and uncertainty in real life requires the use of advanced automatic learning methods to find out better approximators and suitable relationship in real data behavior. Neuro fuzzy systems have been proved to be excellent universal approximators. In this paper we propose a new based function Interval Type-2 Fuzzy Neural Network denoted ”Beta basis function Interval Type-2 Fuzzy Neural Network”, the BIT2FNN. The main idea is to involve type-2 beta fuzzy sets in the design process of fuzzy networks. The proposed architecture is based on beta type-2 fuzzy sets in the antecedent part, while the consequent part achieves the TSK (Takagi–Sugeno–Kang) fuzzy output strategy. Thanks to the beta function flexibility, the network achieve a good performance and shows a good resistance to noisy data. First order derivatives of type-1 and type-2 Beta functions were developed for the first time for designing fuzzy logic systems based on given input–output pairs. The backpropagation algorithm was used for the learning process of antecedent fuzzy beta parameters and the consequent part. The performance of the proposed model of Beta fuzzy logic system is evaluated with mainly two problems of time series applications : the Mackey Glass Chaotic Time-Series prediction problem with different setting of parameters and levels of noise and the ECG heart-rate Time Series monitoring problem.

Prediction of NOx emission of a power plant boiler based on adaptive simplified T-S model

The combustion process of power plant boiler often has strong nonlinearity, uncertainty and time-varying nature. Offline models are often difficult to predict the emission of NOx accurately. In this paper, online identification method based on simplified T-S model for prediction of NOx emission is proposed. Firstly, the online subtractive clustering algorithm is used to determine the number of clusters and the clustering center adaptively. After obtaining the parameters, the recursive least squares algorithm is used to identify the parameters of each local model. Finally, the Mackey-Glass chaotic time series and field data of NOx emission of a power plant boiler are predicted by the above algorithm. The results show the effectiveness of the algorithm.

The combination of circle topology and leaky integrator neurons remarkably improves the performance of echo state network on time series prediction.

Recently, echo state network (ESN) has attracted a great deal of attention due to its high accuracy and efficient learning performance. Compared with the traditional random structure and classical sigmoid units, simple circle topology and leaky integrator neurons have more advantages on reservoir computing of ESN. In this paper, we propose a new model of ESN with both circle reservoir structure and leaky integrator units. By comparing the prediction capability on Mackey-Glass chaotic time series of four ESN models: classical ESN, circle ESN, traditional leaky integrator ESN, circle leaky integrator ESN, we find that our circle leaky integrator ESN shows significantly better performance than other ESNs with roughly 2 orders of magnitude reduction of the predictive error. Moreover, this model has stronger ability to approximate nonlinear dynamics and resist noise than conventional ESN and ESN with only simple circle structure or leaky integrator neurons. Our results show that the combination of circle topology and leaky integrator neurons can remarkably increase dynamical diversity and meanwhile decrease the correlation of reservoir states, which contribute to the significant improvement of computational performance of Echo state network on time series prediction.

A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks

In this research, we propose a novel algorithm for learning of the recurrent neural networks called as the fractional back-propagation through time (FBPTT). Considering the potential of the fractional calculus, we propose to use the fractional calculus-based gradient descent method to derive the FBPTT algorithm. The proposed FBPTT method is shown to outperform the conventional back-propagation through time algorithm on three major problems of estimation namely nonlinear system identification, pattern classification and Mackey–Glass chaotic time series prediction.

Long-term forecasting of time series based on linear fuzzy information granules and fuzzy inference system

Long-term time series forecasting is a challenging problem both in theory and in practice. Although the idea of information granulation has been shown to be an essential concept and algorithmic pursuit in time series prediction, there is still an acute need for developing a sound conceptual framework for time series prediction so that information granulation can capture the essence of collections of data better, including average and trend information. In this paper, a novel type of fuzzy information granule involving a time-dependent (non-stationary) membership function is proposed to structure numerical time series into granular time series. We show that the underlying arithmetic along with the concept of distance for this type of information granules can be expressed in a simple way, which facilitates the ensuing processing of information granules. With this regard, distances between observation granules and antecedent granules presented in fuzzy rules can be easily determined. The design of long-term prediction method based on fuzzy inference system is then realized through interpolation completed with the aid of fuzzy rules. Experiments involving chaotic Mackey–Glass time series and real-world time series demonstrate that the proposed model produces better long-term forecasting than some existing numeric models such as Autoregressive (AR) models, nonlinear autoregressive (NAR) neural networks, Support Vector Regression (SVR) and fuzzy inference systems involving triangular and interval information granules.

Distributed Reservoir Computing with Sparse Readouts [Research Frontier

In a network of agents, a widespread problem is the need to estimate a common underlying function starting from locally distributed measurements. Real-world scenarios may not allow the presence of centralized fusion centers, requiring the development of distributed, message-passing implementations of the standard machine learning training algorithms. In this paper, we are concerned with the distributed training of a particular class of recurrent neural networks, namely echo state networks (ESNs). In the centralized case, ESNs have received considerable attention, due to the fact that they can be trained with standard linear regression routines. Based on this observation, in our previous work we have introduced a decentralized algorithm, framed in the distributed optimization field, in order to train an ESN. In this paper, we focus on an additional sparsity property of the output layer of ESNs, allowing for very efficient implementations of the resulting networks. In order to evaluate the proposed algorithm, we test it on two well-known prediction benchmarks, namely the Mackey-Glass chaotic time series and the 10th order nonlinear auto regressive moving average (NARMA) system.

Novel continuous function prediction model using an improved Takagi–Sugeno fuzzy rule and its application based on chaotic time series

A novel continuous function prediction model (CFPM) is proposed to resolve prediction problem whose input and output are both continuous functions (CFs). CFPM can simplify sample space reconstruction by using the coefficients of CFs, and use an improved Takagi–Sugeno (TS) fuzzy rule to predict output CF by optimizing the tendency of input CFs. The improved TS fuzzy rule handles each input CF as a consequent parameter and can obtain the nonlinear tendency. After learning process by using opinion-leader-based particle swarm optimization, output CF is determined. In the data prediction based on chaotic time series, CF can either be obtained directly or be fitted by discrete data points, thus the prediction range is enlarged because more discrete data points can be generated once output CF is determined.Two experiments and three cases based on chaotic time series are performed to validate CFPM. The Mackey–Glass chaotic time series is used to prove CFPM validation, while the NN3 time series is used to evaluate CFPM performance. The cases on exhaust gas temperature (EGT), EGT margin and delta EGT are used to show that CFPM is valuable in health status prediction for a particular aircraft engine in the practical engineering field.

Mackey-Glass noisy chaotic time series prediction by a swarm-optimized neural network

In this study, an artificial neural network (ANN) based on particle swarm optimization (PSO) was developed for the time series prediction. The hybrid ANN+PSO algorithm was applied on Mackey-Glass noiseless chaotic time series in the short-term and long-term prediction. The performance prediction is evaluated and compared with similar work in the literature, particularly for the long-term forecast. Also, we present properties of the dynamical system via the study of chaotic behaviour obtained from the time series prediction.Then, this standard hybrid ANN+PSO algorithm was complemented with a Gaussian stochastic procedure (called stochastic hybrid ANN+PSO) in order to obtain a new estimator of the predictions that also allowed us compute uncertainties of predictions for noisy Mackey-Glass chaotic time series. We study the impact of noise for three cases with a white noise level (σN) contribution of 0.01, 0.05 and 0.1.

A systematic design of interval type-2 fuzzy logic system using extreme learning machine for electricity load demand forecasting

This paper presents a novel design of interval type-2 fuzzy logic systems (IT2FLS) by utilizing the theory of extreme learning machine (ELM) for electricity load demand forecasting. ELM has become a popular learning algorithm for single hidden layer feed-forward neural networks (SLFN). From the functional equivalence between the SLFN and fuzzy inference system, a hybrid of fuzzy-ELM has gained attention of the researchers. This paper extends the concept of fuzzy-ELM to an IT2FLS based on ELM (IT2FELM). In the proposed design the antecedent membership function parameters of the IT2FLS are generated randomly, whereas the consequent part parameters are determined analytically by the Moore–Penrose pseudo inverse. The ELM strategy ensures fast learning of the IT2FLS as well as optimality of the parameters. Effectiveness of the proposed design of IT2FLS is demonstrated with the application of forecasting nonlinear and chaotic data sets. Nonlinear data of electricity load from the Australian National Electricity Market for the Victoria region and from the Ontario Electricity Market are considered here. The proposed model is also applied to forecast Mackey-glass chaotic time series data. Comparative analysis of the proposed model is conducted with some traditional models such as neural networks (NN) and adaptive neuro fuzzy inference system (ANFIS). In order to verify the structure of the proposed design of IT2FLS an alternate design of IT2FLS based on Kalman filter (KF) is also utilized for the comparison purposes.

Calibrating wavelet neural networks by distance orientation similarity fuzzy C-means for approximation problems

Improperly tuned wavelet neural network (WNN) has been shown to exhibit unsatisfactory generalization performance. In this study, the tuning is done by an improved fuzzy C-means algorithm, that utilizes a novel similarity measure. This similarity measure takes the orientation as well as the distance into account. The modified WNN was first applied to a benchmark problem. Performance assessments with other approaches were made subsequently. Next, the feasibility of the proposed WNN in forecasting the chaotic Mackey–Glass time series and a real world application problem, i.e., blood glucose level prediction, were studied. An assessment analysis demonstrated that this presented WNN was superior in terms of prediction accuracy.

Wind power time series prediction using optimized kernel extreme learning machine method

Since wind has an intrinsically complex and stochastic nature, accurate wind power prediction is necessary for the safety and economics of wind energy utilization. Aiming at the prediction of very short-term wind power time series, a new optimized kernel extreme learning machine (O-KELM) method with evolutionary computation strategy is proposed on the basis of single-hidden layer feedforward neural networks. In comparison to the extreme learning machine (ELM) method, the number of the hidden layer nodes need not be given, and the unknown nonlinear feature mapping of the hidden layer is represented with a kernel function. In addition, the output weights of the networks can also be analytically determined by using regularization least square algorithm, hence the kernel extreme learning machine (KELM) method provides better generalization performance at a much faster learning speed. In the O-KELM, the structure and the parameters of the KELM are optimized by using three different optimization algorithms, i.e., genetic algorithm (GA), differential evolution (DE), and simulated annerling (SA), meanwhile, the output weights are obtained by a least squares algorithm just the same as by the ELM, but using Tikhonovs regularization in order to further improve the performance of the O-KELM. The utilized optimization algorithms of the O-KELM are respectively used to select the set of input variables, regularization coefficient as well as hyperparameter of kernel function. The proposed method is first applied to the direct six-step prediction for Mackey-Glass chaotic time series, under the same condition as the existing optimized ELM method. From the analysis of the simulation results it can be verified that the prediction accuracy of the proposed O-KELM method is increased by about one order of magnitude over that of the optimized ELM method. Furthermore, the DE-KELM algorithm can achieve the lowest root mean square error (RMSE). The O-KELM method is then applied to real-world wind power prediction instance, i.e., the Western Dataset from NERL. The 10-minute ahead single-step prediction as well as 20-minute ahead, 30-minute ahead, 40-minute ahead multi-step prediction for wind power time series are respectively implemented to evaluate the O-KELM method. Experimental results of each of the short-term wind power time series predictions at different time horizons confirm that the proposed O-KELM method tends to have better prediction accuracy than the optimized ELM method. Moreover, the GA-KELM algorithm outperforms other two O-KELM algorithms at future 10-minute, 20-minute, 40-minute ahead prediction in terms of the RMSE value. The DE-KELM algorithm outperforms other algorithms at future 30-minute ahead prediction in terms of the normalized mean square error (NMSE) and the RMSE value. The results from these applications demonstrate the effectiveness and feasibility of the proposed O-KLEM method. Therefore, the O-KELM method has a potential future in the field of wind power prediction.

Constructing optimized interval type-2 TSK neuro-fuzzy systems with noise reduction property by quantum inspired BFA

In this paper, a modified rule generation approach with self-constructing property for neuro-fuzzy system modelling is proposed. In structure identification stage, input–output patterns are divided into the clusters and interval type-2 membership functions are generated roughly. Interval type-2 Takagi–Sugeno–Kang (TSK) neuro-fuzzy structure is fine tuned by quantum inspired bacterial foraging algorithm (QBFA) in parameter identification stage to achieve higher precision, a recursive least squares (RLS) estimator is used to update consequent parameters. Comparisons with two type-1 neuro-fuzzy systems on three nonlinear functions and chaotic Mackey-Glass time series show that the proposed systems can approximate the target with little error. Experiments are also executed involving the proposed systems for modelling flue gas denitrification efficiency of a thermal power plant. It is verified by the results that interval type-2 neuro-fuzzy structure can learn knowledge from input–output data set with the aid of QBFA and hybrid training progresses are able to improve its performance.

Weighted fuzzy interpolative reasoning for sparse fuzzy rule-based systems based on piecewise fuzzy entropies of fuzzy sets

In this paper, we propose a new method for weighted fuzzy interpolative reasoning in sparse fuzzy rule-based systems based on piecewise fuzzy entropies of fuzzy sets. First, the proposed method uses the representative values of antecedent fuzzy sets, the representative values of observation fuzzy sets, and the representative values of consequence fuzzy sets of fuzzy rules to get the characteristic points of the fuzzy interpolative result represented by a fuzzy set. Then, it calculates the piecewise fuzzy entropies between any two characteristic points of the antecedent fuzzy sets, the piecewise fuzzy entropies between any two characteristic points of the observation fuzzy sets, and the piecewise fuzzy entropies between any two characteristic points of the consequence fuzzy sets of the fuzzy rules, respectively. Then, it calculates the weights of the antecedent fuzzy sets of each fuzzy rule, respectively, and calculates the weight of each fuzzy rule. Then, it calculates the piecewise fuzzy entropies between any two characteristic points of the fuzzy interpolative result. Finally, it uses the secant method to calculate the degree of membership of each obtained characteristic point of the fuzzy interpolative result. The experimental results show that the proposed method outperforms the existing methods for dealing with the multivariate regression problems, the Mackey–Glass chaotic time series prediction problem, and the time series prediction problems.

Gradient Radial Basis Function Based Varying-Coefficient Autoregressive Model for Nonlinear and Nonstationary Time Series

We propose a gradient radial basis function based varying-coefficient autoregressive (GRBF-AR) model for modeling and predicting time series that exhibit nonlinearity and homogeneous nonstationarity. This GRBF-AR model is a synthesis of the gradient RBF and the functional-coefficient autoregressive (FAR) model. The gradient RBFs, which react to the gradient of the series, are used to construct varying coefficients of the FAR model. The Mackey-Glass chaotic time series are used to evaluate the performance of the proposed method. It is shown that the GRBF-AR model not only achieves much more parsimonious structure but also much better prediction performance than that of GRBF network.

A priori data-driven multi-clustered reservoir generation algorithm for echo state network.

Echo state networks (ESNs) with multi-clustered reservoir topology perform better in reservoir computing and robustness than those with random reservoir topology. However, these ESNs have a complex reservoir topology, which leads to difficulties in reservoir generation. This study focuses on the reservoir generation problem when ESN is used in environments with sufficient priori data available. Accordingly, a priori data-driven multi-cluster reservoir generation algorithm is proposed. The priori data in the proposed algorithm are used to evaluate reservoirs by calculating the precision and standard deviation of ESNs. The reservoirs are produced using the clustering method; only the reservoir with a better evaluation performance takes the place of a previous one. The final reservoir is obtained when its evaluation score reaches the preset requirement. The prediction experiment results obtained using the Mackey-Glass chaotic time series show that the proposed reservoir generation algorithm provides ESNs with extra prediction precision and increases the structure complexity of the network. Further experiments also reveal the appropriate values of the number of clusters and time window size to obtain optimal performance. The information entropy of the reservoir reaches the maximum when ESN gains the greatest precision.

Time series long-term forecasting model based on information granules and fuzzy clustering

In spite of the impressive diversity of models of time series, there is still an acute need to develop constructs that are both accurate and transparent. Meanwhile, long-term time series prediction is challenging and of great interest to both practitioners and research community. The role of information granulation is to organize detailed numerical data into some meaningful, semantically sound entities. With this regard, the design of time series forecasting models used the information granulation is interpretable and easily comprehended by humans. In order to cluster information granules, a modified fuzzy c-means which does not require that data have the same dimensionality is proposed. Then, we develop forecasting model combining the modified fuzzy c-means and information granulation for solving the problem of time series long-term prediction. Synthetic time series, chaotic Mackey–Glass time series, power demand, daily temperatures, stock index, and wind speed are used in a series of experiments. The experimental results show that the proposed model produces better forecasting results than several existing models.

Adaptive natural gradient learning algorithms for Mackey–Glass chaotic time prediction

In this paper, we introduce the adaptive natural gradient method into the multilayer perceptrons (MLPs) and radial basis function (RBF) networks. We give a good performance for the Mackey–Glass chaotic time series prediction, and compare it with the LMA. Results show that the adaptive natural gradient methods for MLPs and RBFs, which are the online learning, can give almost the same performance with the LMA (MLPs), which is the batch mode learning. However, the performance of LMA (RBFs) is very poor and is very sensitive with the initial parameters.

Impact of Noise on a Dynamical System: Prediction and Uncertainties from a Swarm-Optimized Neural Network.

An artificial neural network (ANN) based on particle swarm optimization (PSO) was developed for the time series prediction. The hybrid ANN+PSO algorithm was applied on Mackey-Glass chaotic time series in the short-term x(t + 6). The performance prediction was evaluated and compared with other studies available in the literature. Also, we presented properties of the dynamical system via the study of chaotic behaviour obtained from the predicted time series. Next, the hybrid ANN+PSO algorithm was complemented with a Gaussian stochastic procedure (called stochastic hybrid ANN+PSO) in order to obtain a new estimator of the predictions, which also allowed us to compute the uncertainties of predictions for noisy Mackey-Glass chaotic time series. Thus, we studied the impact of noise for several cases with a white noise level (σ N) from 0.01 to 0.1.