Mackey-Glass Time Series Research Articles

Incremental Fuzzy C-Regression Clustering From Streaming Data for Local-Model-Network Identification

In this paper, a new approach to evolving fuzzy model identification from streaming data is given. The structure of the model is given as a local model network in Takagi–Sugeno form, and the partitioning of the input–output space is based on metrics in which these local models are defined as prototypes of the clusters. This means that the clusters and the local models share the same parameters; therefore, the number of parameters of the evolving system is much lower in comparison to similar systems of comparable complexity, and the problems of parameter identifiability are not a particular issue. The algorithm adds the local models in an incremental fashion and recursively adapts the local model parameters. The proposed algorithm is tested on three examples to demonstrate the main features. The first example is a simple simulated example with intersecting clusters; the second is a very well-known benchmark that treats the Mackey–Glass time series; the third is an example that shows the classification of the data from a laser rangefinder. These examples show the great potential of the proposed approach in certain applications.

Gray Wolf Optimization and Least Square Estimatation As A New Learning Algorithm For Interval Type-II ANFIS

Gray Wolfe Optimization (GWO) is one of the meta-heuristic method and it is a popular technique in Many engineering and economic applications. GWO and Least Square Estimatation (LSE) are used to optimize the antecedents and consequents parameters of interval type-2 ANFIS respectively. We are checking the new learning algorithm by using the interval type-2 ANFIS in prediction of Mackey-Glass time series and the results were very encouraging compared to other algorithms.

Long-term prediction of time series based on stepwise linear division algorithm and time-variant zonary fuzzy information granules

In the study of the long-term prediction of time series with high noise and nonlinearity, the trend information and fluctuation range of the sequence data is usually more valuable and practical than the forecasting of the specific values at time points. A method named as stepwise linear division (SLD) is constructed for the variable-length partition of fuzzy information granule (FIG), characteristics of which are extracted from the original data distribution of time series. Secondly, based on the above partition algorithm, a novel fuzzy information granule is first proposed, which can characterize the variant trend of the data, the range of fluctuation and the degree of dispersion. Moreover, the reliability of the prediction results can be quantified. After granulating time series, a rule-based fuzzy inference system is established to achieve the prediction of time series. Synthetic sequences and real-life time series, including chaotic Mackey–Glass time series, temperature, sunspot numbers, milk production and financial data are utilized in experiments to verify the effectiveness of the proposed scheme. The experimental results demonstrate that the proposed model can contain more rich semantic information and produce better long-term prediction performance compared with existing numeric models and fuzzy inference systems.

Design of type-2 Fuzzy Logic Systems Based on Improved Ant Colony Optimization

An Improved Ant Colony Optimization (IACO) is proposed to design A2-C1 type fuzzy logic system (FLS) in the paper. The design includes parameters adjustment and rules selection, and the performance of the intelligent fuzzy system, which can be improved by choosing the most optimal parameters and reducing the redundant rules. In order to verify the feasibility of the proposed algorithm, the intelligence fuzzy logic systems based on the algorithms are applied to predict the Mackey-Glass chaos time series. The simulations show that both the IACO and ACO have better tracking performances. The results compared with classical algorithm BP ( back-propagation design) shows the tracking performance of IACO is more precise, the result compared with ACO shows that either the training result or the testing result, the tracking performance of IACO is better, and IACO has a faster convergence rate than ACO, the results compared with the Intelligent type-1 fuzzy logic systems show that both the A2-C1 type FLS based on IACO and ACO have better tracking performance than type-1 fuzzy logic system.

Complex-Valued Ordinary Differential Equation Modeling for Time Series Identification

Time series identification is one of the key approaches to dealing with time series data and discovering the change rules. Therefore, time series forecasting can be treated as one of the most challenging issues in this field. In order to improve the forecasting performance, we propose a novel time series prediction model based on a complex-valued ordinary differential equation (CVODE) to predict time series. A multi expression programming (MEP) algorithm is utilized to optimize the structure of the CVODE model. So as to achieve the optimal complex-valued coefficients, a novel optimization algorithm based on a complex-valued crow search algorithm (CVCSA) is proposed. The chaotic Mackey-Glass time series, small-time scale traffic measurements, Nasdaq-100 index, and Shanghai stock exchange composite index are utilized to evaluate the performance of our method. The results prove that our proposed method could predict more accurately than state-of-the-art real-valued neural networks and an ordinary differential equation (ODE). The CVCSA has faster convergence speed and stronger optimization ability than the crow search algorithm (CSA) and particle swarm optimization (PSO).

Deep learning algorithm for data-driven simulation of noisy dynamical system

We present a deep learning model, DE-LSTM, for the simulation of a stochastic process with an underlying nonlinear dynamics. The deep learning model aims to approximate the probability density function of a stochastic process via numerical discretization and the underlying nonlinear dynamics is modeled by the Long Short-Term Memory (LSTM) network. It is shown that, when the numerical discretization is used, the function estimation problem can be solved by a multi-label classification problem. A penalized maximum log likelihood method is proposed to impose a smoothness condition in the prediction of the probability distribution. We show that the time evolution of the probability distribution can be computed by a high-dimensional integration of the transition probability of the LSTM internal states. A Monte Carlo algorithm to approximate the high-dimensional integration is outlined. The behavior of DE-LSTM is thoroughly investigated by using the Ornstein–Uhlenbeck process and noisy observations of nonlinear dynamical systems; Mackey–Glass time series and forced Van der Pol oscillator. It is shown that DE-LSTM makes a good prediction of the probability distribution without assuming any distributional properties of the stochastic process. For a multiple-step forecast of the Mackey–Glass time series, the prediction uncertainty, denoted by the 95% confidence interval, first grows, then dynamically adjusts following the evolution of the system, while in the simulation of the forced Van der Pol oscillator, the prediction uncertainty does not grow in time even for a 3,000-step forecast.

Recursive Particle Filter-Based RBF Network on Time Series Prediction of Measurement Data

Most physical phenomena in nature are chaotic or close to chaos. An irregular time series can be generated or measured with a purely deterministic equation of motion in nonlinear and chaotic systems. This paper presents an adaptive state-space particle filtering (PF)-based trained radial basis function (RBF) network for chaotic and nonstationary observation–prediction. The recursive Bayesian filtering algorithm, which uses the particle representation of density function, is adopted to accomplish nonlinear and non-Gaussian state estimation and achieve improved convergence rate and quality of solution. Four sampling importance resampling approaches, namely, multinomial, systematic, stratified, and residual resampling methods, are considered to resolve weight degeneracy. The effectiveness of our proposed methods is investigated using two chaotic time series and three measurement datasets, including the Mackey–Glass time series, Rossler time series, monthly Lake Erie levels, monthly water usage series, and SML2010 data set. The performances are evaluated through an extensive simulation by computing the average mean square error, mean absolute percentage error, and average relative variance metrics. Simulation results show that the proposed PF-based RBF structure can provide more effective and accurate prediction performances compared with the conventional gradient descent, extended Kalman filter (EKF), and decoupled EKF algorithms (DEKF).

Stability of Evolving Fuzzy Systems Based on Data Clouds

Evolving fuzzy systems (EFSs) are now well developed and widely used, thanks to their ability to self-adapt both their structures and parameters online. Since the concept was first introduced two decades ago, many different types of EFSs have been successfully implemented. However, there are only very few works considering the stability of the EFSs, and these studies were limited to certain types of membership functions with specifically predefined parameters, which largely increases the complexity of the learning process. At the same time, stability analysis is of paramount importance for control applications and provides the theoretical guarantees for the convergence of the learning algorithms. In this paper, we introduce the stability proof of a class of EFSs based on data clouds, which are grounded at the AnYa type fuzzy systems and the recently introduced empirical data analytics (EDA) methodological framework. By employing data clouds, the class of EFSs of AnYa type considered in this paper avoids the traditional way of defining membership functions for each input variable in an explicit manner and its learning process is entirely data driven. The stability of the considered EFS of AnYa type is proven through the Lyapunov theory, and the proof of stability shows that the average identification error converges to a small neighborhood of zero. Although, the stability proof presented in this paper is specially elaborated for the considered EFS, it is also applicable to general EFSs. The proposed method is illustrated with Box–Jenkins gas furnace problem, one nonlinear system identification problem, Mackey–Glass time series prediction problem, eight real-world benchmark regression problems as well as a high-frequency trading prediction problem. Compared with other EFSs, the numerical examples show that the considered EFS in this paper provides guaranteed stability as well as a better approximation accuracy.

Robust Adaptive Filtering With $q$ -Gaussian Kernel Mean $p$-Power Error

In this letter, a novel information theoretic measure, namely $\boldsymbol {q}$ -Gaussian kernel mean $\boldsymbol {p}$ -power error (QKMPE), is proposed by defining the mean $\boldsymbol {p}$ -power error in the $\boldsymbol {q}$ -Gaussian kernel space, which is a generalization of the kernel mean $\boldsymbol {p}$ -power error measure. Furthermore, a recursive kernel adaptive filter algorithm, named as recursive least $\boldsymbol {q}$ -Gaussian kernel mean $\boldsymbol {p}$ -power, is derived under the least QKMPE criterion for robust learning in noisy environment. This new proposed algorithm reveals superior performance against Gaussian-type noise as well as the non-Gaussian perturbation, especially when the data contain large outliers. Experimental results in the context of Mackey–Glass time series prediction confirm the effectiveness of the proposed algorithm.

A Hybridized Forecasting Method Based on Weight Adjustment of Neural Network Using Generalized Type-2 Fuzzy Set

This paper proposes a hybridized forecasting method on weight adjustment of neural networks with back-propagation learning using general type-2 fuzzy sets. Initialization of weights and their adjustment in neural network are important areas of research as it increases the computation speed to get the optimized result. Higher order fuzzy logic systems are able to deal with the high levels of uncertainties present in the majority of real-world problems. Here, the concept of interval-based zSlices has been used to obtain the general type-2 fuzzy weights for the neural network architecture. We implement this proposed methodology on benchmark Mackey–Glass time series data (for $$\tau =17$$ ) and present a comparison of the results obtained with our approach and those of the existing approaches. The method is also applied on enrollment data of University of Alabama, closing price index of Shenzhen stock exchange, closing price index of Shanghai stock exchange, Canadian lynx data, and the results are presented.

Correntropy-Based Evolving Fuzzy Neural System

In this paper, a correntropy-based evolving fuzzy neural system (CEFNS) is proposed for approximation of nonlinear systems. Different from the commonly used mean-square error criterion, correntropy has a strong outliers rejection ability through capturing the higher moments of the error distribution. Considering the merits of correntropy, this paper brings contributions to build evolving fuzzy neural system (EFNS) based on the correntropy concept to achieve a more stable evolution of the rule base and update of the rule parameters instead of the commonly used mean-square error criterion. The correntropy-EFNS (CEFNS) begins with an empty rule base, and all rules are evolved online based on the correntropy criterion. The consequent part parameters are tuned based on the maximum correntropy criterion, where the correntropy is used as the cost function so as to improve the non-Gaussian noise rejection ability. The steady-state convergence performance of the CEFNS is studied through the calculation of the steady-state excess mean square error (EMSE) in two cases: Gaussian noise; and non-Gaussian noise. Finally, the CEFNS is validated through a benchmark system identification problem, a Mackey-Glass time series prediction problem as well as five other real-world benchmark regression problems under both noise-free and noisy conditions. Compared with other EFNSs, the simulation results show that the proposed CEFNS produces better approximation accuracy using the least number of rules and training time and also owns superior non-Gaussian noise handling capability.

Non-iterative T–S fuzzy modeling with random hidden-layer structure for BFG pipeline pressure prediction

The pressure of blast furnace gas (BFG) pipeline is one of the most significant criteria for the scheduling of byproduct gas system in steel industry. Given the measuring problems under current industrial conditions, a non-iterative T–S fuzzy model is established in this study for on-line modeling. Firstly, a random hidden-layer structure for the antecedent part of fuzzy rules is designed for fuzzy partitioning, which improves the efficiency of on-line training and makes the model adaptive for industrial application. A fast sparse coding identification method is then proposed to determine the consequent parameters of the fuzzy rules, which is capable of enhancing the robustness and the generalization performance under various production situations. To verify the effectiveness of the proposed method, the noise additive Mackey–Glass time series data and the practical data coming from a steel plant are employed. Furthermore, the statistics results from the practice indicate that the proposed method has good adaptability and can provide reliable measurement results for industrial applications.

Optimization of quantum-inspired neural network using memetic algorithm for function approximation and chaotic time series prediction

Abstract Heuristic and deterministic optimization methods are extensively applied for the training of artificial neural networks. Both of these methods have their own advantages and disadvantages. Heuristic stochastic optimization methods like genetic algorithm perform global search, but they suffer from the problem of slow convergence rate near global optimum. On the other hand deterministic methods like gradient descent exhibit a fast convergence rate around global optimum but may get stuck in a local optimum. Motivated by these problems, a hybrid learning algorithm combining genetic algorithm (GA) with gradient descent (GD), called HGAGD, is proposed in this paper. The new algorithm combines the global exploration ability of GA with the accurate local exploitation ability of GD to achieve a faster convergence and also a better accuracy of final solution. The HGAGD is then employed as a new training method to optimize the parameters of a quantum-inspired neural network (QINN) for two different applications. Firstly, two benchmark functions are chosen to demonstrate the potential of the proposed QINN with the HGAGD algorithm in dealing with function approximation problems. Next, the performance of the proposed method in forecasting Mackey–Glass time series and Lorenz attractor is studied. The results of these studies show the superiority of the introduced approach over other published approaches.

Sequential Outlier Criterion for Sparsification of Online Adaptive Filtering.

In this paper, we deal with the learning problem when using an adaptive filtering method. For the learning system in filtering, the knowledge is obtained and updated based on the newly acquired information that is extracted and learned from the sequential samples over time. Effective measurement on the informativeness of a sample and reasonable subsequent treatment on the sample will improve the learning performance. This paper proposes a sequential outlier criterion for sparsification of online adaptive filtering. The method is proposed to achieve effective informativeness measurement of online filtering to obtain a more accurate and more compact network in the learning process. In the proposed method, the measurement on the samples' informativeness is established based on the historical sequentially adjacent samples, and then the informative-measured samples are treated individually by the learning system based on whether the sample is informative, redundant, or abnormal. With our method, a more sensible learning process can be achieved with valid knowledge extracted, and the optimal network in the learning system can be obtained. Simulations based on static function estimation, Mackey-Glass time series prediction, and Lorenz chaotic time series prediction demonstrate that the proposed method can provide more effective classification on samples and more accurate networks in online adaptive filtering.

Prediction of chaotic time series based on the fractional-order maximum correntropy criterion algorithm

Recently, adaptive filters have been widely used to perform the prediction of chaotic time series. Generally, the Gaussian noise is considered for the system noise. However, many non-Gaussian noises, e.g., impulse noise and alpha noise, exist in real systems. Adaptive filters are therefore required to reduce such non-Gaussian noises for practical applications. For improving the robustness against non-Gaussian noise, the maximum correntropy criterion (MCC) is successfully used to derive various robust adaptive filters. In these robust adaptive filters, the steepest ascent method based on the first-order derivative is generally utilized to construct the weight update form. It is well known that the traditional derivative can be generalized by the fractional-order derivative effectively. Therefore, to further improve the performance of adaptive filters based on the MCC, the fractional-order derivative is applied to the MCC-based algorithm, generating a novel fractional-order maximum correntropy criterion (FMCC) algorithm. Under the non-Gaussian noises, the proposed FMCC algorithm can be applied to predicting the chaotic time series effectively. In the proposed FMCC algorithm, the weight update form is constructed by using a combination of the first-order derivative based term and the fractional-order derivative based term. The Riemann-Liouville definition is utilized for calculating the fractional-order derivative in the proposed FMCC algorithm. The order of the fractional-order derivative is a crucial parameter of the proposed FMCC algorithm. However, it is difficult to obtain the optimal fractional order for different nonlinear systems theoretically. Therefore, the influence of the fractional order on the prediction performance is determined by trials for different nonlinear systems. The appropriate fractional order corresponds to the optimum of prediction accuracy, and can be chosen in advance. Simulations in the context of prediction of Mackey-Glass time series and Lorenz time series demonstrate that in the case of non-Gaussian noises the proposed FMCC algorithm achieves better prediction accuracy and faster convergence rate than the least mean square (LMS) algorithm, the MCC algorithm, and the fractional-order least mean square (FLMS) algorithm. In addition, the computational complexity of different filters is compared with each other under the example of the prediction of Marckey-Glass time series by using mean consumed time. It can be found that the computational complexity of FMCC algorithm is higher than those of the MCC and the LMS algorithms, but only slightly higher than that of the FLMS algorithm. As a result, comparing with other filters, the FMCC algorithm can improve the prediction performances of chaotic time series at the cost of the increasing computational complexity.

Comparison of T-Norms and S-Norms for Interval Type-2 Fuzzy Numbers in Weight Adjustment for Neural Networks

A comparison of different T-norms and S-norms for interval type-2 fuzzy number weights is proposed in this work. The interval type-2 fuzzy number weights are used in a neural network with an interval backpropagation learning enhanced method for weight adjustment. Results of experiments and a comparative research between traditional neural networks and the neural network with interval type-2 fuzzy number weights with different T-norms and S-norms are presented to demonstrate the benefits of the proposed approach. In this research, the definitions of the lower and upper interval type-2 fuzzy numbers with random initial values are presented; this interval represents the footprint of uncertainty (FOU). The proposed work is based on recent works that have considered the adaptation of weights using type-2 fuzzy numbers. To confirm the efficiency of the proposed method, a case of data prediction is applied, in particular for the Mackey-Glass time series (for τ = 17). Noise of Gaussian type was applied to the testing data of the Mackey-Glass time series to demonstrate that the neural network using a interval type-2 fuzzy numbers method achieves a lower susceptibility to noise than other methods.

A clustering algorithm based TS fuzzy model for tracking dynamical system data

This paper presents an optimal fuzzy partition based Takagi Sugeno Fuzzy Model (TSFM) in which a novel clustering algorithm, known as Modified Fuzzy C-Regression Model (MFCRM), has been proposed. The objective function of MFCRM algorithm has been developed by considering of geometrical structure of input data and linear functional relation between input–output data. The MFCRM partitions the data space to create fuzzy subspaces (rules). A new validation criterion has been developed for detecting the right number of rules (subspaces) in a given data set. The obtained fuzzy partition is used to build the fuzzy structure and identify the premise parameters. Once, right number of rules and premise parameters have been identified, then consequent parameters have been identified by orthogonal least square (OLS) approach. The cluster validation index has been tested on synthetic data set. The effectiveness of MFCRM based TSFM has been validated on benchmark examples, such as Boiler Turbine system, Mackey–Glass time series data and Box–Jenkins model. The model performance is also validated through high-dimensional data such as Auto-MPG data and Boston Housing data.

Volumetric behavior quantification to characterize trajectory in phase space

This paper presents a methodology to extract a number of quantifier features to characterize volumetric behavior of trajectories in phase space. These features quantify expanding and contracting behaviors and complexity that can be used in nonlinear and chaotic signals classification or clustering problems. One of the features is directly extracted from the distance matrix and seven features are extracted from a matrix that is subsequently obtained from the distance matrix. To illustrate the proposed quantifiers, Mackey–Glass time series and Lorenz system were employed and feature evaluation was performed. It is shown that the proposed quantifier features are robust to different initializations and can quantify volumetric behavior characteristics. In addition, the ability of these features to differentiate between signals with different parameters is compared with some common nonlinear features such as fractal dimensions and recurrence quantification analysis features.

A prediction method based on wavelet transform and multiple models fusion for chaotic time series

In order to improve the prediction accuracy of chaotic time series, a prediction method based on wavelet transform and multiple models fusion is proposed. The chaotic time series is decomposed and reconstructed by wavelet transform, and approximate components and detail components are obtained. According to different characteristics of each component, least squares support vector machine (LSSVM) is used as predictive model for approximation components. At the same time, an improved free search algorithm is utilized for predictive model parameters optimization. Auto regressive integrated moving average model (ARIMA) is used as predictive model for detail components. The multiple prediction model predictive values are fusion by Gauss–Markov algorithm, the error variance of predicted results after fusion is less than the single model, the prediction accuracy is improved. The simulation results are compared through two typical chaotic time series include Lorenz time series and Mackey–Glass time series. The simulation results show that the prediction method in this paper has a better prediction.

Comparative analysis of three approaches of antecedent part generation for an IT2 TSK FLS

Since extreme learning machine is a non-iterative estimation procedure, it is faster than gradient-based algorithms which are iterative. Moreover, the extreme learning machine does not have any design parameters such as learning rate, covariance matrix, etc. The rigorous proof of universal approximation of extreme learning machine with much milder conditions makes it a preferable choice in many different approaches. Although this algorithm is optimal for the parameters which appear linearly in the consequent part of interval type-2 fuzzy logic systems, it is not optimal for the parameters of the antecedent part as it uses random parameters. In this paper, heuristic optimization approaches such as genetic algorithm and artificial bee colony are used to optimize the parameters of the antecedent part of interval type-2 fuzzy logic systems. As these methods are global optimizers, there is less possibility that they will fall in a local minima and are suitable for the selection of the parameters of the antecedent part. A comparative analysis of the optimal parameters with the randomly and manually generated parameters is presented here using noise-free and noisy Mackey-Glass time series data sets and a real world data set. Simulation results support this idea over randomly and manually generated parameters.