Mackey-Glass Chaotic Time Series Prediction Research Articles

Simplification of ANFIS based on importance-confidence-similarity measures

Adaptive-network-based fuzzy inference system (ANFIS) is a well-known neuro-fuzzy system, which is widely implemented in nonlinear system approximation, prediction, control, and pattern classification. However, either too many fuzzy sets or fuzzy rules are indispensable to establish an accurate ANFIS for a high-dimensional system, which increases the complexity and limits the widespread application of the ANFIS. In this paper, an approach to simplify the ANFIS is developed based on importance-confidence-similarity (ICS) measures. The simplification process consists of selection of fuzzy rules with large importance-confidence (IC) values and reduction of similar fuzzy rules. The importance is defined as the average normalized firing strength of the rule. The confidence which reflects the approximation performance of a T-S fuzzy rule is estimated by the weighted orthogonal distance from the output data to the rule hyperplane (the consequent of a linear T-S fuzzy rule). The IC measures provide a comprehensive basis for the selection of the appropriate fuzzy rules. Furthermore, to the best of our knowledge, an approach to measure similarities of the T-S fuzzy rules including the rule premises and consequents is first proposed in this paper. We also develop a direct and simple method to remove the similar fuzzy rules based on a hierarchical tree and the IC measures. The proposed simplification approach is applied to Mackey-Glass chaotic time series prediction problem and the strip temperature increment prediction problem of the indirect-fired furnace of the annealing heating line. The results show that our proposed method achieves a balance between simplicity and accuracy of the ANFIS for the high-dimensional system.

An Efficient Filtering Algorithm against Impulse Noise in Communication Systems

<p>The kernel adaptive filter (KAF), which processes data in the reproducing kernel Hilbert space (RKHS), can improve the performance of conventional adaptive filters in nonlinear systems. However, the presence of impulse noise can seriously degrade the performance of KAF. In this paper, we propose a kernel modified-sign least-mean-square algorithm (KMSLMS) to mitigate the impact of impulse noise in communication systems. Moreover, we apply the nearest-instance-centroid estimation (NICE) algorithm to reduce the computational complexity of our KMSLMS algorithm, called the NICE-KMSLMS algorithm. Finally, computer simulations were used to evaluate the effectiveness of our proposed method. Compared with the conventional kernel least-mean-square algorithm (KLMS), our proposed method can improve the testing mean-squared error (MSE) by 2.32 dB and 7.39 dB for the nonlinear channel equalization and Mackey-Glass chaotic time series prediction problems, respectively. Furthermore, the testing MSE degradation caused by combining the NICE algorithm with our KMSLMS algorithm is negligible but can save about 55% computational cost in terms of the required mean size.</p> <p> </p>

Underwater Acoustic Signal Prediction Based on MVMD and Optimized Kernel Extreme Learning Machine

Aiming at the chaotic characteristics of underwater acoustic signal, a prediction model of grey wolf-optimized kernel extreme learning machine (OKELM) based on MVMD is proposed in this paper for short-term prediction of underwater acoustic signals. To solve the problem of K value selection in variational mode decomposition, a new K value selection method MVMD is proposed from the perspective of mutual information, which avoids the blindness of variational mode decomposition (VMD) in the preset modal number. Based on the prediction model of kernel extreme learning machine (KELM), this paper uses grey wolf optimization (GWO) algorithm to optimize and select its regularization parameters and kernel parameters and proposes an optimized kernel extreme learning machine OKELM. To further improve the prediction performance of the model, combined with MVMD, an underwater acoustic signal prediction model based on MVMD-OKELM is established. MVMD-OKELM prediction model is applied to Mackey–Glass chaotic time series prediction and underwater acoustic signal prediction and is compared with ARIMA, EMD-OKELM, and other prediction models. The experimental results show that the proposed MVMD-OKELM prediction model has a higher prediction accuracy and can be effectively applied to the prediction of underwater acoustic signal series.

Kernel Affine Projection P-norm (KAPP) Filtering under Alpha Stable Distribution Noise Environment

Aiming at improving the performance of the nonlinear adaptive filtering under the alpha-stable distribution noise environment, Kernel Affine Projection P-norm (KAPP) algorithm based on the minimum dispersion coefficient criterion and the affine projection is deduced. The accuracy of the gradient estimation is enhanced by using the input signals and the error signals at multiple times. The simulation results on Mackey–Glass chaotic time series prediction show that the KAPP algorithm has faster convergence speed, better steady-state performance and stronger robustness under the Gaussian noise and stable distributed noise environment.

The modified sufficient conditions for echo state property and parameter optimization of leaky integrator echo state network

Leaky integrator echo state network (Leaky-ESN), an improved echo state network, is suitable for learning very slow dynamic systems and replaying the learnt system at different speeds. To ensure the echo state property of Leaky-ESN, spectral radius of reservoir connection weight matrix needs to be less than leaking rate, which means that the output feedback of Leaky-ESN is actually ignored. Leaky-ESN without output feedback is equivalent to being a feedforward type neural network. Therefore, in this paper, we consider Leaky-ESN model with output feedback. Firstly, we give the modified sufficient conditions for echo state property, which are actually some inequality constraints about the control parameters (mainly leaking rate, spectral radius of reservoir connection weight matrix and the scaling of output feedback) and the maximum singular values of feedback and output connection weight matrices. Secondly, to reduce the influence of the initial values of the control parameters on Leaky-ESN, we use the barrier method to optimize the control parameters. The barrier method can convert the constrained optimization problem into the unconstrained optimization problem. Newton’s method and its improve method, for example, eigenvalue modification methods, are further used to solve the unconstrained optimization problem and then obtain the control parameters of Leaky-ESN. Finally, Mackey–Glass chaotic time series prediction is selected to validate the method proposed in this paper. Simulation results show that the method proposed in this paper can make Leaky-ESN model more stable and higher approximation precision.

Accurate fuzzy predictive models through complexity reduction based on decision of needed fuzzy rules

Abstract Fuzzy predictive models (FPMs) are employed in a wide variety of domains for their high prediction accuracy and good interpretability. For a FPM, the fuzzy rules from data or expert experience are always firstly extracted to establish a relatively complete fuzzy rule base. However, it is found that a large amount of extracted fuzzy rules which are always not used for prediction increase the complexity of the FPM. Moreover, inaccurate fuzzy rules may be used for prediction to degrade the accuracy of the FPM. Therefore, we proposed a method to establish an accurate FPM through complexity reduction based on decision of needed fuzzy rules. To avoid extractions of the rules which are not needed for prediction, reduce the complexity of the FPM, and suppress the influence of some inaccurate fuzzy rules, we add the decision of fuzzy rules needed for prediction based on membership values to predictive modeling before the extraction of fuzzy rules. And then an improved WM method based on optimization of centers of output fuzzy subsets for fuzzy rules (COiWM), which was proven to possess high completeness, robustness, and accuracy in our previous work, is used to generate the needed fuzzy rules directly from historical data. The rules which can not to be generated are extrapolated based on these data-generated fuzzy rules. At last, two fuzzy rule bases are built and used together to fuzzy predictive modeling. Experimental results of a case study on short term daily maximum electric load forecasting and Mackey–Glass chaotic time-series prediction show that our proposed method reduces the complexity of the FPMs and enhances the prediction accuracy.

A Beta basis function Interval Type-2 Fuzzy Neural Network for time series applications

The huge complexity and uncertainty in real life requires the use of advanced automatic learning methods to find out better approximators and suitable relationship in real data behavior. Neuro fuzzy systems have been proved to be excellent universal approximators. In this paper we propose a new based function Interval Type-2 Fuzzy Neural Network denoted ”Beta basis function Interval Type-2 Fuzzy Neural Network”, the BIT2FNN. The main idea is to involve type-2 beta fuzzy sets in the design process of fuzzy networks. The proposed architecture is based on beta type-2 fuzzy sets in the antecedent part, while the consequent part achieves the TSK (Takagi–Sugeno–Kang) fuzzy output strategy. Thanks to the beta function flexibility, the network achieve a good performance and shows a good resistance to noisy data. First order derivatives of type-1 and type-2 Beta functions were developed for the first time for designing fuzzy logic systems based on given input–output pairs. The backpropagation algorithm was used for the learning process of antecedent fuzzy beta parameters and the consequent part. The performance of the proposed model of Beta fuzzy logic system is evaluated with mainly two problems of time series applications : the Mackey Glass Chaotic Time-Series prediction problem with different setting of parameters and levels of noise and the ECG heart-rate Time Series monitoring problem.

A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks

In this research, we propose a novel algorithm for learning of the recurrent neural networks called as the fractional back-propagation through time (FBPTT). Considering the potential of the fractional calculus, we propose to use the fractional calculus-based gradient descent method to derive the FBPTT algorithm. The proposed FBPTT method is shown to outperform the conventional back-propagation through time algorithm on three major problems of estimation namely nonlinear system identification, pattern classification and Mackey–Glass chaotic time series prediction.

Weighted fuzzy interpolative reasoning for sparse fuzzy rule-based systems based on piecewise fuzzy entropies of fuzzy sets

In this paper, we propose a new method for weighted fuzzy interpolative reasoning in sparse fuzzy rule-based systems based on piecewise fuzzy entropies of fuzzy sets. First, the proposed method uses the representative values of antecedent fuzzy sets, the representative values of observation fuzzy sets, and the representative values of consequence fuzzy sets of fuzzy rules to get the characteristic points of the fuzzy interpolative result represented by a fuzzy set. Then, it calculates the piecewise fuzzy entropies between any two characteristic points of the antecedent fuzzy sets, the piecewise fuzzy entropies between any two characteristic points of the observation fuzzy sets, and the piecewise fuzzy entropies between any two characteristic points of the consequence fuzzy sets of the fuzzy rules, respectively. Then, it calculates the weights of the antecedent fuzzy sets of each fuzzy rule, respectively, and calculates the weight of each fuzzy rule. Then, it calculates the piecewise fuzzy entropies between any two characteristic points of the fuzzy interpolative result. Finally, it uses the secant method to calculate the degree of membership of each obtained characteristic point of the fuzzy interpolative result. The experimental results show that the proposed method outperforms the existing methods for dealing with the multivariate regression problems, the Mackey–Glass chaotic time series prediction problem, and the time series prediction problems.

Adaptive natural gradient learning algorithms for Mackey–Glass chaotic time prediction

In this paper, we introduce the adaptive natural gradient method into the multilayer perceptrons (MLPs) and radial basis function (RBF) networks. We give a good performance for the Mackey–Glass chaotic time series prediction, and compare it with the LMA. Results show that the adaptive natural gradient methods for MLPs and RBFs, which are the online learning, can give almost the same performance with the LMA (MLPs), which is the batch mode learning. However, the performance of LMA (RBFs) is very poor and is very sensitive with the initial parameters.

Levenberg-Marquardt Algorithm for Mackey-Glass Chaotic Time Series Prediction

For decades, Mackey-Glass chaotic time series prediction has attracted more and more attention. When the multilayer perceptron is used to predict the Mackey-Glass chaotic time series, what we should do is to minimize the loss function. As is well known, the convergence speed of the loss function is rapid in the beginning of the learning process, while the convergence speed is very slow when the parameter is near to the minimum point. In order to overcome these problems, we introduce the Levenberg-Marquardt algorithm (LMA). Firstly, a rough introduction is given to the multilayer perceptron, including the structure and the model approximation method. Secondly, we introduce the LMA and discuss how to implement the LMA. Lastly, an illustrative example is carried out to show the prediction efficiency of the LMA. Simulations show that the LMA can give more accurate prediction than the gradient descent method.

Universal Approximation of a Class of Interval Type-2 Fuzzy Neural Networks in Nonlinear Identification

Neural networks (NNs), type-1 fuzzy logic systems (T1FLSs), and interval type-2 fuzzy logic systems (IT2FLSs) have been shown to be universal approximators, which means that they can approximate any nonlinear continuous function. Recent research shows that embedding an IT2FLS on an NN can be very effective for a wide number of nonlinear complex systems, especially when handling imperfect or incomplete information. In this paper we show, based on the Stone-Weierstrass theorem, that an interval type-2 fuzzy neural network (IT2FNN) is a universal approximator, which uses a set of rules and interval type-2 membership functions (IT2MFs) for this purpose. Simulation results of nonlinear function identification using the IT2FNN for one and three variables and for the Mackey-Glass chaotic time series prediction are presented to illustrate the concept of universal approximation.

FASTER TRAINING USING FUSION OF ACTIVATION FUNCTIONS FOR FEED FORWARD NEURAL NETWORKS

Multilayer feed-forward neural networks are widely used based on minimization of an error function. Back propagation (BP) is a famous training method used in the multilayer networks but it often suffers from the drawback of slow convergence. To make the learning faster, we propose 'Fusion of Activation Functions' (FAF) in which different conventional activation functions (AFs) are combined to compute final activation. This has not been studied extensively yet. One of the sub goals of the paper is to check the role of linear AFs in combination. We investigate whether FAF can enable the learning to be faster. Validity of the proposed method is examined by performing simulations on challenging nine real benchmark classification and time series prediction problems. The FAF has been applied to 2-bit, 3-bit and 4-bit parity, the breast cancer, Diabetes, Heart disease, Iris, wine, Glass and Soybean classification problems. The algorithm is also tested with Mackey-Glass chaotic time series prediction problem. The algorithm is shown to work better than other AFs used independently in BP such as sigmoid (SIG), arctangent (ATAN), logarithmic (LOG).

A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks

In real life, information about the world is uncertain and imprecise. The cause of this uncertainty is due to: deficiencies on given information, the fuzzy nature of our perception of events and objects, and on the limitations of the models we use to explain the world. The development of new methods for dealing with information with uncertainty is crucial for solving real life problems. In this paper three interval type-2 fuzzy neural network (IT2FNN) architectures are proposed, with hybrid learning algorithm techniques (gradient descent backpropagation and gradient descent with adaptive learning rate backpropagation). At the antecedents layer, a interval type-2 fuzzy neuron (IT2FN) model is used, and in case of the consequents layer an interval type-1 fuzzy neuron model (IT1FN), in order to fuzzify the rule’s antecedents and consequents of an interval type-2 Takagi–Sugeno–Kang fuzzy inference system (IT2-TSK-FIS). IT2-TSK-FIS is integrated in an adaptive neural network, in order to take advantage the best of both models. This provides a high order intuitive mechanism for representing imperfect information by means of use of fuzzy If–Then rules, in addition to handling uncertainty and imprecision. On the other hand, neural networks are highly adaptable, with learning and generalization capabilities. Experimental results are divided in two kinds: in the first one a non-linear identification problem for control systems is simulated, here a comparative analysis of learning architectures IT2FNN and ANFIS is done. For the second kind, a non-linear Mackey–Glass chaotic time series prediction problem with uncertainty sources is studied. Finally, IT2FNN proved to be more efficient mechanism for modeling real-world problems.

Chaotic time series prediction based on RBF neural networks with a new clustering algorithm

Two-phase learning method is considered in this paper to configure the RBF neural networks for chaotic time series prediction. When determining the hidden-layer centers with the unsupervised learning algorithm, a new distance measure is presented based on Gaussian basis, and the strategy of input-output clustering is employed in combination. The punishment factor in Gaussian basis distance is designed based on Fisher separable ratio, which can improve the clustering performance. Moreover, the introduction of input-output clustering strategy establishes the relation between the clustering performance and the prediction performance. Therefore, the RBF neural networks constructed by this method can not only assure the compact structure, but also improve the prediction performance. This method is applied to Mackey-Glass, Lorenz and Logistic chaotic time series prediction, and the results indicate its validity.

On-Line Evolution of Takagi-Sugeno Fuzzy Models

Evolving Takagi-Sugeno (eTS) fuzzy models and the method for their on-line identification has been recently introduced for both MISO and MIMO case. In this paper, the mechanism for rule-base evolution, one of the central points of the algorithm together with the recursive clustering and modified recursive least squares (RLS) estimation, is studied in detail. Different scenarios are considered for the rule base upgrade and modification. The radius of influence of each fuzzy rule is considered to be a vector instead of a scalar as in the original eTS approach, allowing different areas of the data space to be covered by each input variable. Simulation results using a well-known benchmark (Mackey-Glass chaotic time-series prediction) are presented.

Improving the fuzzy system performance by fuzzy system ensemble

This paper proposes a fuzzy system ensemble (FSE) that can improve the system performance in non-linear and complex problems. Each fuzzy system in the FSE is built from two design stages, each stage of which is performed by different genetic algorithms (GAs). The first stage generates a fuzzy rule base that covers as many of the training examples as possible. The second stage builds fine-tuned membership functions that make the system error as small as possible. These two stages are repeated independently upon the different partitions of input-output variables. The system error will be reduced further by invoking the FSE that combines multiple fuzzy systems with an equal system-error weighting method where the weight constant is inversely proportional to the FS's error. Applications of the FSE to both the truck backer-upper control and the Mackey-Glass chaotic time-series prediction are presented. For the truck control problem, control performance of the proposed method is compared with the approach of Wang and Mendel [IEEE Transactions on System, Man, and Cybernetics 22 (1992) 1414] in terms of either the rate of successful controls reaching to the goal or the average traveling distance. For the chaotic time-series prediction problem, prediction accuracy of the proposed method is compared with that of other fuzzy or neural network predictors in terms of non-dimensional error index (NDEI).