Chaotic Time Series Prediction Problem Research Articles

A Modified Sine Cosine Algorithm for Numerical Optimization

Sine cosine algorithm (SCA) is a recently developed meta-heuristic method based on the characteristics of sine and cosine functions. However, SCA algorithm may suffer premature convergence in solving complex optimization problems. To mitigate this limitation, a modified SCA (MSCA) is developed to achieve an effective balance between exploration and exploitation. Specifically, the conversion parameter can be configured adaptively by using the individual fitness information in the evolution process to guide the individual search behavior. After that, an inertia weight is embedded in the position updating equation to further improve the search accuracy and accelerate the convergence speed. The performance of MSCA algorithm is investigated on 13 benchmark functions and the chaotic time series prediction problem. Experimental results demonstrate that MSCA algorithm is more effective than the other optimization methods.

Simplification of ANFIS based on importance-confidence-similarity measures

Adaptive-network-based fuzzy inference system (ANFIS) is a well-known neuro-fuzzy system, which is widely implemented in nonlinear system approximation, prediction, control, and pattern classification. However, either too many fuzzy sets or fuzzy rules are indispensable to establish an accurate ANFIS for a high-dimensional system, which increases the complexity and limits the widespread application of the ANFIS. In this paper, an approach to simplify the ANFIS is developed based on importance-confidence-similarity (ICS) measures. The simplification process consists of selection of fuzzy rules with large importance-confidence (IC) values and reduction of similar fuzzy rules. The importance is defined as the average normalized firing strength of the rule. The confidence which reflects the approximation performance of a T-S fuzzy rule is estimated by the weighted orthogonal distance from the output data to the rule hyperplane (the consequent of a linear T-S fuzzy rule). The IC measures provide a comprehensive basis for the selection of the appropriate fuzzy rules. Furthermore, to the best of our knowledge, an approach to measure similarities of the T-S fuzzy rules including the rule premises and consequents is first proposed in this paper. We also develop a direct and simple method to remove the similar fuzzy rules based on a hierarchical tree and the IC measures. The proposed simplification approach is applied to Mackey-Glass chaotic time series prediction problem and the strip temperature increment prediction problem of the indirect-fired furnace of the annealing heating line. The results show that our proposed method achieves a balance between simplicity and accuracy of the ANFIS for the high-dimensional system.

Cuckoo search algorithm based on cloud model and its application

Cuckoo search algorithm is an efficient random search method for numerical optimization. However, it is very sensitive to the setting of the step size factor. To address this issue, a new cuckoo search algorithm based on cloud model is developed to dynamically configure the step size factor. More specifically, the idea of giving consideration to both fuzziness and randomness of cloud model is innovatively introduced into cuckoo search algorithm, and the appropriate step size factor can be determined according to the membership degree and an exponential function, so as to realize the adaptive adjustment of the control parameter. After that, simulation experiments are conducted on 25 benchmark functions with different dimensions and two chaotic time series prediction problems to comprehensively evaluate the superiority of the proposed algorithm. Numerical results demonstrate that the developed method is more competitive than the other five CS and several non-CS algorithms.

An Efficient Filtering Algorithm against Impulse Noise in Communication Systems

<p>The kernel adaptive filter (KAF), which processes data in the reproducing kernel Hilbert space (RKHS), can improve the performance of conventional adaptive filters in nonlinear systems. However, the presence of impulse noise can seriously degrade the performance of KAF. In this paper, we propose a kernel modified-sign least-mean-square algorithm (KMSLMS) to mitigate the impact of impulse noise in communication systems. Moreover, we apply the nearest-instance-centroid estimation (NICE) algorithm to reduce the computational complexity of our KMSLMS algorithm, called the NICE-KMSLMS algorithm. Finally, computer simulations were used to evaluate the effectiveness of our proposed method. Compared with the conventional kernel least-mean-square algorithm (KLMS), our proposed method can improve the testing mean-squared error (MSE) by 2.32 dB and 7.39 dB for the nonlinear channel equalization and Mackey-Glass chaotic time series prediction problems, respectively. Furthermore, the testing MSE degradation caused by combining the NICE algorithm with our KMSLMS algorithm is negligible but can save about 55% computational cost in terms of the required mean size.</p> <p> </p>

An introduction to brain emotional learning inspired models (BELiMs) with an example of BELiMs’ applications

Brain emotional learning-inspired models (BELiMs) is a new category of computational intelligence (CI) paradigms. The general structure of BELiMs is based on the neural structure of the emotion system which processes and evaluates fear-induced stimuli, to produce emotional responses. The function of a BELiM is implemented by assigning adaptive networks to different parts of its structure. The primary motivation for developing BELiMs is to address model and time complexity issues associated with supervised machine learning artificial neural networks and neuro-fuzzy methods. One of the applications of BELiMs is chaotic time series prediction problems. A BEliM can be used as a time series prediction model. This paper introduces BELiMs as a new CI paradigm and explains historical, theoretical, structural and functional aspects of BELiMs. I also validate and evaluate the performance of BELiMs as a time series prediction model by examining different variations of BELiMs on benchmark time series data sets and comparing obtained results with different CI models.

A Beta basis function Interval Type-2 Fuzzy Neural Network for time series applications

The huge complexity and uncertainty in real life requires the use of advanced automatic learning methods to find out better approximators and suitable relationship in real data behavior. Neuro fuzzy systems have been proved to be excellent universal approximators. In this paper we propose a new based function Interval Type-2 Fuzzy Neural Network denoted ”Beta basis function Interval Type-2 Fuzzy Neural Network”, the BIT2FNN. The main idea is to involve type-2 beta fuzzy sets in the design process of fuzzy networks. The proposed architecture is based on beta type-2 fuzzy sets in the antecedent part, while the consequent part achieves the TSK (Takagi–Sugeno–Kang) fuzzy output strategy. Thanks to the beta function flexibility, the network achieve a good performance and shows a good resistance to noisy data. First order derivatives of type-1 and type-2 Beta functions were developed for the first time for designing fuzzy logic systems based on given input–output pairs. The backpropagation algorithm was used for the learning process of antecedent fuzzy beta parameters and the consequent part. The performance of the proposed model of Beta fuzzy logic system is evaluated with mainly two problems of time series applications : the Mackey Glass Chaotic Time-Series prediction problem with different setting of parameters and levels of noise and the ECG heart-rate Time Series monitoring problem.

A learning and niching based backtracking search optimisation algorithm and its applications in global optimisation and ANN training

A backtracking search optimisation algorithm that uses historic population information for learning was proposed recently for solving optimisation problems. However, the learning ability and the robustness of this algorithm remain relatively poor. To improve the performance of the backtracking search algorithm (BSA), a modified backtracking search optimisation algorithm (MBSA), based on learning and niching strategies, is presented in this paper. Three main strategies, a learning strategy, a niching strategy, and a mutation strategy, are incorporated into the proposed MBSA algorithm. Learning the best individual in current generation and the best position achieved so far is used to improve the convergence speed. Niching and mutation strategies are used to improve the diversity of the MBSA. Finally, some benchmark functions and three chaotic time series prediction problems based on neural networks are simulated to test the effectiveness of MBSA, and the results are compared with those obtained using some other evolutionary algorithms (EAs). The simulation results indicate that the MBSA outperforms other EAs for most functions and chaotic time series.

Weighted fuzzy interpolative reasoning for sparse fuzzy rule-based systems based on piecewise fuzzy entropies of fuzzy sets

In this paper, we propose a new method for weighted fuzzy interpolative reasoning in sparse fuzzy rule-based systems based on piecewise fuzzy entropies of fuzzy sets. First, the proposed method uses the representative values of antecedent fuzzy sets, the representative values of observation fuzzy sets, and the representative values of consequence fuzzy sets of fuzzy rules to get the characteristic points of the fuzzy interpolative result represented by a fuzzy set. Then, it calculates the piecewise fuzzy entropies between any two characteristic points of the antecedent fuzzy sets, the piecewise fuzzy entropies between any two characteristic points of the observation fuzzy sets, and the piecewise fuzzy entropies between any two characteristic points of the consequence fuzzy sets of the fuzzy rules, respectively. Then, it calculates the weights of the antecedent fuzzy sets of each fuzzy rule, respectively, and calculates the weight of each fuzzy rule. Then, it calculates the piecewise fuzzy entropies between any two characteristic points of the fuzzy interpolative result. Finally, it uses the secant method to calculate the degree of membership of each obtained characteristic point of the fuzzy interpolative result. The experimental results show that the proposed method outperforms the existing methods for dealing with the multivariate regression problems, the Mackey–Glass chaotic time series prediction problem, and the time series prediction problems.

A hybridization of teaching–learning-based optimization and differential evolution for chaotic time series prediction

Chaotic time series prediction problems have some very interesting properties and their prediction has received increasing interest in the recent years. Prediction of chaotic time series based on the phase space reconstruction theory has been applied in many research fields. It is well known that prediction of a chaotic system is a nonlinear, multivariable and multimodal optimization problem for which global optimization techniques are required in order to avoid local optima. In this paper, a new hybrid algorithm named teaching---learning-based optimization (TLBO)---differential evolution (DE), which integrates TLBO and DE, is proposed to solve chaotic time series prediction. DE is incorporated into update the previous best positions of individuals to force TLBO jump out of stagnation, because of its strong searching ability. The proposed hybrid algorithm speeds up the convergence and improves the algorithm's performance. To demonstrate the effectiveness of our approaches, ten benchmark functions and three typical chaotic nonlinear time series prediction problems are used for simulating. Conducted experiments indicate that the TLBO---DE performs significantly better than, or at least comparable to, TLBO and some other algorithms.

BRAIN EMOTIONAL LEARNING-BASED PATTERN RECOGNIZER

In this article, the brain emotional learning-based pattern recognizer (BELPR) is proposed to solve multiple input–multiple output classification and chaotic time series prediction problems. BELPR is based on an extended computational model of the human brain limbic system that consists of an emotional stimuli processor. The BELPR is model free and learns the patterns in a supervised manner and evaluates the output(s) using the activation function tansig. In the numerical studies, various comparisons are made between BELPR and a multilayer perceptron (MLP) with a back-propagation learning algorithm. The methods are tested to classify 12 UCI (University of California, Irvine) machine learning data sets and to predict activity indices of the Earth's magnetosphere. The main features of BELPR are higher accuracy, decreased time and spatial complexity, and faster training.

A novel self-constructing Radial Basis Function Neural-Fuzzy System

This paper proposes a novel self-constructing least-Wilcoxon generalized Radial Basis Function Neural-Fuzzy System (LW-GRBFNFS) and its applications to non-linear function approximation and chaos time sequence prediction. In general, the hidden layer parameters of the antecedent part of most traditional RBFNFS are decided in advance and the output weights of the consequent part are evaluated by least square estimation. The hidden layer structure of the RBFNFS is lack of flexibility because the structure is fixed and cannot be adjusted effectively according to the dynamic behavior of the system. Furthermore, the resultant performance of using least square estimation for output weights is often weakened by the noise and outliers.This paper creates a self-constructing scenario for generating antecedent part of RBFNFS with particle swarm optimizer (PSO). For training the consequent part of RBFNFS, instead of traditional least square (LS) estimation, least-Wilcoxon (LW) norm is employed in the proposed approach to do the estimation. As is well known in statistics, the resulting linear function by using the rank-based LW norm approximation to linear function problems is usually robust against (or insensitive to) noises and outliers and therefore increases the accuracy of the output weights of RBFNFS. Several nonlinear functions approximation and chaotic time series prediction problems are used to verify the efficiency of self-constructing LW-GRBFNIS proposed in this paper. The experimental results show that the proposed method not only creates optimal hidden nodes but also effectively mitigates the noise and outliers problems.

A NOVEL EFFICIENT LEARNING ALGORITHM FOR SELF-GENERATING FUZZY NEURAL NETWORK WITH APPLICATIONS

In this paper, a novel efficient learning algorithm towards self-generating fuzzy neural network (SGFNN) is proposed based on ellipsoidal basis function (EBF) and is functionally equivalent to a Takagi-Sugeno-Kang (TSK) fuzzy system. The proposed algorithm is simple and efficient and is able to generate a fuzzy neural network with high accuracy and compact structure. The structure learning algorithm of the proposed SGFNN combines criteria of fuzzy-rule generation with a pruning technology. The Kalman filter (KF) algorithm is used to adjust the consequent parameters of the SGFNN. The SGFNN is employed in a wide range of applications ranging from function approximation and nonlinear system identification to chaotic time-series prediction problem and real-world fuel consumption prediction problem. Simulation results and comparative studies with other algorithms demonstrate that a more compact architecture with high performance can be obtained by the proposed algorithm. In particular, this paper presents an adaptive modeling and control scheme for drug delivery system based on the proposed SGFNN. Simulation study demonstrates the ability of the proposed approach for estimating the drug's effect and regulating blood pressure at a prescribed level.

Multivariate chaotic time series prediction based on extreme learning machine

For multivariate chaotic time series prediction problem, a prediction based on input variable selection and extreme learning machine is proposed in this paper. The multivariate chaotic time series is reconstructed in phase space, and a mutual information based method is used to select the input variables, which have high statistics information with the output variables. The extreme learning machine is conducted to model the multivariate chaotic time series in the phase space by utilizing its approximation capability. In order to improve the prediction accuracy, a model selection algorithm is conducted for extreme learning machine to choose an expected minimum risk prediction model. Simulation results based on Lorenz, Rssler multivariate chaotic time series and Rssler hyperchaotic time series show the effectiveness of the proposed method.

A Generalized Ellipsoidal Basis Function Based Online Self-constructing Fuzzy Neural Network

In this paper, we propose a Generalized ellipsoidal basis function based online self-constructing fuzzy neural network (GEBF-OSFNN) which extends the ellipsoidal basis function (EBF)-based fuzzy neural networks (FNNs) by permitting input variables to be modeled by dissymmetrical Gaussian functions (DGFs). Due to the flexibility and dissymmetry of left and right widths of the DGF, the partitioning made by DGFs in the input space is more flexible and more interpretable, and therefore results in a parsimonious FNN with high performance under the online learning algorithm. The geometric growing criteria and the error reduction ratio (ERR) method are used as growing and pruning strategies respectively to realize the structure learning algorithm which implements an optimal and compact network structure. The GEBF-OSFNN starts with no hidden neurons and does not need to partition the input space a priori. In addition, all free parameters in premises and consequents are adjusted online based on the ?-completeness of fuzzy rules and the linear least square (LLS) approach, respectively. The performance of the GEBF-OSFNN paradigm is compared with other well-known algorithms like RAN, RANEKF, MRAN, ANFIS, OLS, RBF-AFS, DFNN, GDFNN GGAP-RBF, OS-ELM, SOFNN and FAOS-PFNN, etc., on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world benchmark problems. Simulation results demonstrate that the proposed GEBF-OSFNN approach can facilitate a more powerful and more parsimonious FNN with better performance of approximation and generalization.

Chaotic Time Series Prediction Using Immune Optimization Theory

To solve chaotic time series prediction problem, a novel Prediction approach for chaotic time series based on Immune Optimization Theory (PIOT) is proposed. In PIOT, the concepts and formal definitions of antigen, antibody and affinity being used for time series prediction are given, and the mathematical models of immune optimization operators being used for establishing time series prediction model are exhibited. Chaotic time series is analyzed and corresponding sample space is reconstructed by phase space reconstruction method; then, the prediction model of chaotic time series is constructed by immune optimization theory; finally, using this prediction model to forecast chaotic time series. To demonstrate the effectiveness of PIOT, the three typical chaotic nonlinear time series are generated by nonlinear dynamics systems that are Lorenz, Mackey–Glass and Henon, respectively, and are used for simulating prediction. The simulation results show that PIOT is a feasible and effective prediction method, and meanwhile provides a novel prediction approach for chaotic time series.

Chaotic Time Series Prediction Using Immune Optimization Theory

To solve chaotic time series prediction problem, a novel Prediction approach for chaotic time series based on Immune Optimization Theory (PIOT) is proposed. In PIOT, the concepts and formal definitions of antigen, antibody and affinity being used for time series prediction are given, and the mathematical models of immune optimization operators being used for establishing time series prediction model are exhibited. Chaotic time series is analyzed and corresponding sample space is reconstructed by phase space reconstruction method; then, the prediction model of chaotic time series is constructed by immune optimization theory; finally, using this prediction model to forecast chaotic time series. To demonstrate the effectiveness of PIOT, the three typical chaotic nonlinear time series are generated by nonlinear dynamics systems that are Lorenz, Mackey–Glass and Henon, respectively, and are used for simulating prediction. The simulation results show that PIOT is a feasible and effective prediction method, and ...

FASTER TRAINING USING FUSION OF ACTIVATION FUNCTIONS FOR FEED FORWARD NEURAL NETWORKS

Multilayer feed-forward neural networks are widely used based on minimization of an error function. Back propagation (BP) is a famous training method used in the multilayer networks but it often suffers from the drawback of slow convergence. To make the learning faster, we propose 'Fusion of Activation Functions' (FAF) in which different conventional activation functions (AFs) are combined to compute final activation. This has not been studied extensively yet. One of the sub goals of the paper is to check the role of linear AFs in combination. We investigate whether FAF can enable the learning to be faster. Validity of the proposed method is examined by performing simulations on challenging nine real benchmark classification and time series prediction problems. The FAF has been applied to 2-bit, 3-bit and 4-bit parity, the breast cancer, Diabetes, Heart disease, Iris, wine, Glass and Soybean classification problems. The algorithm is also tested with Mackey-Glass chaotic time series prediction problem. The algorithm is shown to work better than other AFs used independently in BP such as sigmoid (SIG), arctangent (ATAN), logarithmic (LOG).

A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks

In this paper, we present a fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks (FAOS-PFNN), where a novel structure learning algorithm incorporating a pruning strategy into new growth criteria is developed. The proposed growing procedure without pruning not only speeds up the online learning process but also facilitates a more parsimonious fuzzy neural network while achieving comparable performance and accuracy by virtue of the growing and pruning strategy. The FAOS-PFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In the parameter learning phase, all the free parameters of hidden units, regardless of whether they are newly created or originally existing, are updated by the extended Kalman filter (EKF) method. The effectiveness and superiority of the FAOS-PFNN paradigm is compared with other popular approaches like resource allocation network (RAN), RAN via the extended Kalman filter (RANEKF), minimal resource allocation network (MRAN), adaptive-network-based fuzzy inference system (ANFIS), orthogonal least squares (OLS), RBF-AFS, dynamic fuzzy neural networks (DFNN), generalized DFNN (GDFNN), generalized GAP-RBF (GGAP-RBF), online sequential extreme learning machine (OS-ELM) and self-organizing fuzzy neural network (SOFNN) on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world regression problems. Simulation results demonstrate that the proposed FAOS-PFNN algorithm can achieve faster learning speed and more compact network structure with comparably high accuracy of approximation and generalization.

Multiple clusters echo state network for chaotic time series prediction

The chaotic time series prediction problem is considered. A novel type of cortex-like neural network model, i.e. multi-clusters echo state network model （MCESN）, regulated by a group of five growth-factors, is proposed. It is shown that characters of MCESN’ topology can be effectively determined by the growth-factors group; and that it is the MCESN possessing both small-world and scale-free properties of complex network that corresponds to the better prediction performance. In addition, Monte Carlo simulation experiments show that MCESN not only can be trained by easy algorithm, but also can achieve higher accuracy and less standard deviation prediction results than classical echo state networks.

A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks

In real life, information about the world is uncertain and imprecise. The cause of this uncertainty is due to: deficiencies on given information, the fuzzy nature of our perception of events and objects, and on the limitations of the models we use to explain the world. The development of new methods for dealing with information with uncertainty is crucial for solving real life problems. In this paper three interval type-2 fuzzy neural network (IT2FNN) architectures are proposed, with hybrid learning algorithm techniques (gradient descent backpropagation and gradient descent with adaptive learning rate backpropagation). At the antecedents layer, a interval type-2 fuzzy neuron (IT2FN) model is used, and in case of the consequents layer an interval type-1 fuzzy neuron model (IT1FN), in order to fuzzify the rule’s antecedents and consequents of an interval type-2 Takagi–Sugeno–Kang fuzzy inference system (IT2-TSK-FIS). IT2-TSK-FIS is integrated in an adaptive neural network, in order to take advantage the best of both models. This provides a high order intuitive mechanism for representing imperfect information by means of use of fuzzy If–Then rules, in addition to handling uncertainty and imprecision. On the other hand, neural networks are highly adaptable, with learning and generalization capabilities. Experimental results are divided in two kinds: in the first one a non-linear identification problem for control systems is simulated, here a comparative analysis of learning architectures IT2FNN and ANFIS is done. For the second kind, a non-linear Mackey–Glass chaotic time series prediction problem with uncertainty sources is studied. Finally, IT2FNN proved to be more efficient mechanism for modeling real-world problems.