Data-Driven Prediction of Nonlinear Systems via Self-Organizing Fuzzy Neural Network with Temporal-Spatial Feature

<div class="htmlview paragraph">This paper proposes a novel algorithm. This algorithm is called Self-Organizing Fuzzy Neural Network (SOFNN). SOFNN revolutionizes how researchers apply control theories, image/signal processing on control systems and other applications. In general, SOFNN is an identification technique that automatically initiates, builds and fine-tunes the required network parameters. SOFNN evaluates required structures without predefined parameters or expressions regarding systems. SOFNN sets out to learn and configure a system's characteristics. Self-constructing and self-tuning features enable SOFNN to handle complex, non-linear, and time-varying systems with higher accuracy, making systems identification easier. SOFNN constructs and fine-tunes the system parameter through two phases. The two phases are the construction and the parameter-tuning phase. The two phases run concurrently allowing SOFNN to identify systems on-line. Because of the self-construction feature, SOFNN has global clustering feature. The global clustering feature means the network is capable of covering every possible incident of the variables universe of discourse. Simulation results confirm the ability of SOFNN to capture both complexity and ambiguity.</div>

Fuzzy neural networks are hybrid systems that combine the theories of fuzzy logic and neural networks. By incorporating in these hybrid systems the ability to self-organize their network structure, self-organizing fuzzy neural networks (SOFNNs) are created. The SOFNNs have enhanced ability to identify adaptive models, mainly for representing nonlinear and time-varying complex systems, where little or insufficient expert knowledge is available to describe the underlying behavior. Problems that arise in these systems are large dimensions, time-varying characteristics, large amounts of data and noisy measurements, as well as the need for an interpretation of the resulting model. This chapter presents an algorithm for on-line identification of a self-organizing fuzzy neural network (SOFNN). The SOFNN provides a singleton or Takagi-Sugeno (TS) type fuzzy model. It therefore facilitates extracting fuzzy rules from the training data. The algorithm is formulated to guarantee the convergence of both the estimation error and the linear network parameters. It generates a fuzzy neural model with a high accuracy and compact structure. Superior performance of the algorithm is demonstrated through its applications for function approximation, system identification, and time-series prediction in both industrial and biological systems.

For real-time condition monitoring and fault detection of dual-lane controlled systems, reduced order models and long-term prediction are required. In this paper fault detection of reduced order model of nonlinear systems based on long-term prediction is proposed by using self-organizing fuzzy neural network (SOFNN). The main advantages of SOFNN are that, firstly, it is very user friendly as it can automatically determine the model structure and identify the model parameters without requiring the in-depth knowledge about fuzzy systems and neural networks; secondly, it provides the excellent modeling accuracy. Data gathered at an aero engine test-bed serve as the test vehicle to demonstrate the long-term prediction. A fault detection system is designed by using SOFNN. SOFNN is trained and used to simulate system dynamic characteristic. The simulation result is compared with actual output, and then fault error is drawn. The simulation result shows that, SOFNN can simulate the system more accurately, thus the change of residual error is easy to be detected. This assures the validity of this fault detection system.

This paper presents a number of enhancements to the self-organizing fuzzy neural network (SOFNN). Firstly, the SOFNN is described and a modification to the learning algorithm to improve computational efficiency is introduced. Secondly, a sensitivity analysis (SA) of the predefined SOFNN parameters is presented using electroencephalogram (EEG) data recorded from three subjects during left/right motor imagery-based brain-computer interface (BCI) experiments. This SA was carried out to determine if a general set of parameters could be used for predicting various non-stationary EEG time-series dynamics for multiple subjects. The SOFNN modifications significantly enhance computational efficiency and the SA results suggest that it may be possible to select a general set of parameters for different motor imagery-based EEG signals thus potentially enhancing the SOFNNs autonomy for application in a BCI.

In order to resolve the computational complexity for local map matching of hierarchical simultaneous localization and mapping (SLAM), a novel self-organizing fuzzy neural networks (SOFNN) based approach was proposed in this paper. The matching component for local maps in the hierarchical SLAM is realized by an SOFNN. A subset of signature elements included in a local map was chosen by a clustering algorithm, then was inputted to the SOFNN. The criteria to complete a local map, and the structure learning and parameter learning algorithms for our SOFNN were discussed.

A fuzzy neural network (FNN) is an effective learning system that combines neural network and fuzzy logic, which has achieved great success in nonlinear system modeling. However, when the input is practical complex data with external disturbance, the existing FNN cannot extract effective input features sufficiently, leading to unsatisfactory performances in learning speed and accuracy. It also fails to achieve a better generalization capability because of its fixed structure size (the number of rule neurons). In this article, an efficient self-organizing FNN (SOFNN) with incremental deep pretraining (IDPT), called IDPT-SOFNN, is developed to overcome these shortcomings. First, IDPT is designed to extract effective features and consider them as the input of the SOFNN. Different from the existing pretraining, the self-growing structure of IDPT improves pretraining efficiency with a more compact structure. Second, the SOFNN can dynamically add and delete neurons according to the current error and error-reduction rate. In this case, it can obtain better modeling performance with a more compact structure as well. Third, as a novel hybrid model with the cascade dual-self-organizing algorithm, the IDPT-SOFNN combines the advantage of IDPT and SOFNN. Moreover, the convergence and stability are analyzed. Finally, simulation studies and comparisons demonstrate that the proposed IDPT-SOFNN has better performances than its peers in learning speed, accuracy, and generalization capability.

This paper introduces a number of modifications to the learning algorithm of the self-organizing fuzzy neural network (SOFNN) to improve computational efficiency. It is shown that the modified SOFNN favorably compares to other evolving fuzzy systems in terms of accuracy and structural complexity. An analysis of the SOFNN's effectiveness when applied in an electroencephalogram (EEG)-based brain-computer interface (BCI) involving the neural-time-series-prediction-preprocessing (NTSPP) framework is also presented, where a sensitivity analysis (SA) of the SOFNN hyperparameters was performed using EEG data recorded from three subjects during left/right-motor-imagery-based BCI experiments. The aim of this one-time SA was to eliminate the need to choose subject- and signal-specific hyperparameters for the SOFNN and thus apply the SOFNN in the NTSPP framework as a parameterless self-organizing framework for EEG preprocessing. The results indicate that a general set of NTSPP parameters chosen via the SA provide the best results when tested in a BCI system. Therefore, with this general set of SOFNN parameters and its self-organizing structure, in conjunction with parameterless feature extraction and linear discriminant classification, a fully parameterless BCI that lends itself well to autonomous adaptation is realizable.

Short-term load forecasting has become increasingly important since the rise of the competitive energy markets and has become one of the major areas of research in recent years. Toward to this important topic, this paper proposes a new approach- the self-organizing fuzzy neural network (SOFNN) modeling method for the short-term load forecasting. The main advantages of this approach are that, firstly, it is very user friendly as SOFNN can automatically determine the model structure and identify the model parameters without requiring the in-depth knowledge about fuzzy systems and neural networks; secondly, it provides the excellent forecasting accuracy. Applying this approach based on the real data, the achieved average mean absolute percentage error (MAPE) for load forecasting is less than 1%.

This paper proposes an intelligent tracking control for the chaotic systems via backstepping approach. The intelligent tracking control system is comprised of a neural controller and a robust controller. The neural controller containing a self-organizing fuzzy neural network (SOFNN) identifier is the principal controller, and the robust controller is designed to dispel the effect of minimum approximation error introduced by the SOFNN identifier. The SOFNN identifier is used to online estimate the chaotic dynamic function with structure and parameter learning phases of fuzzy neural network. Finally, simulation results verify that the proposed intelligent tracking control can achieve favorable tracking performance.

This paper describes a modification to the learning algorithm and architecture of the self-organizing fuzzy neural network (SOFNN) to improve learning ability. Previously the SOFNN's computational efficiency was improved using a new method of checking the network structure after it has been modified. Instead of testing the entire structure every time it has been modified, a record is kept of each neuron's firing strength for all data previously clustered by the network. This record is updated as training progresses and is used to reduce the computational load of checking network structure changes, to ensure performance degradation does not occur, resulting in significantly reduced training times. To exploit the temporal information contained in the record of saved firing strengths, a new architecture of the SOFNN is proposed in this paper where recurrent feedback connections are added to neurons in layer three of the structure. Recurrent connections allow the network to learn the temporal information from the data and, in contrast to pure feed forward architectures, which exhibit static input-output behavior in advance, recurrent models are able to store information from the past (e.g., past measurements of the time-series) and are therefore better suited to analyzing dynamic systems. Each recurrent feedback connection includes a weight which must be learned. In this work a learning approach is proposed where the recurrent feedback weight is updated online (not iteratively) and proportional to the aggregate firing activity of each fuzzy neuron. It is shown that this modification, which conforms to the requirements for autonomy and has no additional hyperparameters, can significantly improve the performance of the SOFNN's prediction capacity under certain constraints

Reinforcement learning (RL) plays an important role in artificial intelligent (AI) realization. It is a trial-and-error machine learning algorithm for agent's adaptive behavior acquisition in unknown environments. In this paper, an improved fuzzy neural network (iFNN) is proposed with shortcut connection concept for RL. iFNN is based on a self-organizing fuzzy neural network (SOFNN) which is a data-driven fuzzy inference system for RL, and iFNN changes the structure of SOFNN by adding the input vector to units of the middle layer, i.e., fuzzy rules. Furthermore, iFNN is also adopted into multi-layered fuzzy neural network (MLFNN) which is a variation of SOFNN with a deep structure. Goal-navigated exploration experiment results showed the effectiveness of the proposed iFNN.

In this work the self-organizing fuzzy neural network (SOFNN) is employed to create an accurate and easily calibrated approach to multiple-step-ahead prediction for the NN5 forecasting competition 2008. The competition dataset consists of 111 daily empirical time series of cash-machine withdrawals. The objective for the competition was to forecast future transactions up to 56 days ahead with the highest prediction accuracy using a single methodology. The SOFNN is a highly efficient and accurate algorithm for time series-prediction which learns from data incrementally and can autonomously adapt its structure in the learning process to cope with drifts in the data dynamics. It can also modify its architecture autonomously to suit different prediction horizons, embedding dimensions and time lags. Standard neural networks(NNs) and autoregressive(AR) models are employed as benchmarks for comparison. It is shown through a statistical analysis of the results, that the SOFNN significantly outperforms the NN and AR methods.

As a pivotal role in the control, optimization, and monitoring of contemporary industrial processes, soft sensors are frequently employed in the prediction of key quality variables. To achieve accurate prediction of key quality variables in industrial processes, a soft sensor modeling method based on the self-organizing fuzzy neural network with the clustering, merging, and splitting scheme (SOFNN-CMS) is proposed. First, the supervised fuzzy C-means clustering algorithm is proposed to identify the appropriate initial center and width of the fuzzy neural network, obtaining appropriate initial fuzzy rules. Then, a neuron merging and splitting strategy is designed to adjust the structure of the fuzzy neural network, by merging and splitting the hidden neurons according to the distance of clusters, increasing the adaptability of the fuzzy neural network. Besides, to accelerate the convergence of estimation errors, an improved Levenberg Marquardt algorithm is utilized to update neural network parameters in the training phase, realizing the soft sensor modeling of key quality variables. The effectiveness of the proposed SOFNN-CMS neural network is demonstrated on two benchmark problems and an industrial debutanizer column. Finally, the experiments showcase that the proposed SOFNN-CMS neural network can obtain better soft sensor modeling performance with a compact structure.

An approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network

This chapter describes a number of modifications to the learning algorithm and architecture of the self-organizing fuzzy neural network (SOFNN) to improve its computational efficiency and learning ability. To improve the SOFNN’s computational efficiency, a new method of checking the network structure after it has been modified is proposed. Instead of testing the entire structure every time it has been modified, a record is kept of each neuron’s firing strength for all data previously clustered by the network. This record is updated as training progresses and is used to reduce the computational load of checking network structure changes, to ensure performance degradation does not occur, resulting in significantly reduced training times. It is shown that the modified SOFNN compares favorably to other evolving fuzzy systems in terms of accuracy and structural complexity. In addition, a new architecture of the SOFNN is proposed where recurrent feedback connections are added to neurons in layer three of the structure. Recurrent connections allow the network to learn the temporal information from data and, in contrast to pure feed forward architectures which exhibit static input-output behavior in advance, recurrent models are able to store information from the past (e.g., past measurements of the time-series) and are therefore better suited to analyzing dynamic systems. Each recurrent feedback connection includes a weight which must be learned. In this work a learning approach is proposed where the recurrent feedback weight is updated online (not iteratively) and proportional to the aggregate firing activity of each fuzzy neuron. It is shown that this modification can significantly improve the performance of the SOFNN’s prediction capacity under certain constraints.