Structure Learning Phase Research Articles

PSO-K2PC: Bayesian structure learning using optimized K2 algorithm for parents-children detection

Bayesian networks, revered for their adeptness in modeling uncertainty and predicting outcomes, encounter a formidable hurdle during the structure learning phase – an NP-hard problem, posing insurmountable computational challenges for large networks. To surmount this barrier and advance the field, we propose an innovative optimization of the K2PC algorithm for Bayesian network structure learning. Derived from the popular K2 algorithm, our novel optimization ingeniously tackles K2PC's vulnerability to predetermined node order. Leveraging the power of a particle swarm optimization algorithm, we adeptly seek the optimal node ordering, yielding exceptional results. Through rigorous evaluations on benchmark networks, our proposed method surpasses prior approaches in structure difference and accuracy, affirming its potential as a promising avenue for Bayesian network structure learning in large, complex networks. We posit that our novel approach constitutes an important advance in the field of Bayesian network structure learning, with the potential to stimulate additional progress through further scientific investigation.

A Multi-Agent Architecture for the Design of Hierarchical Interval Type-2 Beta Fuzzy System

This paper presents a new methodology for building and evolving hierarchical fuzzy systems. For the system design, a tree-based encoding method is adopted to hierarchically link low-dimensional fuzzy systems. Such tree structural representation has by nature a flexible design offering more adjustable and modifiable structures. The proposed hierarchical structure employs a type-2 beta fuzzy system to cope with the faced uncertainties, and the resulting system is called the hierarchical interval type-2 beta fuzzy system (HT2BFS). For the system optimization, two main tasks of structure learning and parameter tuning are applied. The structure learning phase aims to evolve and learn the structures of a population of the HT2BFS in a multi-objective context taking into account the optimization of both the accuracy and the interpretability metrics. The parameter tuning phase is applied to refine and adjust the parameters of the system. To accomplish these two tasks in the most optimal way, we further employ a multi-agent architecture to provide both a distributed and a cooperative management of the optimization tasks. Agents are divided into two different types based on their functions: a structure agent and a parameter agent. The main function of the structure agent is to perform a multi-objective evolutionary structure learning step by means of the multi-objective immune programming algorithm. The parameter agents have the function of managing different hierarchical structures simultaneously to refine their parameters by means of the hybrid harmony search algorithm. In this architecture, agents use cooperation and communication concepts to create high-performance HT2BFSs. The performance of the proposed system is evaluated by several comparisons with various state-of-the-art approaches on noise-free and noisy time series prediction datasets and regression problems. The results clearly demonstrate a great improvement in accuracy rate, convergence speed, and the number of used rules as compared to other existing approaches.

Efficient Approach for RLS Type Learning in TSK Neural Fuzzy Systems.

This paper presents an efficient approach for the use of recursive least square (RLS) learning algorithm in Takagi-Sugeno-Kang neural fuzzy systems. In the use of RLS, reduced covariance matrix, of which the off-diagonal blocks defining the correlation between rules are set to zeros, may be employed to reduce computational burden. However, as reported in the literature, the performance of such an approach is slightly worse than that of using the full covariance matrix. In this paper, we proposed a so-called enhanced local learning concept in which a threshold is considered to stop learning for those less fired rules. It can be found from our experiments that the proposed approach can have better performances than that of using the full covariance matrix. Enhanced local learning method can be more active on the structure learning phase. Thus, the method not only can stop the update for insufficiently fired rules to reduce disturbances in self-constructing neural fuzzy inference network but also raises the learning speed on structure learning phase by using a large backpropagation learning constant.

Neuro-fuzzy inverse model control structure of robotic manipulators utilized for physiotherapy applications

This paper presents a new neuro-fuzzy controller for robot manipulators. First, an inductive learning technique is applied to generate the required inverse modeling rules from input/output data recorded in the off-line structure learning phase. Second, a fully differentiable fuzzy neural network is developed to construct the inverse dynamics part of the controller for the online parameter learning phase. Finally, a fuzzy-PID-like incremental controller was employed as Feedback servo controller. The proposed control system was tested using dynamic model of a six-axis industrial robot. The control system showed good results compared to the conventional PID individual joint controller.

A Self-Organizing Fuzzy Neural Network Based on a Growing-and-Pruning Algorithm

A novel growing-and-pruning (GP) approach is proposed, which optimizes the structure of a fuzzy neural network (FNN). This GP-FNN is based on radial basis function neurons, which have center and width vectors. The structure-learning phase and the parameter-training phase are performed concurrently. The structure-learning approach relies on the sensitivity analysis of the output. A set of fuzzy rules can be inserted or reduced during the learning process. The parameter-training algorithm is implemented using a supervised gradient decent method. The convergence of the GP-FNN-learning process is also discussed in this paper. The proposed method effectively generates a fuzzy neural model with a highly accurate and compact structure. Simulation results demonstrate that the proposed GP-FNN has a self-organizing ability, which can determine the structure and parameters of the FNN automatically. The algorithm performs better than some other existing self-organizing FNN algorithms.

Observer-based decentralized fuzzy neural sliding mode control for interconnected unknown chaotic systems via network structure adaptation

In this paper, an observer-based fuzzy neural sliding mode control (OFNSMC) scheme for interconnected unknown chaotic systems is developed. The OFNSMC system is composed of a computation controller and a robust controller. The computation controller containing a self-structuring fuzzy neural network (SFNN) identifier is the principle controller, and the robust controller is designed to achieve L 2 tracking performance. The SFNN identifier uses the structure and parameter learning phases to perform the estimation of the interconnected unknown chaotic system dynamics. The structure learning phase consists of the growing of membership functions, the splitting of fuzzy rules and the pruning of fuzzy rules, and thus the SFNN identifier can avoid the time-consuming trial-and-error tuning procedure for determining the network structure of fuzzy neural network. The total states of the interconnected chaotic systems are not assumed to be available for measurement. Also, the unknown nonlinearities of the interconnected chaotic systems are not restricted to the systems output only. To demonstrate the effectiveness of the proposed method, simulation results are illustrated in this paper.

Research on structure dynamic neural networks

A model of structure dynamic neural network, which simulates the learning skills such as human beings and animals, is proposed in this paper. This model contains two main steps: 1) the structure learning phase possesses the ability of online generation and ensures the number of the neural nodes of the neural network; 2) the parameter learning phase adjusts the interconnection weights of neural network to achieve favourable approximation performance. The structure learning algorithm consists of growing and pruning methods, and then, the Lyapunov stability theory is used to analyse the stability of this new algorithm. Finally, this new dynamic neural network is used to track the non-linear functions; simulation results show that this new algorithm can achieve favourable performance.

Identification-based chaos control via backstepping design using self-organizing fuzzy neural networks

This paper proposes an identification-based adaptive backstepping control (IABC) for the chaotic systems. The IABC system is comprised of a neural backstepping controller and a robust compensation controller. The neural backstepping controller containing a self-organizing fuzzy neural network (SOFNN) identifier is the principal controller, and the robust compensation controller is designed to dispel the effect of minimum approximation error introduced by the SOFNN identifier. The SOFNN identifier is used to online estimate the chaotic dynamic function with structure and parameter learning phases of fuzzy neural network. The structure learning phase consists of the growing and pruning of fuzzy rules; thus the SOFNN identifier can avoid the time-consuming trial-and-error tuning procedure for determining the neural structure of fuzzy neural network. The parameter learning phase adjusts the interconnection weights of neural network to achieve favorable approximation performance. Finally, simulation results verify that the proposed IABC can achieve favorable tracking performance.

Self-Organizing Adaptive Fuzzy Neural Control for a Class of Nonlinear Systems

This paper proposes a self-organizing adaptive fuzzy neural control (SAFNC) via sliding-mode approach for a class of nonlinear systems. The proposed SAFNC system is comprised of a computation controller and a supervisory controller. The computation controller including a self-organizing fuzzy neural network (SOFNN) identifier is the principal controller. The SOFNN identifier is used to online estimate the controlled system dynamics with the structure and parameter learning phases of fuzzy neural network (FNN), simultaneously. The structure learning phase possesses the ability of online generation and elimination of fuzzy rules to achieve optimal neural structure, and the parameter learning phase adjusts the interconnection weights of neural network to achieve favorable approximation performance. The supervisory controller is used to achieve the L2-norm bound tracking performance with a desired attenuation level. Moreover, all the parameter learning algorithms are derived based on Lyapunov function candidate, thus the system stability can be guaranteed. Finally, simulation results show that the SAFNC can achieve favorable tracking performances.

Robust self-organizing fuzzy-neural control using asymmetric Gaussian membership functions

A robust self-organizing fuzzy-neural control (RSOFNC) system is proposed in this paper. The RSOFNC system is comprised of a self-structuring fuzzy neural network (SFNN) controller and a robust controller. The SFNN controller is the principal controller and the robust controller is designed to achieve L2 tracking performance. In the SFNN controller design, a SFNN with the asymmetric Gaussian membership functions is used to online approximate an ideal controller via the structure and parameter learning phases. The structure learning phase consists of the growing of membership functions and the pruning of fuzzy rules, and thus the SFNN can avoid the time-consuming trial-and-error tuning procedure for determining the network structure of fuzzy neural network. Finally, the proposed RSOFNC system is applied to control a second-order chaotic system. The simulation results show that the proposed RSOFNC system can achieve favorable tracking performance.

MULTIVARIABLE DYNAMIC SYSTEMS MODELING AND CONTROL USING A NEW PARTICLE SWARM ALGORITHM-LOCAL MODEL NETWORK

- This paper introduces the Particle Swarm Algorithm (PSA)-based Local Model network (LMN) for modeling and controlling dynamic systems. Structurally, the proposed PSA-LMN merges the fuzzy set theory and wavelets in a unified form. Learning this network comprises two phases, structure learning phase and parameters learning phase. The former is performed using the Adaptive Resonance Theory (ART) algorithm while the latter is performed using the PSA. The PSA is employed to optimize parameters of the fuzzy sets, the wavelets and the free weights of the proposed LMN. Two simulation nonlinear plants are used to test the soundness of the proposed network; one is a single input single output nonlinear plant and the other is multi-variable medical plant. The latter is employed to test the proposed network in control purposes compared with Genetic Algorithm (GA)-based LMN. Better results were obtained using the proposed PSA-based LMN.

NEURO-FUZZY MODELLING AND CONTROL OF ROBOT MANIPULATORS FOR TRAJECTORY TRACKING

This paper presents a new neuro-fuzzy controller for robot manipulators. First, an inductive learning technique is applied to generate the required modelling rules from input/output measurements recorded in the off-line structure learning phase. Second, a fully differentiable fuzzy neural network is developed to construct the inverse dynamics part of the controller for the on-line parameter learning phase. Finally, a fuzzy-PID-like incremental controller was employed as feedback servo-controller. The proposed control system was tested using dynamic model of a six-axis industrial robot. The control system showed good results compared to the conventional-PID individual joint controller.

An adaptive neural fuzzy filter and its applications

A new kind of nonlinear adaptive filter, the adaptive neural fuzzy filter (ANFF), based upon a neural network's learning ability and fuzzy if-then rule structure, is proposed in this paper. The ANFF is inherently a feedforward multilayered connectionist network which can learn by itself according to numerical training data or expert knowledge represented by fuzzy if-then rules. The adaptation here includes the construction of fuzzy if-then rules (structure learning), and the tuning of the free parameters of membership functions (parameter learning). In the structure learning phase, fuzzy rules are found based on the matching of input-output clusters. In the parameter learning phase, a backpropagation-like adaptation algorithm is developed to minimize the output error. There are no hidden nodes (i.e., no membership functions and fuzzy rules) initially, and both the structure learning and parameter learning are performed concurrently as the adaptation proceeds. However, if some linguistic information about the design of the filter is available, such knowledge can be put into the ANFF to form an initial structure with hidden nodes. Two major advantages of the ANFF can thus be seen: 1) a priori knowledge can be incorporated into the ANFF which makes the fusion of numerical data and linguistic information in the filter possible; and 2) no predetermination, like the number of hidden nodes, must be given, since the ANFF can find its optimal structure and parameters automatically.