Fuzzy Polynomial Neural Networks Research Articles

A self-organizing deep network architecture designed based on LSTM network via elitism-driven roulette-wheel selection for time-series forecasting

In this study, we propose a new self-organizing deep network architecture of fuzzy polynomial neural networks (FPNN) based on Fuzzy rule-based Polynomial Neurons (FPNs) and a long short-term memory (LSTM) network to solve the task of time-series forecasting. In the existing regression model based on polynomial neural networks (PNN), it is difficult to achieve high quality performance when predicting time series data, because this model lacks the ability to extract temporal and spatial information. Therefore, we propose a new architecture consisting of one LSTM (temporal) layer and several fuzzy polynomial (spatial) layers to overcome the above-mentioned shortcomings of PNN and enhance its predictive ability to approximate the data. The temporal layer consists of LSTM neurons that have inherently strong modeling capabilities to learn sequential information. The spatial layers are composed of Rule-based Polynomial Neurons (FPNs) that can effectively reflect the complex nonlinear structure found in the input space and granulate it using of the Fuzzy C-Means (FCM) clustering method. An elitism-driven roulette-wheel selection (E_RWS) is used to select appropriate neurons. E_RWS not only ensures that the neuron with the strongest fitting ability is selected but also increases the diversity of candidate neurons. According to the experimental results, the proposed model has a high prediction performance and outperforms many state-of-the-art prediction methods when applied to the real-world time-series.

Selforganizing Hybrid Fuzzy Polynomial Neural Network Classifier Driven Through Dynamically Adaptive Structure and Compound Regularization Technique

Selforganizing Hybrid Fuzzy Polynomial Neural Network Classifier Driven Through Dynamically Adaptive Structure and Compound Regularization Technique

Selforganizing Hybrid Fuzzy Polynomial Neural Network Classifier Driven Through Dynamically Adaptive Structure and Compound Regularization Technique

Selforganizing Hybrid Fuzzy Polynomial Neural Network Classifier Driven Through Dynamically Adaptive Structure and Compound Regularization Technique

Design of progressive fuzzy polynomial neural networks through gated recurrent unit structure and correlation/probabilistic selection strategies

This study focuses on two critical design aspects of a progressive fuzzy polynomial neural network (PFPNN): the influence of the gated recurrent unit (GRU) structure and the implementation of fitness-based candidate neuron selection (FCNS) through two probabilistic strategies. The primary objectives are to enhance modeling accuracy and to reduce the computational load associated with nonlinear regression tasks. Compared with the existing fuzzy rule-based modeling architecture, the proposed dynamic model consists of the GRU structure and the hybrid fuzzy polynomial architecture. In the initial two layers of the PFPNN, we introduce three types of polynomial and fuzzy rules into the GRU neurons (GNs) and fuzzy polynomial neurons (FPNs), which can effectively reveal potential complex relationships in the data space. The synergy of the FCNS strategies and the ℓ2 regularization learning method is to design a progressive regression model adept at melding the GRU structure with a self-organizing architecture. The proposed GRU structure and polynomial-based neurons significantly improve the modeling accuracy for time-series datasets. The rational utilization of FCNS strategies can reinforce the network structure and discover the potential performance of neurons of the network. Furthermore, the inclusion of ℓ2 norm regularization provides additional stability to the proposed model and mitigates the overfitting issue commonly encountered in many existing learning methods. We validated the proposed neural networks using six time-series, four machine learning, and two real-world datasets. The PFPNN outperformed other models in the comparison studies in 83.3% of the datasets, emphasizing its superiority in terms of developing a stable deep structure from diverse candidate neurons and reducing computational overhead.

A new development of ANFIS–GMDH optimized by PSO to predict pile bearing capacity based on experimental datasets

Prediction of ultimate pile bearing capacity with the aid of field experimental results through artificial intelligence (AI) techniques is one of the most significant and complicated problem in pile analysis and design. The aim of this research is to develop a new AI predictive models for predicting pile bearing capacity. The first predictive model was developed based on the combination of adaptive neuro-fuzzy inference system (ANFIS) and group method of data handling (GMDH) structure optimized by particle swarm optimization (PSO) algorithm called as ANFIS–GMDH–PSO model; the second model introduced as fuzzy polynomial neural network type group method of data handling (FPNN–GMDH) model. A database consists of different piles property and soil characteristics, collected from literature including CPT and pile loading test results which applied for training and testing process of developed models. Also a common artificial neural network (ANN) model was applied as a reference model for comparing and verifying among hybrid developed models for prediction. The modelling results indicated that improved ANFIS–GMDH model achieved relatively higher performance compared to ANN and FPNN–GMDH models in terms of accuracy and reliability level based on standard statistical performance indices such as coefficient of correlation (R), mean square error, root mean square error and error standard deviation values.

유전자 프로그래밍을 이용한 일반화된 퍼지 다항식 뉴럴 네트워크 최적화

유전자 프로그래밍을 이용한 일반화된 퍼지 다항식 뉴럴 네트워크 최적화

Hybrid fuzzy polynomial neural networks with the aid of weighted fuzzy clustering method and fuzzy polynomial neurons

It is well-known that any nonlinear complex system can be modeled by using a collection of "if źthen" fuzzy rules. In spite of a number of successful models reported in the literature, there are still two open issues: (1) one is not able to reflect the heterogeneous partition of the input space; (2) it becomes very difficult to deal effectively with high dimensionality of the problem (data). In this study, we present a parallel fuzzy polynomial neural networks (PFPNNs) with the aid of heterogeneous partition of the input space. Like fuzzy rules encountered in fuzzy models, the PFPNNs comprises a collection of premise and consequent parts. In the design of the premise part of the rule a weighted fuzzy clustering method is used not only to realize a nonuniform partition of the input space but to overcome a possible curse dimensionality. While in the design of consequent part, fuzzy polynomial neural networks are exploited to construct optimal local models (high order polynomials) that describe the relationship between the input variables and output variable within some local region of the input space. Two types of information granulation-based fuzzy polynomial neurons are developed for FPNNs. Particle swarm optimization (PSO) is employed to adjust the design parameters of parallel fuzzy polynomial neural networks. To evaluate the performance of PFPNNs a series of experiments based on several benchmarks are included. A comparative analysis demonstrates that the proposed model comes with higher accuracy and generalization capabilities in comparison with some previous models reported in the literature.

A design of granular-oriented self-organizing hybrid fuzzy polynomial neural networks

In this study, we introduce a new design methodology of granular-oriented self-organizing Hybrid Fuzzy polynomial neural networks (HFPNN) that is based on multi-layer perceptron with context-based polynomial neurons (CPNs) or polynomial neurons (PNs). In contrast to the typical architectures encountered in polynomial neural networks (PNN), our main objective is to develop a design strategy of HFPNN as follows: (a) The first layer of the proposed network consists of context-based polynomial neuron (CPN). Here CPN is fully reflective of the structure encountered in numeric data which are granulated with the aid of context-based fuzzy c-means (C-FCM) clustering method. The context-based clustering supporting the design of information granules is completed in the space of the input data (input variables) while the formation of the clusters here is guided by a collection of some predefined fuzzy sets (so-called contexts) specified in the output space. (b) The proposed design procedure being applied to each layer of HFPNN leads to the selection of the preferred nodes of the network (CPNs or PNs) whose local characteristics (such as the number of contexts, the number of clusters, a collection of the specific subset of input variables, and the order of the polynomial) can be easily adjusted. These options contribute to the flexibility as well as simplicity and compactness of the resulting architecture of the network. For the evaluation of the performance of the proposed HFPNN, we use well-known machine learning data coming from the machine learning repository.

Genetic Optimization of Fuzzy Polynomial Neural Networks

In this paper, we introduce a new topology of fuzzy-neural networks-fuzzy-set-based polynomial neural networks (FSPNNs). The two underlying design mechanisms of such networks involve genetic optimization and information granulation (IG). The resulting constructs come in the form of fuzzy polynomial neural networks with fuzzy-set-based polynomial neurons, regarded as their generic processing elements. First, we introduce a comprehensive design methodology using which we determine the optimal structure of the FSPNNs. This methodology hinges on the extended group method of data handling and fuzzy-set-based rules. It concerns the optimization of the FSPNN-related parameters such as the number of input variables, the order of the polynomial, the number of membership functions, and a collection of a specific subset of input variables realized through the mechanism of genetic optimization. Second, the fuzzy rules used in the networks exploit a notion of information granules defined over a system's variables and formed through the process of IG. This granulation is realized with the aid of the hard C-means clustering algorithm. The performance of the network is quantified through experimentation in which we use a number of modeling benchmarks already experimented with in the realm of fuzzy or neurofuzzy modeling.

IG-based genetically optimized fuzzy polynomial neural networks with fuzzy set-based polynomial neurons

In this study, we introduce and investigate a new topology of fuzzy-neural networks—fuzzy polynomial neural networks (FPNN) that is based on a genetically optimized multiplayer perceptron with fuzzy set-based polynomial neurons (FSPNs). We also develop a comprehensive design methodology involving mechanisms of genetic optimization and information granulation. In the sequel, the genetically optimized FPNN (gFPNN) is formed with the use of fuzzy set-based polynomial neurons (FSPNs) composed of fuzzy set-based rules through the process of information granulation. This granulation is realized with the aid of the C-means clustering (C-Means). The design procedure applied in the construction of each layer of an FPNN deals with its structural optimization involving the selection of the most suitable nodes (or FSPNs) with specific local characteristics (such as the number of input variable, the order of the polynomial, the number of membership functions, and a collection of specific subset of input variables) and address main aspects of parametric optimization. Along this line, two general optimization mechanisms are explored. The structural optimization is realized via genetic algorithms (GAs) and HCM method whereas in case of the parametric optimization we proceed with a standard least square estimation (learning). Through the consecutive process of structural and parametric optimization, a flexible neural network is generated in a dynamic fashion. The performance of the designed networks is quantified through experimentation where we use two modeling benchmarks already commonly utilized within the area of fuzzy or neurofuzzy modeling.

Fuzzy polynomial neural networks for approximation of the compressive strength of concrete

The main purpose of this paper is to develop fuzzy polynomial neural networks (FPNN) to predict the compressive strength of concrete. Two different architectures of FPNN are addressed (Type1 and Type2) and their training methods are discussed. In this research, the proposed FPNN is a combination of fuzzy neural networks (FNNs) and polynomial neural networks (PNNs). Here, while the FNN demonstrates the premises (If-Part) of the fuzzy model, the PNN is implemented as its consequence (Then-Part). To enhance the performance of the network, back propagation (BP), and list square error (LSE) algorithms are utilized for the tuning of the system. Six different FPNN architectures are constructed, trained, and tested using the experimental data of 458 different concrete mix-designs collected from three distinct sources. The data are organized in a format of six input parameters of concrete ingredients and one output as 28-day compressive strength of the mix-design. Using root means square (RMS) and correlation factors (CFs), the models are evaluated and compared with training and testing data pairs. The results show that FPNN-Type1 has strong potential as a feasible tool for prediction of the compressive strength of concrete mix-design. However, the FPNN-Type2 is recognized as unfeasible model to this purpose.

Evolutionary design of hybrid self-organizing fuzzy polynomial neural networks with the aid of information granulation

We introduce a new architecture of information granulation-based and genetically optimized Hybrid Self-Organizing Fuzzy Polynomial Neural Networks (HSOFPNN). Such networks are based on genetically optimized multi-layer perceptrons. We develop their comprehensive design methodology involving mechanisms of genetic optimization and information granulation. The architecture of the resulting HSOFPNN combines fuzzy polynomial neurons (FPNs) that are located at the first layer of the network with polynomial neurons (PNs) forming the remaining layers of the network. The augmented version of the HSOFPNN, ''IG_gHSOFPNN'', for brief, embraces the concept of information granulation and subsequently exhibits higher level of flexibility and leads to simpler architectures and rapid convergence speed to optimal structure in comparison with the HSOFPNNs and SOFPNNs. The GA-based design procedure being applied at each layer of HSOFPNN leads to the selection of preferred nodes of the network (FPNs or PNs) whose local characteristics (such as the number of input variables, the order of the polynomial, a collection of the specific subset of input variables, the number of membership functions for each input variable, and the type of membership function) can be easily adjusted. In the sequel, two general optimization mechanisms are explored. The structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is afterwards carried out in the setting of a standard least square method-based learning. The obtained results demonstrate a superiority of the proposed networks over the existing fuzzy and neural models.

A New Approach to the Development of Genetically Optimized Multilayer Fuzzy Polynomial Neural Networks

In this paper, the authors propose and investigate a new category of neurofuzzy networks-fuzzy polynomial neural networks (FPNNs)-and develop a comprehensive design methodology involving mechanisms of genetic optimization and, in particular, genetic algorithms (GAs). The conventional FPNNs developed so far are based on mechanisms of self-organization and evolutionary optimization. The design of the network exploits the extended group method of data handling, with some essential parameters of the network being provided by the designer and kept fixed throughout the overall development process. This restriction may hamper a possibility of producing an optimal architecture of the model. The design proposed in this paper addresses this issue. The augmented genetically optimized FPNN (gFPNN) results in a structurally optimized structure and comes with a higher level of flexibility in comparison with the one encountered in the conventional FPNN. The GA-based design procedure that is applied to each layer of FPNN leads to the selection of the preferred nodes (or fuzzy polynomial neurons) available within the FPNN. In the sequel, two general optimization mechanisms are explored. First, the structural optimization is realized via GAs, whereas the ensuing, detailed parametric optimization is carried out in the setting of a standard least-square-method-based learning. The performance of gFPNN is quantified through experimentation where a number of modeling benchmarks are being used, i.e., synthetic and experimental data already experimented within fuzzy or neurofuzzy modeling. The obtained results demonstrate superiority of the proposed networks over the existing fuzzy and neural models

Genetically Optimized Fuzzy Polynomial Neural Network and Its Application to Multi-variable Software Process

In this paper, we propose a new architecture of Fuzzy Polynomial Neural Networks(FPNN) by means of genetically optimized Fuzzy Polynomial Neuron(FPN) and discuss its comprehensive design methodology involving mechanisms of genetic optimization, especially Genetic Algorithms(GAs). The conventional FPNN developed so far are based on mechanisms of self-organization and evolutionary optimization. The design of the network exploits the extended Group Method of Data Handling(GMDH) with some essential parameters of the network being provided by the designer and kept fixed throughout the overall development process. This restriction may hamper a possibility of producing an optimal architecture of the model. The proposed FPNN gives rise to a structurally optimized network and comes with a substantial level of flexibility in comparison to the one we encounter in conventional FPNNs. It is shown that the proposed advanced genetic algorithms based Fuzzy Polynomial Neural Networks is more useful and effective than the existing models for nonlinear process. We experimented with Medical Imaging System(MIS) dataset to evaluate the performance of the proposed model.

Genetically optimized fuzzy polynomial neural networks

In this paper, we introduce a new topology of fuzzy polynomial neural networks (FPNNs) that is based on a genetically optimized multilayer perceptron with fuzzy polynomial neurons (FPNs). The study offers a comprehensive design methodology involving mechanisms of genetic optimization, especially those exploiting genetic algorithms (GAs). Let us recall that the design of the FPNNs uses an extended group method of data handling (GMDH) and uses a fixed scheme of fuzzy inference (such as simplified, linear, and regression polynomial fuzzy inference) in each FPN of the network. It also considers a fixed number of input nodes (as being selected in advance by a network designer) at FPNs (or nodes) located in each layer. However such design process does not guarantee that the resulting FPNs will always result in an optimal networks architecture. Here, the development of the FPNN gives rise to a structurally optimized topology and comes with a substantial level of flexibility which becomes apparent when contrasted with the one we encounter in the conventional FPNNs. The design of each layer of the FPNN deals with its structural optimization involving a selection of preferred nodes (or FPNs) with specific local characteristics (such as the number of input variables, the order of the polynomial forming a consequent part of fuzzy rules and a collection of the specific subset of input variables) and addresses detailed aspects of parametric optimization. Along this line, two general optimization mechanisms are explored. The structural optimization is realized via GAs. In case of the parametric optimization we proceed with a standard least square method-based learning. Through the consecutive process of such structural and parametric optimization, an optimized and flexible fuzzy neural network becomes generated in a dynamic fashion. To evaluate the performance of the genetically optimized FPNN (gFPNN), we experimented with two time series data (gas furnace and chaotic time series) as well as some synthetic data. A comparative analysis reveals that the proposed FPNN exhibits higher accuracy and superb predictive capability in comparison to some previous models available in the literature.

Genetically Optimized Multi-Layer Fuzzy Polynomial Neural Networks: Analysis and Design

In this study, we introduce a new category of neurofuzzy networks – Fuzzy Polynomial Neural Networks and develop a comprehensive design methodology involving mechanisms of genetic optimization, and genetic algorithms, in particular. The augmented genetically optimized FPNN (referred to as gFPNN) is a structurally optimized architecture which comes with a higher level of flexibility in comparison to the one we have encountered in the conventional FPNN. The GA-based design procedure being applied to each layer of FPNN leads to the selection of preferred nodes (or FPNs) available within the FPNN. In the sequel, two general optimization mechanisms are explored. First, the structural optimization is realized via GAs whereas for the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The performance of the gFPNN is quantified through experimentation where we use a number of modeling benchmarks – synthetic and experimental data already experimented with in fuzzy or neurofuzzy modeling.

Genetically optimized fuzzy polynomial neural networks with fuzzy set-based polynomial neurons

In this paper, we propose and investigate a new category of neurofuzzy networks—fuzzy polynomial neural networks (FPNN) endowed with fuzzy set-based polynomial neurons (FSPNs) We develop a comprehensive design methodology involving mechanisms of genetic optimization, and genetic algorithms (GAs) in particular. The conventional FPNNs developed so far are based on the mechanisms of self-organization, fuzzy neurocomputing, and evolutionary optimization. The design of the network exploits the FSPNs as well as the extended group method of data handling (GMDH). Let us stress that in the previous development strategies some essential parameters of the networks (such as the number of input variables, the order of the polynomial, the number of membership functions, and a collection of the specific subset of input variables) being available within the network are provided by the designer in advance and kept fixed throughout the overall development process. This restriction may hamper a possibility of developing an optimal architecture of the model. The design proposed in this study addresses this issue. The augmented and genetically developed FPNN (gFPNN) results in a structurally optimized structure and comes with a higher level of flexibility in comparison to the one we encounter in the conventional FPNNs. The GA-based design procedure being applied at each layer of the FPNN leads to the selection of the most suitable nodes (or FSPNs) available within the FPNN. In the sequel, two general optimization mechanisms are explored. First, the structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The performance of the gFPNN is quantified through experimentation in which we use a number of modeling benchmarks—synthetic and experimental data being commonly used in fuzzy or neurofuzzy modeling. The obtained results demonstrate the superiority of the proposed networks over the models existing in the references.

A new approach to self-organizing fuzzy polynomial neural networks guided by genetic optimization

In this study, we introduce a new topology of Fuzzy Polynomial Neural Networks (FPNN) that is based on a genetically optimized multilayer perceptron with fuzzy polynomial neurons (FPNs) and discuss its comprehensive design methodology. The underlying methodology involves mechanisms of genetic optimization, especially genetic algorithms (GAs). Let us recall that the design of the “conventional” FPNNs uses an extended Group Method of Data Handling (GMDH) and exploits a fixed fuzzy inference type located at each FPN of the FPNN as well as considers a fixed number of input nodes at FPNs (or nodes) located in each layer. The proposed FPNN gives rise to a structurally optimized structure and comes with a substantial level of flexibility in comparison to the one we encounter in conventional FPNNs. The structural optimization is realized via GAs whereas in the case of the parametric optimization we proceed with a standard least square method based learning. Through the consecutive process of such structural and parametric optimization, an optimized and flexible fuzzy neural network is generated in a dynamic fashion. The performance of the proposed gFPNN is quantified through experimentation that exploits standard data already being used in fuzzy modeling. The results reveal superiority of the proposed networks over the existing fuzzy and neural models.

퍼지관계와 유전자 알고리즘에 기반한 진화론적 최적 퍼지다항식 뉴럴네트워크: 해석과 설계

본 연구에서는 퍼지관계 및 진화론적 최적 다층 퍼셉트론에 기초한 퍼지다항식 뉴럴네트워크(FPNN)의 새로운 구조를 소개하고, 포괄적인 설계방법론을 토의하며, 그리고 일련의 수치적인 실험이 수행된다. 진화론적 최적 FPNN(EFPNN)의 구축을 위해 컴퓨터지능(CI)의 기반 기술을 이용한다. EFPNN의 구조는 규칙베이스 퍼지뉴럴네트워크와 다항식 뉴럴네트워크의 결합에 의한 유전자 최적 구동 하이브리드 시스템의 시너지 이용으로 얻어진다. 퍼지뉴럴네트워크는 EFPNN의 전체규칙 구조의 전반부에 기여하고, EFPNN의 후반부는 다항식 뉴럴네트워크를 사용하여 설계된다. EFPNN의 후반부를 위한 유전론적 최적 다항식 뉴럴네트워크의 개발은 두 최적화 기법에 의해 수행된다. 즉 구조적 최적화는 유전자알고리즘에 의해 수행되고, 파라미터 최적화는 최소자승법 기반의 학습을 통해 행하여진다. EFPNN의 성능 평가를 위해, 모델은 몇 가지 수치 예제를 이용한다. 비교에 의한 해석은 제안된 EFPNN이 이전에 제시된 다른 지능형 모델보다 높은 정확도 뿐만 아니라 좀 더 우수한 예측능력을 가지는 모델임을 보여준다. In this study, we introduce a new topology of Fuzzy Polynomial Neural Networks(FPNN) that is based on fuzzy relation and evolutionally optimized Multi-Layer Perceptron, discuss a comprehensive design methodology and carry out a series of numeric experiments. The construction of the evolutionally optimized FPNN(EFPNN) exploits fundamental technologies of Computational Intelligence. The architecture of the resulting EFPNN results from a synergistic usage of the genetic optimization-driven hybrid system generated by combining rule-based Fuzzy Neural Networks(FNN) with polynomial neural networks(PNN). FNN contributes to the formation of the premise part of the overall rule-based structure of the EFPNN. The consequence part of the EFPNN is designed using PNN. As the consequence part of the EFPNN, the development of the genetically optimized PNN(gPNN) dwells on two general optimization mechanism: the structural optimization is realized via GAs whereas in case of the parametric optimization we proceed with a standard least square method-based learning. To evaluate the performance of the EFPNN, the models are experimented with the use of several representative numerical examples. A comparative analysis shows that the proposed EFPNN are models with higher accuracy as well as more superb predictive capability than other intelligent models presented previously.

진화론적 최적 규칙베이스 퍼지다항식 뉴럴네트워크

In this paper, a new architecture and comprehensive design methodology of genetically optimized Rule-based Fuzzy Polynomial Neural Networks(gRFPNN) are introduced and a series of numeric experiments are carried out. The architecture of the resulting gRFPNN results from asynergistic usage of the hybrid system generated by combining rule-based Fuzzy Neural Networks(FNN) with polynomial neural networks (PNN). FNN contributes to the formation of the premise part of the overall rule-based structure of the gRFPNN. The consequence part of the gRFPNN is designed using PNNs. At the premise part of the gRFPNN, FNN exploits fuzzy set based approach designed by using space partitioning in terms of individual variables and comes in two fuzzy inference forms: simplified and linear. As the consequence part of the gRFPNN, the development of the genetically optimized PNN dwells on two general optimization mechanism: the structural optimization is realized via GAs whereas in case of the parametric optimization we proceed with a standard least square method-based learning. To evaluate the performance of the gRFPNN, the models are experimented with the use of several representative numerical examples. A comparative analysis shows that the proposed gRFPNN are models with higher accuracy as well as more superb predictive capability than other intelligent models presented previously.