Backpropagation Learning Scheme Research Articles

Artificial Neural Network Approach for Relativistic Polytropes

Over the last few decades, artificial neural networks (ANN) have played an essential role in many areas of human activity and have found application in many branches of natural sciences. ANNs have been widely used to tackle problems related to linear and nonlinear differential equations, and numerous paradigms for ANN architecture have been employed. This research proposes a computational technique based on ANN and the Taylor series to solve difficulties connected to the relativistic gas spheres' Tolman-Oppenheimer-Volkoff equations (TOV). We used ANN to study two cases related to relativistic polytropes. The first is to simulate both the Emden and the relativistic functions. The second is to predict the zeros of the Emden function and its corresponding relativistic functions. In its feed-forward back-propagation learning scheme, we used the ANN framework. The efficiency of the proposed algorithm is evaluated by running it through seven models. Comparing the analytical and the ANN solutions gives good agreement for the two cases under study.

Modeling fractional polytropic gas spheres using artificial neural network

Lane–Emden differential equations describe different physical and astrophysical phenomena that include forms of stellar structure, isothermal gas spheres, gas spherical cloud thermal history, and thermionic currents. This paper presents a computational approach to solve the problems related to fractional Lane–Emden differential equations based on neural networks. Such a solution will help solve the fractional polytropic gas spheres problems which have different applications in physics, astrophysics, engineering, and several real-life issues. We used artificial neural network (ANN) framework in its feed-forward back-propagation learning scheme. The efficiency and accuracy of the presented algorithm are checked by testing it on four fractional Lane–Emden equations and compared with the exact solutions for the polytropic indices n = 0,1,5 and those of the series expansions for the polytropic index n = 3. The results we obtained prove that using the ANN method is feasible and accurate and may outperform other methods.

HyFIS: adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems

This paper proposes an adaptive neuro-fuzzy system, HyFIS (Hybrid neural Fuzzy Inference System), for building and optimising fuzzy models. The proposed model introduces the learning power of neural networks to fuzzy logic systems and provides linguistic meaning to the connectionist architectures. Heuristic fuzzy logic rules and input–output fuzzy membership functions can be optimally tuned from training examples by a hybrid learning scheme comprised of two phases: rule generation phase from data; and rule tuning phase using error backpropagation learning scheme for a neural fuzzy system. To illustrate the performance and applicability of the proposed neuro-fuzzy hybrid model, extensive simulation studies of nonlinear complex dynamic systems are carried out. The proposed method can be applied to an on-line incremental adaptive learning for the prediction and control of nonlinear dynamical systems. Two benchmark case studies are used to demonstrate that the proposed HyFIS system is a superior neuro-fuzzy modelling technique.

Adaptive fuzzy command acquisition with reinforcement learning

Proposes a four-layered adaptive fuzzy command acquisition network (AFCAN) for adaptively acquiring fuzzy command via interactions with the user or environment. It can catch the intended information from a sentence (command) given in natural language with fuzzy predicates. The intended information includes a meaningful semantic action and the fuzzy linguistic information of that action. The proposed AFCAN has three important features. First, we can make no restrictions whatever on the fuzzy command input, which is used to specify the desired information, and the network requires no acoustic, prosodic, grammar, and syntactic structure, Second, the linguistic information of an action is learned adaptively and it is represented by fuzzy numbers based on /spl alpha/-level sets. Third, the network can learn during the course of performing the task. The AFCAN can perform off-line as well as online learning. For the off-line learning, the mutual-information (MI) supervised learning scheme and the fuzzy backpropagation (FBP) learning scheme are employed when the training data are available in advance. The former learning scheme is used to learn meaningful semantic actions and the latter learn linguistic information. The AFCAN can also perform online learning interactively when it is in use for fuzzy command acquisition. For the online learning, the MI-reinforcement learning scheme and the fuzzy reinforcement learning scheme are developed for the online learning of meaningful actions and linguistic information, respectively. An experimental system is constructed to illustrate the performance and applicability of the proposed AFCAN.

A fuzzy adaptive learning control network with on-line structure and parameter learning.

This paper addresses a general connectionist model, called Fuzzy Adaptive Learning Control Network (FALCON), for the realization of a fuzzy logic control system. An on-line supervised structure/parameter learning algorithm is proposed for constructing the FALCON dynamically. It combines the backpropagation learning scheme for parameter learning and the fuzzy ART algorithm for structure learning. The supervised learning algorithm has some important features. First of all, it partitions the input state space and output control space using irregular fuzzy hyperboxes according to the distribution of training data. In many existing fuzzy or neural fuzzy control systems, the input and output spaces are always partitioned into "grids". As the number of input/output variables increase, the number of partitioned grids will grow combinatorially. To avoid the problem of combinatorial growing of partitioned grids in some complex systems, the proposed learning algorithm partitions the input/output spaces in a flexible way based on the distribution of training data. Second, the proposed learning algorithm can create and train the FALCON in a highly autonomous way. In its initial form, there is no membership function, fuzzy partition, and fuzzy logic rule. They are created and begin to grow as the first training pattern arrives. The users thus need not give it any a priori knowledge or even any initial information on these. In some real-time applications, exact training data may be expensive or even impossible to obtain. To solve this problem, a Reinforcement Fuzzy Adaptive Learning Control Network (RFALCON) is further proposed. The proposed RFALCON is constructed by integrating two FALCONs, one FALCON as a critic network, and the other as an action network. By combining temporal difference techniques, stochastic exploration, and a proposed on-line supervised structure/parameter learning algorithm, a reinforcement structure/parameter learning algorithm is proposed, which can construct a RFALCON dynamically through a reward/penalty signal. The ball and beam balancing system is presented to illustrate the performance and applicability of the proposed models and learning algorithms.

Neural networks with modified backpropagation learning applied to structural optimization

A multilayer feedforward neural network with a modification to the standard backpropagation training of neural nets is considered. Information on the gradients of target outputs with respect to network inputs is used in this modification. This modified neural network is then applied for approximating design requirements of aerospace composite components, such as an aircraft engine guide vane and a satellite reflector assembly. This neural network functional approximator is then used with an optimization algorithm for determining the optimal parameters for component design requirements. The modified neural network provides for faster convergence of learning process and requires a smaller number of hidden layer nodes for convergence to similar training error levels. HE application of neural networks as universal approximators has been the focus of tremendous amount of activity in the past few years.16 Neural networks have been used for approximating and modeling both memoryless systems and dynamical systems with memory. This paper does not go into the various neural net architectures and training schemes, and the interested reader could refer to standard texts on this subject matter.1'2 This paper considers static neural networks (systems without memory) formed by interconnections without loops, called feedforward networks.2 In addition, this work considers a backpropagation learning scheme with adaptation of learning rate. There have been many additions to, or adaptations of, the basic backpropagation scheme, such as adaptive learning rate, learning with momentum, and the like,1'2 for improving convergence of the learning scheme, and these are not addressed in this paper. This paper explores a modification of the backpropagation learning scheme using information on the gradient of the target outputs with respect to the inputs of the network. The performance of the modified learning scheme is compared with that of the standard backpropagation learning scheme.2 A feedforward network with a single hidden layer trained with this modified learning scheme is applied in structural optimization problems. Figure 1 shows a feedforward net with a single hidden layer. The restriction to a single hidden layer is not a limitation of the modified learning scheme, and the new training scheme can be used on networks with multiple hidden layers also. The use of neural networks for structural response approximation has been reported by several authors.46 The primary reason for investigating such properties is to assess the applicability of trained neural networks for quick and accurate approximation of response functions during structural design optimization. Structural optimization, utilizing either gradient-based numerical search or genetic search algorithms, requires repeated function (response) evaluations during the search process. A single finite element analysis of large-scale structures requires a significant amount of computational time and resource and hence the need for robust approxima

An analog feed-forward neural network with on-chip learning

An analog continuous-time neural network with on-chip learning is presented. The 4-3-2 feed-forward network with a modified back-propagation learning scheme was build using micropower building blocks in a double poly, double metal 2μ CMOS process. The weights are stored in non-volatile UV-light programmable analog floating gate memories. A differential signal representation is used to design simple building blocks which may be utilized to build very large neural networks. Measured results from on-chip learning are shown and an example of generalization is demonstrated. The use of micro-power building blocks allows very large networks to be implemented without significant power consumption.

Adaptive fuzzy control of unstable nonlinear systems.

This paper addresses the structure and an associated on-line learning algorithm of a feedforward multilayer connectionist network for realizing the basic elements and functions of a traditional fuzzy logic controller. The proposed Fuzzy Adaptive Learning Control Network (FALCON) can be contrasted with the traditional fuzzy logic control systems in their network structure and learning ability. An on-line structure/parameter learning algorithm, called FALCON-ART, is proposed for constructing the FALCON dynamically. The FALCON-ART can partition the input/output space in a flexible way based on the distribution of the training data. Hence it can avoid the problem of combinatorial growing of partitioned grids in some complex systems. It combines the backpropagation learning scheme for parameter learning and the fuzzy ART algorithm for structure learning. More notably, the FALCON-ART can on-line partition the input/output spaces, tune membership functions, and find proper fuzzy logic rules dynamically without any a priori knowledge or even any initial information on these. The proposed learning scheme has been successfully used to control two unstable nonlinear systems. They are the seesaw system and the inverted wedge system.

Fuzzy adaptive learning control network with on-line neural learning

This paper addresses the structure and the associated on-line learning algorithms of a feedforward multilayered connectionist network for realizing the basic elements and functions of a traditional fuzzy logic controller. The proposed Fuzzy Adaptive Learning COntrol Network (FALCON) can be contrasted with the traditional fuzzy logic control systems in their network structure and learning ability. The connectionist structure of the proposed FALCON can be constructed from training examples by neural learning techniques to find proper fuzzy partitions, membership functions, and fuzzy logic rules. Two complementary on-line structure/parameter learning algorithms, FALCON-FSM and FALCON-ART, are proposed for constructing the FALCON dynamically. The FALCON-FSM combines the backpropagation learning scheme for parameter learning and a fuzzy similarity measure for structure learning. The FALCON-FSN can find proper fuzzy logic rules, membership functions, and the size of output partitions simultaneously. In the FALCON-FSM algorithm, the input and output spaces are partitioned into “grids”. The grid-typed space partitioning certainly makes both the fuzzy logic controller software emulation and fuzzy chip implementation convenient. However, as the number of input/output variables increases, the number of partitioned grids will grow combinatorially. To avoid the problem of combinatorial growth of partitioned grids in some complex systems, the FALCON-ART algorithm is developed, which can partition the input and output spaces in a more flexible way based on the distribution of the training data. The FALCON-ART combines the backpropagation learning scheme for parameter learning and a fuzzy ART algorithm for structure learning. The FALCON-ART can on-line partition the input and output spaces, tune membership functions and find proper fuzzy logic rules dynamically. Computer simulations were conducted to illustrate the performance and applicability of both FALCON-FSM and FALCON-ART learning algorithms.

A neural fuzzy control system with structure and parameter learning

A general connectionist model, called neural fuzzy control network (NFCN), is proposed for the realization of a fuzzy logic control system. The proposed NFCN is a feedforward multilayered network which integrates the basic elements and functions of a traditional fuzzy logic controller into a connectionist structure which has distributed learning abilities. The NFCN can be constructed from supervised training examples by machine learning techniques, and the connectionist structure can be trained to develop fuzzy logic rules and find membership functions. Associated with the NFCN is a two-phase hybrid learning algorithm which utilizes unsupervised learning schemes for structure learning and the backpropagation learning scheme for parameter learning. By combining both unsupervised and supervised learning schemes, the learning speed converges much faster than the original backpropagation algorithm. The two-phase hybrid learning algorithm requires exact supervised training data for learning. In some real-time applications, exact training data may be expensive or even impossible to obtain. To solve this problem, a reinforcement neural fuzzy control network (RNFCN) is further proposed. The RNFCN is constructed by integrating two NFCNs, one functioning as a fuzzy predictor and the other as a fuzzy controller. By combining a proposed on-line supervised structure-parameter learning technique, the temporal difference prediction method, and the stochastic exploratory algorithm, a reinforcement learning algorithm is proposed, which can construct a RNFCN automatically and dynamically through a reward-penalty signal (i.e., “good” or “bad” signal). Two examples are presented to illustrate the performance and applicability of the proposed models and learning algorithms.

Improving Classification Using a Tree Structured Neural Network

This article introduces a tree structured network for improving the performance of the feedforward neural network FN classifier. The building blocks of the tree are the feedforward neural network with backpropagation learning scheme and the simple logical OR neural network ORNN. The confusion matrix CM, resulting from some preliminary experiments, is used to divide the considered patterns into groups in several primary stages until no more grouping could be obtained. The proposed structure can be used for any pattern classification problem. In this article, the testing environment is the isolated handwritten Arabic character set, which is a problem of reasonable complexity. Two simple kinds of feature vectors are used to represent characters before the FN. The use of the proposed tree structure improved the recognition accuracy from 80% for the single FN to 97.7% using the same simple set of features. The results show that with the proposed method, better classification results could be obtained without having to introduce more complex features.