Universal Learning Network Research Articles

Study of multi-branch structure of Universal Learning Networks

In this paper, multi-branch structure of Universal Learning Networks (ULNs) is studied to verify its effectiveness for obtaining compact models, which have neurons connected with other neurons using more than two branches having nonlinear functions. Multi-branch structure has been proved to have higher representation/generalization ability and lower computational cost than conventional neural networks because of the nonlinear function of the multi-branches and the reduction of the number of neurons to be used. In addition, learning of delay elements of multi-branch ULNs has improved their potential to build up a compact dynamical model with higher performances and lower computational cost when applied for identifying dynamical systems.

A holonic control system based on a universal learning network

A holonic control system based on a universal learning network

Recurrent Neural Networks with Multi-Branch Structure

Universal Learning Networks (ULNs) provide a generalized framework to many kinds of structures of neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the framework of ULNs have been already shown that they have better representation ability in feedforward neural networks (FNNs). Multi-Branch structure of MBNNs can be easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore, RNNs with Multi-Branch structure are proposed and they show that their representation ability is better than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with Multi-Branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with Multi-Branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller sized networks.

Universal learning network and its application for nonlinear system with long time delay

This paper discusses universal learning network (ULN) and its application to control a class of nonlinear systems with long time delay. Two control architectures, model predictive control based on ULN model and single neuron PID (SN-PID) controller based on ULN predictor, are designed to control pH neutralization process, respectively. In addition, to verify the performance of ULN, two comparisons are also made. One is the generalization ability between ULN and back-propagation (BP) network, the other comparison is ULN predictor and Smith predictor, in which the same controller is used. Simulation results prove the applicability and effectiveness of the ULN model. The special architecture and its learning algorithm give ULN more representing abilities to model and control complicated nonlinear systems with long time delay.

An approximate stability analysis of nonlinear systems described by Universal Learning Networks

Stability is one of the most important subjects in control systems. As for the stability of nonlinear dynamical systems, Lyapunov's direct method and linearized stability analysis method have been widely used. But, it is generally recognized that finding an appropriate Lyapunov function is fairly difficult especially for the nonlinear dynamical systems, and also it is not so easy for the linearized stability analysis to find the locally asymptotically stable region. Therefore, it is crucial and highly motivated to develop a new stability analysis method, which is easy to use and can easily study the locally asymptotically stable region at least approximately, if not exactly. On the other hand, as for the calculation of the higher order derivative, Universal Learning Networks (ULNs) are equipped with a systematic mechanism that calculates their first and second order derivatives exactly. So, in this paper, an approximate stability analysis method based on @h approximation is proposed in order to overcome the above problems and its application to a nonlinear dynamical control system is discussed. The proposed method studies the stability of the original trajectory by investigating whether the perturbed trajectory can approach the original trajectory or not. The above investigation is carried out approximately by using the higher order derivatives of ULNs. In summarizing the proposed method, firstly, the absolute values of the first order derivatives of any nodes of the trajectory with respect to any initial disturbances are calculated by using ULNs. If they approach zero at time infinity, then the trajectory is locally asymptotically stable. This is an alternative linearized stability analysis method for nonlinear trajectories without calculating Jacobians directly. In the method, the stability analysis of time-varying systems with multi-branches having any sample delays is possible, because the systems are modeled by ULNs. Secondly, the locally asymptotically stable region, where asymptotical stability is secured approximately, is obtained by finding the area where the first order terms of Taylor expansion are dominant compared to the second order terms with @h approximation assuming that the higher order terms more than the third order are negligibly small in the area. Simulations of an inverted pendulum balancing system are carried out. From the results of the simulations, it is clarified that the stability of the inverted pendulum control system is easily analyzed by the proposed method in terms of studying the locally asymptotically stable region.

Propagation and control of stochastic signals through universal learning networks

The way of propagating and control of stochastic signals through Universal Learning Networks (ULNs) and its applications are proposed. ULNs have been already developed to form a superset of neural networks and have been applied as a universal framework for modeling and control of non-linear large-scale complex systems. However, the ULNs cannot deal with stochastic variables. Deterministic signals can be propagated through a ULN, but the ULN does not provide any stochastic characteristics of the signals propagating through it. The proposed method named Probabilistic Universal Learning Networks (PrULNs) can process stochastic variables and can train network parameters so that the signals behave with the pre-specified stochastic properties. As examples of applications of the proposed method, control and identification of non-linear dynamic systems with noises are studied, and it is shown that the method are useful for dealing with the control and identification of the non-linear stochastic systems contaminated with noises.

Process control of pH neutralization based on adaptive algorithm of universal learning network

This paper presents an adaptive algorithm of universal learning network (ULN) and its application to identify pure time delay of a plant model. Universal learning network can be used in model predictive control for stabilizing a class of nonlinear systems with long time delay. Depending on ULN model with single neuron controller, the control architectures are introduced and applied to pH neutralization process. Simulation results prove the applicability and effectiveness of the ULN model. The general architecture and adaptive learning algorithm give ULN more representing abilities to model and control the nonlinear black box systems with long time delay.

A robust control method for a PV-supplied DC motor using universal learning networks

In this paper, a new robust control method and its application to a photovoltaic (PV) supplied, separately excited DC motor loaded with a constant torque is discussed. The robust controller is designed against the load torque changes by using the first and second ordered derivatives of the universal learning networks (ULNs). These derivatives are calculated using the forward propagation algorithm, which is considered as an extended version of real time recurrent learning (RTRL). In this application, two ULNs are used: The first is the ULN identifier trained offline to emulate the dynamic performance of the DC motor system. The second is the ULN controller, which is trained online not only to make the motor speed follow a selected reference signal, but also to make the overall system operate at the maximum power point of the PV source. To investigate the effectiveness of the proposed robust control method, the simulation is carried out at four different values of the robustness coefficient γ in two different stages: The training stage, in which the simulation is done for a constant load torque. And the control stage, in which the controller performance is obtained when the load torque is changed. The simulation results showed that the robustness of the control system is improved although the motor load torque at the control stage is different from that at the training stage.

Identification and Control of a PV-Supplied Separately Excited DC Motor Using Universal Learning Networks

This paper describes the use of Universal Learning Networks (ULNs) in the identification and control of a separately excited de motor loaded with a centrifugal pump and fed from Photovoltaic (PV) generator via dc-dc buck-boost converter. The Universal Learning Network Identifier (ULNI) is trained offline using the forward propagation algorithm to emulate the dynamic behavior of the de motor system. Then this identifier is used, instead of the motor system, for the online training of the Universal Learning Network Controller (ULNC). As a result, the motor speed can follow all arbitrarily selected reference signal. Furthermore, the overall system call operate at the Maximum Power Point (MPP) of the PV generator. which is the optimal operating point. The simulation results showed a good performance for the identifier and the controller as well.

Universal Learning Networks with Branch Control

Universal Learning Networks with Branch Control

一般化学習ネットワークのインパルス応答に基づく非線形制御方法

一般化学習ネットワークのインパルス応答に基づく非線形制御方法

Improvement of generalization ability for identifying dynamical systems by using universal learning networks

This paper studies how the generalization ability of models of dynamical systems can be improved by taking advantage of the second order derivatives of the outputs with respect to the external inputs. The proposed method can be regarded as a direct implementation of the well-known regularization technique using the higher order derivatives of the Universal Learning Networks (ULNs). ULNs consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm has been derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. First, the method for computing the second order derivatives of ULNs is discussed. Then, a new method for implementing the regularization term is presented. Finally, simulation studies on identification of a nonlinear dynamical system with noises are carried out to demonstrate the effectiveness of the proposed method. Simulation results show that the proposed method can improve the generalization ability of neural networks significantly, especially in terms that (1) the robust network can be obtained even when the branches of trained ULNs are destructed, and (2) the obtained performance does not depend on the initial parameter values.

A new control method of nonlinear systems based on impulse responses of universal learning networks

A new control method of nonlinear dynamic systems is proposed based on the impulse responses of universal learning networks (ULNs), ULNs form a superset of neural networks. They consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm is derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. One of the distinguished features of the proposed control method is that the impulse response of the systems is considered as an extended part of the criterion function and it can be calculated by using the higher order derivatives of ULNs. By using the impulse response as the criterion function, nonlinear dynamics with not only quick response but also quick damping and small steady state error can be more easily obtained than the conventional nonlinear control systems with quadratic form criterion functions of state and control variables.

ブランチ制御を考慮したパラメータ可変一般化学習ネットワーク

ブランチ制御を考慮したパラメータ可変一般化学習ネットワーク

A New Minimax Control Method for Nonlinear Systems Using Universal Learning Networks

In neural network based control systems, if system environments, that is, system parameters and disturbances at training stage, are much different from those at control stage, performances of control systems may become worse. To solve this problem, robust control design is needed. In this paper, we propose a new minimax control method using Universal Learning Networks, in which the criterion function is evaluated at several specific operating points, and at each training step the worst criterion function among the operating points is optimized. Moreover, a sensitivity term calculated on the operating point is included in the criterion function in order to improve the performance of the control system between two operating points. The minimax control method including sensitivity term is shown to have better robustness against the changes of system parameters.

空間分布一般化学習ネットワークを利用した複雑系の相互作用のモデル

空間分布一般化学習ネットワークを利用した複雑系の相互作用のモデル

Universal learning network and its application to robust control

Universal learning networks (ULNs) and robust control system design are discussed, ULNs provide a generalized framework to model and control complex systems. They consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. Therefore, physical systems which can be described by differential or difference equations and also their controllers can be modeled in a unified way. So, ULNs constitute a superset of neural networks or fuzzy neural networks. In order to optimize the systems, a generalized learning algorithm is derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The derivatives are calculated by using forward or backward propagation schemes. These algorithms for calculating the derivatives are extended versions of back propagation through time (BPTT) and real time recurrent learning (RTRL) by Williams in the sense that generalized nonlinear functions and higher order derivatives are dealt with. As an application of ULNs, the higher order derivative, one of the distinguished features of ULNs, is applied to realizing a robust control system in this paper. In addition, it is shown that the higher order derivatives are effective tools to realize sophisticated control of nonlinear systems. Other features of ULNs such as multiple branches with arbitrary time delays and using a priori information will be discussed in other papers.

Universal learning network and its application to chaos control

Universal Learning Networks (ULNs) are proposed and their application to chaos control is discussed. ULNs provide a generalized framework to model and control complex systems. They consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. Therefore, physical systems, which can be described by differential or difference equations and also their controllers, can be modeled in a unified way, and so ULNs may form a super set of neural networks and fuzzy neural networks. In order to optimize the ULNs, a generalized learning algorithm is derived, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The derivatives are calculated by using forward or backward propagation schemes. These algorithms for calculating the derivatives are extended versions of Back Propagation Through Time (BPTT) and Real Time Recurrent Learning (RTRL) of Williams in the sense that generalized node functions, generalized network connections with multi-branch of arbitrary time delays, generalized criterion functions and higher order derivatives can be deal with. As an application of ULNs, a chaos control method using maximum Lyapunov exponent of ULNs is proposed. Maximum Lyapunov exponent of ULNs can be formulated by using higher order derivatives of ULNs, and the parameters of ULNs can be adjusted so that the maximum Lyapunov exponent approaches the target value. From the simulation results, it has been shown that a fully connected ULN with three nodes is able to display chaotic behaviors.

Chaos control on universal learning networks

A new chaos control method is proposed which is useful for taking advantage of chaos and avoiding it. The proposed method is based on the following facts: (1) chaotic phenomena can be generated and eliminated by controlling the maximum Lyapunov exponent of the systems, and (2) the maximum Lyapunov exponent can be formulated and calculated by using higher-order derivatives of universal learning networks (ULNs). ULNs consist of a number of interconnected nodes which may have any continuously differentiable nonlinear functions in them and where each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm has been derived for the ULNs in which both first-order derivatives (gradients) and higher-order derivatives are incorporated. In simulations, parameters of ULNs with bounded node outputs were adjusted for the maximum Lyapunov exponent to approach the target value, and it has been shown that a fully-connected ULN with three sigmoidal function nodes is able to generate and eliminate chaotic behaviors by adjusting these parameters.

Control of Decentralized Systems Based on Nash Equilibrium Concept of Game Theory

In this paper, a new control method for decentralized systems is proposed by using the concept of Nash equilibrium points of non co-operative game. The main reason for applying non co-operative game to decentralized systems is that multi-objective decentralized systems can be easily formulated without considering weighting factor of each objective function. It is supposed that the system stated in this article is composed of many subsystems including their own controllers, so each subsystem can be regarded as a player in the game. Each subsystem and its controller are described by the Universal Learning Networks which have been proposed to provide a universal framework for the class of neural networks. From the above assumptions it is theoretically shown that if the criterion function for each subsystem can be defined individually, the Nash equilibrium points can be calculated by the commonly used gradient learning algorithm of networks. Simulation studies on a decentralized tank network control system show that the Nash equilibrium points can be obtained systematically and effectively by the proposed method.