Standard Training Algorithm Research Articles

Stable Neural Pid Anti-Swing Control For An Overhead Crane

PD with compensation or PID is the most popular algorithms for the overhead crane control. To minimize steady-state error with respect to uncertainties, PID control needs a big integral gain and the PD with compensator requires a large derivative gain. Both of them deteriorate transient performances of the crane control. In this paper, we propose a novel anti-swing control strategy which combines PID control with neural compensation. The main theoretical contributions of this paper are semi global asymptotic stability of the neural PID for the anti-swing control and local asymptotic stability of the neural PID control with a velocity observer are proven with standard weights training algorithms. These conditions give explicit selection methods for the gains of the linear PID control. An experimental study on an overhead crane with this neural PID control is addressed.

Active vibration control of building structures using fuzzy proportional-derivative/proportional-integral-derivative control

Although most real building structure controllers are in the form of proportional-derivative/proportional-integral-derivative (PD/PID), there have been few published theory results of PD/PID on structural vibration control. In order to minimize the regulation error, a PD/PID control needs relatively large derivative and integral gains. These deteriorate the transient performances of the vibration control. In this paper, a natural combination of industrial PD/PID control with fuzzy compensation is proposed. The main contribution of this paper is that the stability of the fuzzy PD/PID control is proven with standard weight training algorithms. These conditions give explicit selection methods for the gains of the PD/PID control. Experimental studies on a two-story building prototype with the controllers are addressed. The experimental results validate our theoretical analysis.

Neural PID Control of Robot Manipulators With Application to an Upper Limb Exoskeleton

In order to minimize steady-state error with respect to uncertainties in robot control, proportional-integral-derivative (PID) control needs a big integral gain, or a neural compensator is added to the classical proportional-derivative (PD) control with a large derivative gain. Both of them deteriorate transient performances of the robot control. In this paper, we extend the popular neural PD control into neural PID control. This novel control is a natural combination of industrial linear PID control and neural compensation. The main contributions of this paper are semiglobal asymptotic stability of the neural PID control and local asymptotic stability of the neural PID control with a velocity observer which are proved with standard weight training algorithms. These conditions give explicit selection methods for the gains of the linear PID control. An experimental study on an upper limb exoskeleton with this neural PID control is addressed.

Bridge the gap between statistical and hand-crafted grammars

LTAG is a rich formalism for performing NLP tasks such as semantic interpretation, parsing, machine translation and information retrieval. Depend on the specific NLP task, different kinds of LTAGs for a language may be developed. Each of these LTAGs is enriched with some specific features such as semantic representation and statistical information that make them suitable to be used in that task. The distribution of these capabilities among the LTAGs makes it difficult to get the benefit from all of them in NLP applications.This paper discusses a statistical model to bridge between two kinds LTAGs for a natural language in order to benefit from the capabilities of both kinds. To do so, an HMM was trained that links an elementary tree sequence of a source LTAG onto an elementary tree sequence of a target LTAG. Training was performed by using the standard HMM training algorithm called Baum–Welch. To lead the training algorithm to a better solution, the initial state of the HMM was also trained by a novel EM-based semi-supervised bootstrapping algorithm.The model was tested on two English LTAGs, XTAG (XTAG-Group, 2001) and MICA's grammar (Bangalore et al., 2009) as the target and source LTAGs, respectively. The empirical results confirm that the model can provide a satisfactory way for linking these LTAGs to share their capabilities together.

On the Weight Convergence of Elman Networks

An Elman network (EN) can be viewed as a feedforward (FF) neural network with an additional set of inputs from the context layer (feedback from the hidden layer). Therefore, instead of the offline backpropagation-through-time (BPTT) algorithm, a standard online (real-time) backpropagation (BP) algorithm, usually called Elman BP (EBP), can be applied for EN training for discrete-time sequence predictions. However, the standard BP training algorithm is not the most suitable for ENs. A low learning rate can improve the training of ENs but can also result in very slow convergence speeds and poor generalization performance, whereas a high learning rate can lead to unstable training in terms of weight divergence. Therefore, an optimal or suboptimal tradeoff between training speed and weight convergence with good generalization capability is desired for ENs. This paper develops a robust extended EBP (eEBP) training algorithm for ENs with a new adaptive dead zone scheme based on eEBP training concepts. The adaptive learning rate and adaptive dead zone optimize the training of ENs for each individual output and improve the generalization performance of the eEBP training. In particular, for the proposed eEBP training algorithm, convergence of the ENs' weights with the adaptive dead zone estimates is proven in the sense of Lyapunov functions. Computer simulations are carried out to demonstrate the improved performance of eEBP for discrete-time sequence predictions.

Training Radial Basis Function Networks with Differential Evolution

In this paper, Differential Evolution (DE) algorithm, a new promising evolutionary algorithm, is proposed to train Radial Basis Function (RBF) network related to automatic configuration of network architecture. Classification tasks on data sets: Iris, Wine, New-thyroid and Glass are conducted to measure the performance of neural networks. Compared with a standard RBF training algorithm in Matlab neural network toolbox, DE achieves more rational architecture for RBF networks. The resulting networks hence obtain strong generalization abilities.

Minimum Generalization Via Reflection: A Fast Linear Threshold Learner

The number of adjustments required to learn the average LTU function of d features, each of which can take on n equally spaced values, grows as approximately n^2d when the standard perceptron training algorithm is used on the complete input space of n points and perfect classification is required. We demonstrate a simple modification that reduces the observed growth rate in the number of adjustments to approximately d^2(log (d) + log(n)) with most, but not all input presentation orders. A similar speed-up is also produced by applying the simple but computationally expensive heuristic “don‘t overgeneralize” to the standard training algorithm. This performance is very close to the theoretical optimum for learning LTU functions by any method, and is evidence that perceptron-like learning algorithms can learn arbitrary LTU functions in polynomial, rather than exponential time under normal training conditions. Similar modifications can be applied to the Winnow algorithm, achieving similar performance improvements and demonstrating the generality of the approach.

The Canonical Form of Nonlinear Discrete-Time Models

Discrete-time models of complex nonlinear processes, whether physical, biological, or economical, are usually under the form of systems of coupled difference equations. In analyzing such systems, one of the first tasks is to find a state-space description of the process—that is, a set of state variables and the associated state equations. We present a methodology for finding a set of state variables and a canonical representation of a class of systems described by a set of recurrent discrete-time, time-invariant equations. In the field of neural networks, this is of special importance since the application of standard training algorithms requires the network to be in a canonical form. Several illustrative examples are presented.

Dynamic Learning Rate Neural Network Training and Composite Structural Damage Detection

A new learning procedure, called dynamic learning rate steepest descent (DSD) method, is proposed for training neural networks. Based on the simple steepest descendent method, the proposed method improves the learning convergence speed significantly without increasing the computational effort, the memory cost, the algorithm simplicity, and the computational locality in the standard layered error backpropagating training algorithm. Through numerical experiments, the current method is shown to have much better learning ability than that of the standard constant learning rate steepest descent method and the accelerated steepest descendent method. The numerical experiments also indicate that the current method is robust to the selection of the initial learning rate, which is critical in the standard steepest descent method. It is also shown to be efficient. The CPU time increase, due to extra operations in the DSD algorithm, is negligible. The DSD method is then used to train a neural network for direct identification of composite structural damage through structural dynamic responses. The result indicates that neural network can be used for real-time flaw detections and advanced structural health monitoring.

The dependence identification neural network construction algorithm

An algorithm for constructing and training multilayer neural networks, dependence identification, is presented in this paper. Its distinctive features are that (i) it transforms the training problem into a set of quadratic optimization problems that are solved by a number of linear equations, (ii) it constructs an appropriate network to meet the training specifications, and (iii) the resulting network architecture and weights can be further refined with standard training algorithms, like backpropagation, giving a significant speedup in the development time of the neural network and decreasing the amount of trial and error usually associated with network development.

Can deterministic penalty terms model the effects of synaptic weight noise on network fault-tolerance?

This paper investigates fault tolerance in feedforward neural networks, for a realistic fault model based on analog hardware. In our previous work with synaptic weight noise we showed significant fault tolerance enhancement over standard training algorithms. We proposed that when introduced into training, weight noise distributes the network computation more evenly across the weights and thus enhances fault tolerance. Here we compare those results with an approximation to the mechanisms induced by stochastic weight noise, incorporated into training deterministically via penalty terms. The penalty terms are an approximation to weight saliency and therefore, in addition, we assess a number of other weight saliency measures and perform comparison experiments. The results show that the first term approximation is an incomplete model of weight noise in terms of fault tolerance. Also the error Hessian is shown to be the most accurate measure of weight saliency.

Nuclear reactor diagnostic system using genetic algorithm (GA)‐trained neural networks

AbstractSeveral nuclear reactor diagnostic systems using neural networks have been proposed in recent years. Neural networks trained by backpropagation, the standard training algorithm, have certain problems such as local minima and long training times.In this paper, neural networks trained by genetic algorithms are used in a nuclear reactor diagnostic system to solve these problems. The system is tested by simulated data modeled on the experimental fast reactor JOYO, and two categories of abnormality (abnormal control rod vibration and abnormal coolant flow) are identified. The comparisons to networks trained by back‐propagation also are discussed.

Code vector density in topographic mappings: Scalar case

The author derives some new results that build on his earlier work (1989) of combining vector quantization (VQ) theory and topographic mapping (TM) theory. A VQ model (with a noisy transmission medium) is used to model the processes that occur in TMs, which leads to the standard TM training algorithm, albeit with a slight modification to the encoding process. To emphasize this difference, the model is called a topographic vector quantizer (TVQ). In the continuum limit of the one-dimensional (scalar) TVQ. It is found that the density of code vectors is proportional to P(x)(a) (alpha=1/3) assuming that the transmission medium introduces additive noise with a zero-mean, symmetric, monotically decreasing probability density. This result is dramatically different from the result that is predicted when the standard TM training algorithm is used with a uniform symmetric neighborhood [-n, +n], and it is noted that this difference arises entirely from using minimum distortion rather than nearest neighbor encoding.