Improved Learning Algorithm Research Articles

Research on Wear Debris Recognition Algorithm Based on IFNN

According to the qualitative characterization of the morphological features of the wear debris, Feature vector mathematics model of wear debris are built,which are based on Foruier parameter refining methord.Then the advantages of fuzzy system and neural network are taken to establish a kind of improved fuzzy neural network (IFNN) models and algorithm.which is used to realize the automatic classification and recognition of wear debris. An improved learning algorithm with the modified fuzzy weight is proposed on the basis of the fuzzy neurons model for the max-min fuzzy operator. The amount of calculation for the improved FNN model is reduced greatly and the convergence velocity is improved. At last ,the experiment results show that the recognition method based on the IFNN is good at algorithm convergence speed and recognition accuracy.

Voting based extreme learning machine

This paper proposes an improved learning algorithm for classification which is referred to as voting based extreme learning machine. The proposed method incorporates the voting method into the popular extreme learning machine (ELM) in classification applications. Simulations on many real world classification datasets have demonstrated that this algorithm generally outperforms the original ELM algorithm as well as several recent classification algorithms.

End-User Feature Labeling via Locally Weighted Logistic Regression

Applications that adapt to a particular end user often make inaccurate predictions during the early stages when training data is limited. Although an end user can improve the learning algorithm by labeling more training data, this process is time consuming and too ad hoc to target a particular area of inaccuracy. To solve this problem, we propose a new learning algorithm based on Locally Weighted Logistic Regression for feature labeling by end users, enabling them to point out which features are important for a class, rather than provide new training instances. In our user study, the first allowing ordinary end users to freely choose features to label directly from text documents, our algorithm was more effective than others at leveraging end users’ feature labels to improve the learning algorithm. Our results strongly suggest that allowing users to freely choose features to label is a promising method for allowing end users to improve learning algorithms effectively.

Estimation of solar radiation using a neural network based on radiative transfer

[1] This study developed an algorithm for estimating solar radiation from space using a neural network (NN) with an improved learning algorithm to approximate radiative transfer code. The NN solver for the solar radiation budget is based on radiative transfer calculations. All data sets for testing and training the NN were generated from radiative transfer code. Thus the NN traces the radiative transfer calculation that is approximated by a learning algorithm. To demonstrate the effectiveness of the NN approach for high-speed estimation and multiparameter problems, the NN was applied to data from a geostationary satellite and a Sun-synchronous subrecurrent orbit satellite. The developed algorithm was applied to data from the Multi-functional Transport Satellite-1 Replacement (MTSAT-1R) geostationary satellite, and estimations were validated against in situ observations for March 2006 at four SKYNET sites. Byproducts of the algorithm include UVA, UVB, and photosynthetically active radiation (PAR) fluxes as well as direct and diffuse components. The NN approach enables semi-real-time analysis of these products by high-speed calculation. In addition, the NN allows for consideration of detailed particle optical parameters in the solar radiation budget without the need for a massive database. The method was also applied to observations from the Advanced Earth Observing Satellite-II/Global Imager (ADEOS-II/GLI) for May 2003. The results showed trends in the direct and diffuse components of downward solar radiation over the North Pacific Ocean. This report outlines the construction of the NN for radiation budget estimation and demonstrates the effectiveness of the NN approach.

Improved Learning Algorithm Based on Semi-Supervised Support Vector

Improved learning algorithm for branch and bound for semi-supervised support vector machines is proposed, according to the greater difference in the optimal solution in different semi-supervised support vector machines for the same data set caused by the local optimization. The lower bound of node in IBBS3VM algorithm is re-defined, which will be pseudo-dual function value as the lower bound of node to avoid the large amount of calculation of 0-1 quadratic programming, reducing the lower bound of each node calculate the time complexity; at the same time, in determining the branch nodes, only based on the credibility of the unlabeled samples without the need to repeatedly carry out the training of support vector machines to enhance the training speed of the algorithm. Simulation analysis shows that IBBS3VM presented in this paper has faster training speed than BBS3VM algorithms, higher precision and stronger robustness than the other semi-supervised support vector machines.

An Improved Hybrid Algorithm Based on PSO and BP for Feedforward Neural Networks

In this paper, an improved hybrid algorithm combining particle swarm optimization (PSO) with backpropagation algorithm (BP) is proposed to train feedforward neural networks (FNN). PSO is a global search algorithm, but the swarm in PSO is easy to lose its diversity, which results in premature convergence. On the other hand, BP algorithm is a gradient-descent-based method which has good local search ability around the global minima. Hence, the presented algorithm in this study combines PSO with BP to perform double search. Moreover, in order to improve the diversity of the swarm in the PSO, each particle in the swarm and its best position are disturbed by a random function, and the best position of all particles are reset as the optimum weights of FNN obtained by BP. The proposed algorithm improves the diversity of the swarm as well as reduces the likelihood of the swarm being trapped into local minima on the error surface. Compared with the traditional learning algorithms, the improved learning algorithm has much better convergence accuracy and rate. Finally, the experimental results are given to verify the efficiency and effectiveness of the proposed algorithm.

PiLiMoT: A Modified Combination of LoLiMoT and PLN Learning Algorithms for Local Linear Neurofuzzy Modeling

Locally linear model tree (LoLiMoT) and piecewise linear network (PLN) learning algorithms are two approaches in local linear neurofuzzy modeling. While both methods belong to the class of growing tree learning algorithms, they use different logics. PLN learning relies on training data, it needs rich training data set and no division test, so it is much faster than LoLiMoT, but it may create adjacent neurons that may lead to singularity in regression matrix. On the other hand, LoLiMoT almost always leads to acceptable output error, but it often needs more rules. In this paper, to exploit the complimentary performance of both algorithms piecewise linear model tree (PiLiMoT) learning algorithm is introduced. In essence, PiLiMoT is a combination of LoLiMoT and PLN learning. The initially proposed algorithm is improved by adding the ability to merge previously divided local linear models, and utilizing a simulated annealing stochastic decision process to select a local model for splitting. Comparing to LoLiMoT and PLN learning, our proposed improved learning algorithm shows the ability to construct models with less number of rules at comparable modeling errors. Algorithms are compared through a case study of nonlinear function approximation. Obtained results demonstrate the advantages of combined modified method.

Tracking control for nonholonomic mobile robots: Integrating the analog neural network into the backstepping technique

This paper proposes an improved learning algorithm of analog compound orthogonal networks and a novel tracking control approach for nonholonomic mobile robots by integrating the neural network into the backstepping technique. The adaptive control is derived from continuously tuning parameters using the analog neural network in the backstepping control law. The proposed control approach for the mobile robot has the properties to quickly drive the position error to zero and to indicate better smooth movement in the tracking control process. These features are due to continuous online learning and adaptive capability of analog neural networks. Simulation results demonstrate that the proposed control approach is effective and has better control performance.

Chaos control of ferroresonance system based on improved RBF neural network

Facing to the ferroresonance over voltage of neutral grounded power system, an improved learning algorithm based on RBF neural networks is used to control the chaos system. The algorithm optimizes the object function to derive learning rule of central vectors, and uses the clustering function of network hidden layers.It improves the regression and learning ability of neural networks. The academic derivation testifies the errors and precision could satisfy demand of chaos control.And simulation calculation also displayed that the rate of convergence of the improved RBF neural network is much quickly and approach ability is much stronger. The numerical experimentation of ferroresonance system testifies the reliability and stability of using the algorithm to control chaos in neutral grounded power system.

An Evolutionary Game Theoretic Perspective on Learning in Multi-Agent Systems

In this paper we revise Reinforcement Learning and adaptiveness in Multi-Agent Systems from an Evolutionary Game Theoretic perspective. More precisely we show there is a triangular relation between the fields of Multi-Agent Systems, Reinforcement Learning and Evolutionary Game Theory. We illustrate how these new insights can contribute to a better understanding of learning in MAS and to new improved learning algorithms. All three fields are introduced in a self-contained manner. Each relation is discussed in detail with the necessary background information to understand it, along with major references to relevant work.

Approximate Optimization Method for Dsign of a Multi-Layered Piezoceramic Plate

The present paper discusses an approximate optimization method using the neural network in order to design a multi-layered composite plate constructed of an isotropic structural layer and multiple piezoceramic layers of class 6mm. The puropse of this design is to determine the ticknesses of the piezoceramic layers, such that the maximum value of the electric potential distribution which controls the thermal displacement on the surface of the structural layer is minimized subject to stress constraints. Instead of the back propagation method, the quasi-Newton method is employed for an updating formula of the connection weights. By restricting the normalization range of the training data, total execution time necessary to the training process are remarkably reduced. Thus, performance of the learning algorithm is greatly improved. Applying the improved learning algorithm to the design of the multi-layered composite plate, fairly good results for the approximate optimization are obtained.

Improved learning algorithms for mixture of experts in multiclass classification

Mixture of experts (ME) is a modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME architecture for adjusting the parameters and the iteratively reweighted least squares (IRLS) algorithm is used to perform maximization in the inner loop [Jordan, M.I., Jacobs, R.A. (1994). Hierarchical mixture of experts and the EM algorithm, Neural Computation, 6(2), 181–214]. However, it is reported in literature that the IRLS algorithm is of instability and the ME architecture trained by the EM algorithm, where IRLS algorithm is used in the inner loop, often produces the poor performance in multiclass classification. In this paper, the reason of this instability is explored. We find out that due to an implicitly imposed incorrect assumption on parameter independence in multiclass classification, an incomplete Hessian matrix is used in that IRLS algorithm. Based on this finding, we apply the Newton–Raphson method to the inner loop of the EM algorithm in the case of multiclass classification, where the exact Hessian matrix is adopted. To tackle the expensive computation of the Hessian matrix and its inverse, we propose an approximation to the Newton–Raphson algorithm based on a so-called generalized Bernoulli density. The Newton–Raphson algorithm and its approximation have been applied to synthetic data, benchmark, and real-world multiclass classification tasks. For comparison, the IRLS algorithm and a quasi-Newton algorithm called BFGS have also been applied to the same tasks. Simulation results have shown that the use of the proposed learning algorithms avoids the instability problem and makes the ME architecture produce good performance in multiclass classification. In particular, our approximation algorithm leads to fast learning. In addition, the limitation of our approximation algorithm is also empirically investigated in this paper.

Neural network dynamic modelling of rock microfracturing sequences under triaxial compressive stress conditions

Rock fracturing processes are very complicated nonlinear dynamic systems. Distributions of acoustic emission (AE) events in the time dimension during microfracturing processes of rock under triaxial compressive stress conditions have fractal structures that proceed as C( t)∝ t D , where the fractal dimension D is 0.43≤ D<1.0. As the fracturing process progresses, the system's state initially changes from ordered to disordered (fractal dimension D decreases from about 1 to about 0.48) and then changes back to ordered (fractal dimension increases from 0.48 to about 0.91). Corresponding to each evolutionary process of the system's states, AE event patterns such as the AE event rate, AE count rate, and amplitude in rock fracturing processes were recognized using neural network techniques. AE event patterns at 8–10 succeeding time points were predicted using the corresponding models. AE event patterns in rock microfracturing processes are effectively described by the neural dynamic model NN( n, h,1). The models so obtained are applicable for extrapolated recognition of AE event patterns with adequate accuracy. An improved learning algorithm is proposed to train the networks with generally improved performance of the models.

Speed up training of the recurrent neural network based on constrained optimization techniques

In this paper, the constrained optimization technique for a substantial problem is explored, that is accelerating training the globally recurrent neural network. Unlike most of the previous methods in feedforward neural networks, the authors adopt the constrained optimization technique to improve the gradientbased algorithm of the globally recurrent neural network for the adaptive learning rate during training. Using the recurrent network with the improved algorithm, some experiments in two real-world problems, namely, filtering additive noises in acoustic data and classification of temporal signals for speaker identification, have been performed. The experimental results show that the recurrent neural network with the improved learning algorithm yields significantly faster training and achieves the satisfactory performance.

An improvement of a back propagation algorithm by the extended Kalman filter and demand forecasting by layered neural networks

Recently, Watanabe et al. proposed a back propagation algorithm via the extended Kalman filter, in which the learning rate was time-varying. In their algorithm the weights and biases are treated as independent variables. It is, however, natural that the weights and biases are not always independent, and generally have mutual correlation. In this paper, we improve the back propagation algorithm by considering that there is mutual correlation among the weights and bias directly connected to the unit. Through some numerical examples, our improved learning algorithm is compared with Watanabe et al.'s algorithm in learning ability. Furthermore, we consider demand forecasting as a kind of pattern recognition, and propose a demand forecasting method using layered neural networks with the improved learning algorithm. The effectiveness of this demand forecasting method is also discussed through some simulations.

The S/N Analysis of Associative Memory

There are some models of associative memory that have improved learning algorithms or dynamics. We obtained the memory capacity of an associative memory using the sign alternating memorization method and the pairing memorization method by means of self-consistent signal-to-noise analysis(SCSNA). In the case using the grouping memorization method, as the number of patterns of a group increases, the memory capacity increases. If more than about 10 patterns are grouped, the memory capacity is about 0.27 as well as the cross-correlation type.

A classification approach using multi-layered neural networks

There has been an increasing interest in the applicability of neural networks in disparate domains. In this paper, we describe the use of multi-layered perceptrons, a type of neural-network topology, for financial classification problems, with promising results. Back-propagation, which is the learning algorithm most often used in multi-layered perceptrons, however, is inherently an inefficient search procedure. We present improved procedures which have much better convergence properties. Using several financial classification applications as examples, we show the efficacy of using multilayered perceptrons with improved learning algorithms. The modified learning algorithms have better performance, in terms of classification/prediction accuracies, than the methods previously used in the literature, such as probit analysis and similarity-based learning techniques.

Dynamics of learning and generalization in perceptrons with constraints

Depending on the kind of constraints imposed on the weights of a perceptron, learning can be a combinatorially hard problem. As an example of this type, I discuss a perception with binary weights comparing results obtained from replica theory, dynamic mean field theory and simulated annealing. Contrary to the replica calculation, dynamics yields information about the performance of a polynomial algorithm in a situation where the best solution cannot be found in polynomial time. I also discuss improved learning algorithms and results for finite size perceptrons.

A feedforward neural network for multiple criteria decision making

Many complex real-world problems are characterized as decision making with multiple, conflicting and noncommensurate objectives. Because of the complexity of factors that are involved, it is usually difficult to derive a decision rule for determining the most desirable alternative. This paper is to demonstrate the potential role of artificial neural networks for multiple criteria decision making. This paper presents a feedforward neural network for solving discrete multiple criteria decision problems under certainty. Starting with formulating multiple criteria decision problems under the theme of supervised learning, this paper specifies two types of multiattribute decision models, proposes a particular form of feedforward neural network, analyzes some desirable properties associated with supervised learning, presents an improved learning algorithm and discusses results of illustrative examples and numerical simulation.

Quantifying inductive bias: AI learning algorithms and Valiant's learning framework

We show that the notion of inductive bias in concept learning can be quantified in a way that directly relates to learning performance in the framework recently introduced by Valiant. Our measure of bias is based on the growth function introduced by Vapnik and Chervonenkis, and on the Vapnik-Chervonenkis dimension. We measure some common language biases, including restriction to conjunctive concepts, conjunctive concepts with internal disjunction, k-DNF and k-CNF concepts. We also measure certain types of bias that result from a preference for simpler hypotheses. Using these bias measurements we analyze the performance of the classical learning algorithm for conjunctive concepts from the perspective of Valiant's learning framework. We then augment this algorithm with a hypothesis simplification routine that uses a greedy heuristic and show how this improves learning performance on simpler target concepts. Improved learning algorithms are also developed for conjunctive concepts with internal disjunction, k-DNF and k-CNF concepts. We show that all our algorithms are within a logarithmic factor of optimal in terms of the number of examples they require to achieve a given level of learning performance in the Valiant framework. Our results hold for arbitrary attribute-based instance spaces defined by either tree-structured or linear attributes.