Natural Gradient Learning Research Articles

Orthogonal extended infomax algorithm.

Objective.The extended infomax algorithm for independent component analysis (ICA) can separate sub- and super-Gaussian signals but converges slowly as it uses stochastic gradient optimization. In this paper, an improved extended infomax algorithm is presented that converges much faster.Approach.Accelerated convergence is achieved by replacing the natural gradient learning rule of extended infomax by a fully-multiplicative orthogonal-group based update scheme of the ICA unmixing matrix, leading to an orthogonal extended infomax algorithm (OgExtInf). The computational performance of OgExtInf was compared with original extended infomax and with two fast ICA algorithms: the popular FastICA and Picard, a preconditioned limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm belonging to the family of quasi-Newton methods.Main results.OgExtInf converges much faster than original extended infomax. For small-size electroencephalogram (EEG) data segments, as used for example in online EEG processing, OgExtInf is also faster than FastICA and Picard.Significance.OgExtInf may be useful for fast and reliable ICA, e.g. in online systems for epileptic spike and seizure detection or brain-computer interfaces.

Adaptive Natural Gradient Method for Learning of Stochastic Neural Networks in Mini-Batch Mode

Gradient descent method is an essential algorithm for learning of neural networks. Among diverse variations of gradient descent method that have been developed for accelerating learning speed, the natural gradient learning is based on the theory of information geometry on stochastic neuromanifold, and is known to have ideal convergence properties. Despite its theoretical advantages, the pure natural gradient has some limitations that prevent its practical usage. In order to get the explicit value of the natural gradient, it is required to know true probability distribution of input variables, and to calculate inverse of a matrix with the square size of the number of parameters. Though an adaptive estimation of the natural gradient has been proposed as a solution, it was originally developed for online learning mode, which is computationally inefficient for the learning of large data set. In this paper, we propose a novel adaptive natural gradient estimation for mini-batch learning mode, which is commonly adopted for big data analysis. For two representative stochastic neural network models, we present explicit rules of parameter updates and learning algorithm. Through experiments on three benchmark problems, we confirm that the proposed method has superior convergence properties to the conventional methods.

Accuracy-Based Learning Classifier Systems for Multistep Reinforcement Learning: A Fuzzy Logic Approach to Handling Continuous Inputs and Learning Continuous Actions

Despite their proven effectiveness, many Michigan learning classifier systems (LCSs) cannot perform multistep reinforcement learning in continuous spaces. To meet this technical challenge, some LCSs have been designed to learn fuzzy logic rules. They can be largely classified into strength-based and accuracy-based systems. The latter is gaining more research attention in the last decade. However, existing accuracy-based learning systems either address primarily single-step learning problems or require the action space to be discrete. In this paper, a new accuracy-based learning fuzzy classifier system (LFCS) is developed to explicitly handle continuous state input and continuous action output during multistep reinforcement learning. Several technical improvements have been achieved while developing the new learning algorithm. Particularly, we have successfully extended ${Q}$ -learning like credit assignment methods to continuous spaces. To enable direct learning of stochastic strategies for action selection, we have also proposed to use a new fuzzy logic system with stochastic action outputs. Moreover, fine-grained learning of fuzzy rules has been achieved effectively in our algorithm by using a natural gradient learning method. It is the first time that these techniques are utilized substantially in any accuracy-based LFCSs. Meanwhile, in comparison with several recently proposed learning algorithms, our algorithm is shown to perform highly competitively on four benchmark learning problems and a robotics problem. The practical usefulness of our algorithm is also demonstrated by improving the performance of a wireless body area network.

Implementation of Imitation Learning using Natural Learner Central Pattern Generator Neural Networks

In this paper a new design of neural networks is introduced, which is able to generate oscillatory patterns. The fundamental building block of the neural network is O-neurons that can generate an oscillation in its transfer functions. Since the natural policy gradient learning has been used in training a central pattern generator paradigm, it is called Natural Learner CPG Neural Networks (NLCPGNN). O-neurons are connected and coupled to each other in order to shape a network and their unknown parameters are found by a natural policy gradient learning algorithm. The main contribution of this paper is design of this learning algorithm which is able to simultaneously search for the weights and topology of the network. This system is capable to obtain any complex motion and rhythmic trajectory via first layer and learn rhythmic trajectories in the second layer and converge towards all these movements. Moreover this two layers system is able to provide various features of a learner model for instance resistance against perturbations, modulation of trajectories amplitude and frequency. Simulation of the learning system in the robot simulator (WEBOTS) that is linked with MATLAB software has been done. Implementation on a real NAO robot demonstrates that the robot has learned desired motion with high accuracy. These results show proposed system produces high convergence rate and low test errors.

Using Learning Classifier Systems to Learn Stochastic Decision Policies

To solve reinforcement learning problems, many learning classifier systems (LCSs) are designed to learn state-action value functions through a compact set of maximally general and accurate rules. Most of these systems focus primarily on learning deterministic policies by using a greedy action selection strategy. However, in practice, it may be more flexible and desirable to learn stochastic policies, which can be considered as direct extensions of their deterministic counterparts. In this paper, we aim to achieve this goal by extending each rule with a new policy parameter. Meanwhile, a new method for adaptive learning of stochastic action selection strategies based on a policy gradient framework has also been introduced. Using this method, we have developed two new learning systems, one based on a regular gradient learning technology and the other based on a new natural gradient learning method. Both learning systems have been evaluated on three different types of reinforcement learning problems. The promising performance of the two systems clearly shows that LCSs provide a suitable platform for efficient and reliable learning of stochastic policies.

Adaptive natural gradient learning algorithms for Mackey–Glass chaotic time prediction

In this paper, we introduce the adaptive natural gradient method into the multilayer perceptrons (MLPs) and radial basis function (RBF) networks. We give a good performance for the Mackey–Glass chaotic time series prediction, and compare it with the LMA. Results show that the adaptive natural gradient methods for MLPs and RBFs, which are the online learning, can give almost the same performance with the LMA (MLPs), which is the batch mode learning. However, the performance of LMA (RBFs) is very poor and is very sensitive with the initial parameters.

Natural gradient learning algorithms for RBF networks.

Radial basis function (RBF) networks are one of the most widely used models for function approximation and classification. There are many strange behaviors in the learning process of RBF networks, such as slow learning speed and the existence of the plateaus. The natural gradient learning method can overcome these disadvantages effectively. It can accelerate the dynamics of learning and avoid plateaus. In this letter, we assume that the probability density function (pdf) of the input and the activation function are gaussian. First, we introduce natural gradient learning to the RBF networks and give the explicit forms of the Fisher information matrix and its inverse. Second, since it is difficult to calculate the Fisher information matrix and its inverse when the numbers of the hidden units and the dimensions of the input are large, we introduce the adaptive method to the natural gradient learning algorithms. Finally, we give an explicit form of the adaptive natural gradient learning algorithm and compare it to the conventional gradient descent method. Simulations show that the proposed adaptive natural gradient method, which can avoid the plateaus effectively, has a good performance when RBF networks are used for nonlinear functions approximation.

Theoretical Foundation for CMA-ES from Information Geometry Perspective

This paper explores the theoretical basis of the covariance matrix adaptation evolution strategy (CMA-ES) from the information geometry viewpoint. To establish a theoretical foundation for the CMA-ES, we focus on a geometric structure of a Riemannian manifold of probability distributions equipped with the Fisher metric. We define a function on the manifold which is the expectation of fitness over the sampling distribution, and regard the goal of update of the parameters of sampling distribution in the CMA-ES as maximization of the expected fitness. We investigate the steepest ascent learning for the expected fitness maximization, where the steepest ascent direction is given by the natural gradient, which is the product of the inverse of the Fisher information matrix and the conventional gradient of the function. Our first result is that we can obtain under some types of parameterization of multivariate normal distribution the natural gradient of the expected fitness without the need for inversion of the Fisher information matrix. We find that the update of the distribution parameters in the CMA-ES is the same as natural gradient learning for expected fitness maximization. Our second result is that we derive the range of learning rates such that a step in the direction of the exact natural gradient improves the parameters in the expected fitness. We see from the close relation between the CMA-ES and natural gradient learning that the default setting of learning rates in the CMA-ES seems suitable in terms of monotone improvement in expected fitness. Then, we discuss the relation to the expectation-maximization framework and provide an information geometric interpretation of the CMA-ES.

A Simplified Natural Gradient LearningAlgorithm

Adaptive natural gradient learning avoids singularities in the parameter space of multilayer perceptrons. However, it requires a larger number of additional parameters than ordinary backpropagation in the form of the Fisher information matrix. This paper describes a new approach to natural gradient learning that uses a smaller Fisher information matrix. It also uses a prior distribution on the neural network parameters and an annealed learning rate. While this new approach is computationally simpler, its performance is comparable to that of adaptive natural gradient learning.

Anomaly Preserving <formula formulatype="inline"> <tex Notation="TeX">$\ell _{\scriptscriptstyle 2,\infty }$</tex> </formula>-Optimal Dimensionality Reduction Over a Grassmann Manifold

In this paper, we address the problem of redundancy reduction of high-dimensional noisy signals that may contain anomaly (rare) vectors, which we wish to preserve. Since anomaly data vectors contribute weakly to the l2-norm of the signal as compared to the noise, l2 -based criteria are unsatisfactory for obtaining a good representation of these vectors. As a remedy, a new approach, named Min-Max-SVD (MX-SVD) was recently proposed for signal-subspace estimation by attempting to minimize the maximum of data-residual l2-norms, denoted as l2,l and designed to represent well both abundant and anomaly measurements. However, the MX-SVD algorithm is greedy and only approximately minimizes the proposed l2,l-norm of the residuals. In this paper we develop an optimal algorithm for the minization of the l2,l-norm of data misrepresentation residuals, which we call Maximum Orthogonal complements Optimal Subspace Estimation (MOOSE). The optimization is performed via a natural conjugate gradient learning approach carried out on the set of n dimensional subspaces in IRm, m ≫ n, which is a Grassmann manifold. The results of applying MOOSE, MX-SVD, and l2- based approaches are demonstrated both on simulated and real hyperspectral data.

Adaptive Improved Natural Gradient Algorithm for Blind Source Separation

We propose an adaptive improved natural gradient algorithm for blind separation of independent sources. First, inspired by the well-known backpropagation algorithm, we incorporate a momentum term into the natural gradient learning process to accelerate the convergence rate and improve the stability. Then an estimation function for the adaptation of the separation model is obtained to adaptively control a step-size parameter and a momentum factor. The proposed natural gradient algorithm with variable step-size parameter and variable momentum factor is therefore particularly well suited to blind source separation in a time-varying environment, such as an abruptly changing mixing matrix or signal power. The expected improvement in the convergence speed, stability, and tracking ability of the proposed algorithm is demonstrated by extensive simulation results in both time-invariant and time-varying environments. The ability of the proposed algorithm to separate extremely weak or badly scaled sources is also verified. In addition, simulation results show that the proposed algorithm is suitable for separating mixtures of many sources (e.g., the number of sources is 10) in the complete case.

Stability analysis of natural gradient learning rules in overdetermined ICA

The present work deals with the ICA problem in the overdetermined (or undercomplete) case, i.e. the sensors outnumber the sources. As appeared recently in the literature, the natural gradient (NG) approach is an efficient way to get fast convergence in ICA learning rules, also in our case of interest. However, up to the author's knowledge, the stability properties of such algorithms have been analyzed only in the complete ICA. In order to face this lack, a general framework for the stability analysis of the overdetermined NG based ICA learning rules is proposed here. In particular, it has been observed that the already existing algorithms do not have separating matrices as equilibrium points, as instead it occurs in case of an alternative standard gradient based ICA learning rule here proposed and in all related NG versions that can be derived from it. This and the property of being locally stable at the equilibrium for such new algorithms make them preferable w.r.t. the others for practical purposes.

Multiplicative updates for non-negative projections

We present here how to construct multiplicative update rules for non-negative projections based on Oja's iterative learning rule. Our method integrates the multiplicative normalization factor into the original additive update rule as an additional term which generally has a roughly opposite direction. As a consequence, the modified additive learning rule can easily be converted to its multiplicative version, which maintains the non-negativity after each iteration. The derivation of our approach provides a sound interpretation of learning non-negative projection matrices based on iterative multiplicative updates—a kind of Hebbian learning with normalization. A convergence analysis is scratched by interpretating the multiplicative updates as a special case of natural gradient learning. We also demonstrate two application examples of the proposed technique, a non-negative variant of the linear Hebbian networks and a non-negative Fisher discriminant analysis, including its kernel extension. The resulting example algorithms demonstrate interesting properties for data analysis tasks in experiments performed on facial images.

Stability Analysis of Natural Gradient Learning Rules in Complete ICA: A Unifying Perspective

This letter deals with the independent component analysis (ICA) problem in the complete case. As appeared recently in the literature, different Riemannian metrics can be defined within the parameter space (i.e., the general linear group), allowing to derive correspondingly various ICA learning rules based on the relative natural gradients (NGs). This letter proposes a general framework to analyze the stability of such learning rules, including the already published study focusing on the Amari's NG approach as a special case thereof. In particular, it is shown that the stability conditions known in the literature still hold in all cases addressed

A signal-adaptive algorithm for blind separation of sources with mixed kurtosis signs

This paper addresses the problem of Blind Source Separation (BSS) and presents a new BSS algorithm with a Signal-Adaptive Activation (SAA) function (SAA-BSS). By taking the sum of absolute values of the normalized kurtoses as a contrast function, the obtained signal-adaptive activation function automatically satisfies the local stability and robustness conditions. The SAA-BSS exploits the natural gradient learning on the Stiefel manifold, and it is an equivariant algorithm with a moderate computational load. Computer simulations show that the SAA-BSS can perform blind separation of mixed sub-Gaussian and super-Gaussian signals and it works more efficiently than the existing algorithms in convergence speed and robustness against outliers.

Adaptive nonlinear PCA algorithms for blind source separation without prewhitening

Blind source separation (BSS) aims at recovering statistically independent source signals from their linear mixtures without knowing the mixing coefficients. Besides independent component analysis, nonlinear principal component analysis (NPCA) is shown to be another useful tool for solving this problem, but it requires that the measured data be prewhitened. By taking into account the autocorrelation matrix of the measured data, we present in this paper a modified NPCA criterion, and develop a least-mean-square (LMS) algorithm and a recursive least-squares algorithm. They can perform the online BSS using directly the unwhitened observations. Since a natural gradient learning is applied and the prewhitening process is removed, the proposed algorithms work more efficiently than the existing NPCA algorithms, as verified by computer simulations on man-made sources as well as practical speech signals.

A Generalized Contrast Function and Stability Analysis for Overdetermined Blind Separation of Instantaneous Mixtures

In this letter, the problem of blind separation of n independent sources from their m linear instantaneous mixtures is considered. First, a generalized contrast function is defined as a valuable extension of the existing classical and nonsymmetrical contrast functions. It is applicable to the overdetermined blind separation (m > n) with an unknown number of sources, because not only independent components but also redundant ones are allowed in the outputs of a separation system. Second, a natural gradient learning algorithm developed primarily for the complete case (m = n) is shown to work as well with an n × m or m × m separating matrix, for each optimizes a certain mutual information contrast function. Finally, we present stability analysis for a newly proposed generalized orthogonal natural gradient algorithm (which can perform the overdetermined blind separation when n is unknown), obtaining an expectable result that its local stability conditions are slightly stricter than those of the conventional natural gradient algorithm using an invertible mixing matrix (m = n).

Implementing Online Natural Gradient Learning: Problems and Solutions

The online natural gradient learning is an efficient algorithm to resolve the slow learning speed and poor performance of the standard gradient descent method. However, there are several problems to implement this algorithm. In this paper, we proposed a new algorithm to solve these problems and then compared the new algorithm with other known algorithms for online learning, including Almeida-Langlois-Amaral-Plakhov algorithm (ALAP), Vario-eta, local adaptive learning rate and learning with momentum etc., using sample data sets from Proben1 and normalized handwritten digits, automatically scanned from envelopes by the U.S. Postal Services. The strong and weak points of these algorithms were analyzed and tested empirically. We found out that using the online training error as the criterion to determine whether the learning rate should be changed or not is not appropriate and our new algorithm has better performance than other existing online algorithms.

A Generalized Contrast Function and Stability Analysis for Overdetermined Blind Separation of Instantaneous Mixtures

In this letter, the problem of blind separation of n independent sources from their m linear instantaneous mixtures is considered. First, a generalized contrast function is defined as a valuable extension of the existing classical and nonsymmetrical contrast functions. It is applicable to the overdetermined blind separation (m > n) with an unknown number of sources, because not only independent components but also redundant ones are allowed in the outputs of a separation system. Second, a natural gradient learning algorithm developed primarily for the complete case (m = n) is shown to work as well with an n × m or m × m separating matrix, for each optimizes a certain mutual information contrast function. Finally, we present stability analysis for a newly proposed generalized orthogonal natural gradient algorithm (which can perform the overdetermined blind separation when n is unknown), obtaining an expectable result that its local stability conditions are slightly stricter than those of the conventional natural gradient algorithm using an invertible mixing matrix (m = n).

Part 2: Multilayer perceptron and natural gradient learning

Since the perceptron was developed for learning to classify input patterns, there have been plenty of studies on simple perceptrons and multilayer perceptrons. Despite wide and active studies in theory and applications, multilayer perceptrons still have many unsettled problems such as slow learning speed and overfitting. To find a thorough solution to these problems, it is necessary to consolidate previous studies, and find new directions for uplifting the practical power of multilayer perceptrons. As a first step toward the new stage of studies on multilayer perceptrons, we give short reviews on two interesting and important approaches; one is stochastic approach and the other is geometric approach. We also explain an efficient learning algorithm developed from the statistical and geometrical studies, which is now well known as the natural gradient learning method.