The Boltzmann g-RHONN: A learning machine for estimating unknown probability distributions

Abstract

This paper considers the following problem: assume that we have an ergodic signal source S that each time transmits a multidimensional signal x according to an unknown ergodic probability distribution with density p(x). Then the problem is to estimate the unknown density p(x). The problem is solved via gradient recurrent high-order neural network (g-RHONN) models whose weights are adjusted according to appropriate learning laws. In the proposed method the signals are considered to be the states of a stochastic gradient dynamical system (Langevin s.d.e.) after its convergence (in a stochastic manner). Then the (unknown) system is identified (approximately) using g-RHONNs. After the learning procedure converges, the energy function of the neural network is the estimate of the logarithm of the unknown probability distribution. Extensions are also provided for estimation of unknown joint probability distributions.