RBF nets, mixture experts, and Bayesian Ying–Yang learning

Abstract

The connections of the alternative model for mixture of experts (ME) to the normalized radial basis function (NRBF) nets and extended normalized RBF (ENRBF) nets are established, and the well-known expectation-maximization (EM) algorithm for maximum likelihood learning is suggested to the two types of RBF nets. This new learning technique determines the parameters of the input layer (including the covariance matrices or so-called receptive fields) and the parameters of the output layer in the RBF nets globally, instead of separately training the input layer by the K-means algorithm and the output layer by the least-squares learning as done in most of the existing RBF learning methods. In addition, coordinated competitive learning (CCL) and adaptive algorithms are proposed to approximate the EM algorithm for considerably speeding up the learning of the original and alternative ME models as well as the NRBF and ENRBF nets. Furthermore, the two ME models are linked to the recent proposed Bayesian Ying–Yang (BYY) learning system and theory such that not only the architecture of ME and RBF nets is shown to be more preferred than multilayer architecture, but also a new model selection criterion has been obtained to determine the number of experts and basis functions. A number of experiments are made on the prediction of foreign exchange rate and trading investment as well as piecewise nonlinear regression and piecewise line fitting. As shown in these experiments, the EM algorithm for NRBF nets and ENRBF nets obviously outperforms the conventional RBF learning technique, CCL speeds up the learning considerably with only a slight sacrifice on performance accuracy, the adaptive algorithm gives significant improvements on financial predication and trading investment, as well as that the proposed criterion can select the number of basis functions successfully. In addition, the ENRBF net and the alternative ME model are also shown to be able to implement curve fitting and detection.