Bayesian inference for hierarchical mixtures-of-experts with applications to regression and classification.

Abstract

This paper studies the problems of inference and prediction in a class of models known as hierarchical mixtures-of-experts (HME). The statistical model underlying an HME is a mixture model in which both the mixture coefficients and the mixture components are generalized linear models. Bayesian inference regarding an HME's parameters is presented in the contexts of regression and classification using Markov chain Monte Carlo methods. A benefit of this Bayesian approach is the ability to obtain a sample from the posterior distribution of any functional of the parameters of the given model. In this way, more information is obtained than provided by a point estimate. The methods are illustrated on a nonlinear regression problem and on a breast cancer classification problem. The results indicate that the HME showed good prediction performance, and also gave the additional benefit of providing for the opportunity to assess the degree of certainty of the model in its predictions.