Variational Inference on Infinite Mixtures of Inverse Gaussian, Multinomial Probit and Exponential Regression

Abstract

We introduce a new class of methods and inference techniques for infinite mixtures of Inverse Gaussian, Multinomial Probit and Exponential Regression, models that belong to the widely applicable framework of Generalized Linear Model (GLM). We characterize the joint distribution of the response and covariates via a Stick-Breaking Prior. This leads to, in the various cases, nonparametric models for an infinite mixture of Inverse Gaussian, Multinomial Probit and Exponential Regression. Estimates of the localized mean function which maps the covariates to the response are presented. We prove the weak consistency for the posterior distribution of the Exponential model (SB-EX) and then propose mean field variational inference algorithms for the Inverse Gaussian, Multinomial Probit and Exponential Regression. Finally, we demonstrate their superior accuracy in comparison to several other regression models such as, Gaussian Process Regression, Dirichlet Process Regression, etc.