Flexible online multivariate regression with variational Bayes and the matrix-variate Dirichlet process

Victor Meng Hwee Ong,Taeryon Choi,Ajay Jasra,David J Nott

doi:10.3934/fods.2019006

Victor Meng Hwee Ong, Taeryon Choi + Show 2 more

Open Access

https://doi.org/10.3934/fods.2019006

Copy DOI

Abstract

Flexible density regression methods, in which the whole distribution of a response vector changes with the covariates, are very useful in some applications. A recently developed technique of this kind uses the matrix-variate Dirichlet process as a prior for a mixing distribution on a coefficient in a multivariate linear regression model. The method is attractive for the convenient way that it allows borrowing strength across different component regressions and for its computational simplicity and tractability. The purpose of the present article is to develop fast online variational Bayes approaches to fitting this model, and to investigate how they perform compared to MCMC and batch variational methods in a number of scenarios.

Highlights

Flexible regression modeling of multivariate data is an active research topic in data sciences, with a wide range of applications
We study variational computational methods for fitting the matrix-variate Dirichlet process mixture model of [37], extending the variational sequential updating and greedy search (VSUGS) approach by [36] for Dirichlet process mixtures of normal densities
The method we develop is computationally efficient, and especially useful as an alternative to Markov chain Monte Carlo (MCMC) for analysis of medium to large datasets

Summary

Introduction

Flexible regression modeling of multivariate data is an active research topic in data sciences, with a wide range of applications. Recent technological developments allow us to collect data with increasing volume and speed, making real time or on-line analyses attractive. While a number of on-line regression procedures exist to identify nonlinear relationships between predictors and response variables for real time data analysis, little has been done for on-line multivariate regression modeling when the whole distribution of the response is modelled flexibly. We consider a fast and flexible method for on-line multivariate regression modeling using variational Bayesian sequential updating algorithms. The model we consider is based on the matrix-variate Dirichlet process of [37] (hereafter MatDP). A MatDP is a Dirichlet process having a matrix-variate normal base measure, and following [37] we use the MatDP as a prior on a mixing distribution for

Objectives

Methods

Conclusion