Skew-normal Mixture of Experts

Abstract

Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification and clustering. For continuous data, MoE usually uses normal experts, that is, expert components following the Gaussian distribution. However, for a set of data containing a group or groups of observations with asymmetric distribution, the use of normal experts may be unsuitable. In this paper, we introduce the skew-normal MoE (SNMoE) which can deal with the issue regarding possibly skewed data distribution. We develop a dedicated expectation conditional maximization (ECM) algorithm to estimate the parameters of the proposed model by monotonically maximizing the observed data log-likelihood. We describe how the presented model can be used in prediction and in model-based clustering of regression data. Numerical experiments carried out on simulated data show the effectiveness of the proposed model in terms modeling non-linear regression functions as well as in model-based clustering. The proposed model is applied to two real-world data sets: the tone perception data and the temperature anomalies data.