Unsupervised pattern recognition of mixed data structures with numerical and categorical features using a mixture regression modelling framework

Abstract

In the present era of “Big Data”, data collection involving massive amount of features with a mix of variable types is commonplace. Mixture model-based techniques for statistical cluster analysis of mixed numerical and categorical feature data have their limitations, due to the difficulty in specifying appropriate component-densities when common multivariate distributions become invalid. This problem is particularly apparent in applications where the outcome feature variables are in a categorical form. An example of such an application is the analysis of binary morbidity data in national health survey, where the aims are to quantify heterogeneous comorbidity patterns of health conditions and identify (risk)-features of individuals that explain the heterogeneity. In this paper, we propose an unsupervised mixture regression model of multivariate generalised Bernoulli distributions for cluster analysis on the basis of categorical outcome features and mixed risk features. The proposed method is illustrated using simulated data and two real data sets concerning comorbidity patterns among 20,788 Australians who participated in the 2007–2008 National Health Survey (NHS) and among 470 patients who were recruited in a randomised controlled trial of a health intervention about in-patient detoxification from alcohol, heroin or cocaine in Boston. The method is also readily applicable to cluster more general mixed-feature data via the framework of consensus clustering.