Finite mixtures of multivariate skew Student’s t distributions with independent logistic skewing functions

Abstract

This paper extends the multivariate skew t distributions with independent logistic skewing functions (MSTIL) introduced in Kwong and Nadarajah (Methodology and Computing in Applied Probability 24 (2022) 1669–1691) to finite mixture models (FM-MSTIL). A stochastic EM-type algorithm is proposed for fitting the FM-MSTIL, and a divisive hierarchical algorithm is proposed for initialisations and model selections. We show that the model can outperform other finite mixture models in the literature for some simulated data sets. The performance of the FM-MSTIL in cluster analysis is also investigated. We show that the FM-MSTIL-R, a nested version of the FM-MSTIL, performs well for automatic gating tasks on some flow cytometry data sets in the FlowCap-I challenge. The FM-MSTIL-R achieved a better overall score than all other competing algorithms in the original challenge. An efficient implementation of the FM-MSTIL is available as an R package in GitHub.