Mutual Information-Based Variable Selection on Latent Class Cluster Analysis

Abstract

Machine learning techniques are becoming indispensable tools for extracting useful information. Among many machine learning techniques, variable selection is a solution used for converting high-dimensional data into simpler data while still preserving the characteristics of the original data. Variable selection aims to find the best subset of variables that produce the smallest generalization error; it can also reduce computational complexity, storage, and costs. The variable selection method developed in this paper was part of a latent class cluster (LCC) analysis—i.e., it was not a pre-processing step but, instead, formed part of LCC analysis. Many studies have shown that variable selection in LCC analysis suffers from computational problems and has difficulty meeting local dependency assumptions—therefore, in this study, we developed a method for selecting variables using mutual information (MI) in LCC analysis. Mutual information (MI) is a symmetrical measure of information that is carried by two random variables. The proposed method was applied to MI-based variable selection in LCC analysis, and, as a result, four variables were selected for use in LCC-based village clustering.