The effects of change in statistical properties of datasets on feature selection stability

Abstract

Data mining extracts previously not known knowledge from huge amount of stored operational data of organizations which can be used for managerial decision making. The datasets are mostly high dimensional due to the advancements in information and communication technologies. Feature selection is an important dimensionality reduction technique to manage the “curse of dimensionality”. The subset of features selected in subsequent iterations of feature selection algorithms must be same or at least similar for the small perturbations of the experimental dataset. The robustness of feature selection algorithms is called as the feature selection stability. High data quality with security/privacy is the major requirement of good privacy preserving data mining technique. This paper explores the change in statistical properties of datasets due to perturbations of datasets by the privacy preserving data mining techniques and their effects in feature selection stability.