Online Streaming Feature Selection Using Sampling Technique and Correlations Between Features

Abstract

Feature selection is an important topic in data mining and machine learning, and has been extensively studied in many literature. In real-world applications, the dimensionality is extremely high, in millions, and keeps growing. Unlike traditional batch learning methods, online learning is more efficient for real-world applications. We define streaming features as features that flow in one by one over time whereas the number of training examples remains fixed. This is in contrast with the other kind of online learning methods that only deal with sequentially added instances. The key challenge for current online streaming feature selection is the large feature space, possibly of unknown or infinite size. To select a small number of features in an online manner more effectively, we propose a novel algorithm using sampling techniques and correlations between features. We evaluate the performance of the proposed algorithms for online streaming feature selection on several public datasets, and demonstrate their applications to real-world problems as image classification in computer vision. From Experiments, we can see that our algorithm consistently surpassed the baseline algorithms for all the situations.