Design and implementation of ACO feature selection algorithm for data stream mining

Abstract

Big data confront many technical challenges that also confront by both academic research communities and commercial IT deployment. Data streams with the curse of dimensionality are founded to be the root sources of Big Data. The commonly used procedure for data sourced from data streams is continuously making batch based model and inducing algorithms which is infeasible for real-time data mining. An optimal feature subset which is derived by mining over high dimensional data search space grows exponentially in size which leads to an intractable demand in computation. In order to solve this problem which is based on high dimensionality and streaming format of data in big data, a new feature selection is proposed in this paper which work on the data collected from e-commerce sites having high dimensionality and particularly designed for mining data, by using ant colony optimization (ACO) type of swarm search for selecting proper feature using the association rule to search more similar features. As proposed method use ant colony optimization (ACO) with association top-k rule which is contributing to achieve reduced pre-processing time required for data mining and improvement in analytical accuracy.