E2PAMEA: A fast evolutionary algorithm for extracting fuzzy emerging patterns in big data environments

Abstract

In this paper, a cooperative-competitive multi-objective evolutionary fuzzy system called E2PAMEA is presented for the extraction of emerging patterns in big data environments. E2PAMEA follows an adaptive schema to automatically employ different genetic operators according to the learning needs, which avoid the tuning of some parameters. It also employs a token-competition-based procedure for updating an elite population where the best set of patterns found so far is stored. In addition, a novel MapReduce procedure for an efficient computation of the evaluation function employed for guiding the search process is proposed. The method, called Bit-LUT employs a pre-evaluation stage where data is represented as a look-up table made of bit sets. This look-up table can be employed later in the chromosome evaluation by means of bitwise operations, reducing the computational complexity of the process.The experimental study carried out shows that E2PAMEA is a promising alternative for the extraction of high-quality emerging patterns in big data. In addition, the proposed Bit-LUT evaluation shows a significant improvement on efficiency with a great scalability capacity on both dimensions of data, which enables the processing of massive datasets faster than other alternatives.