RA-MRS: A high efficient attribute reduction algorithm in big data

Abstract

Efficient attribute reduction algorithm capable of handling high dimensional big data is one of the hot topics of rough set theory, and some related researchers have achieved with |C| jobs. In this paper, we present the definition of a marked reduction set and propose a more efficient attribute reduction algorithm (RA-MRS). The RA-MRS includes a batch processing phase and a vibration optimization phase, which reduce the number of jobs from |C| to log2|C|. Additionally, we provide an effective judgment strategy based on MapReduce, which supports the exception processing mechanism of Java to interrupt and advance the current job. Finally, the proposed algorithm is implemented in parallel based on Spark computing framework. The experimental results show that the proposed RA-MRS algorithm is over 99% faster than the classical PAAR_PR algorithm and 70% faster than the algorithm in the literature (Wang et al., 2020).