A hierarchical-coevolutionary-MapReduce-based knowledge reduction algorithm with robust ensemble Pareto equilibrium

Abstract

Listen

Translate article

With the rapid development of electronic, engineering, information and network technologies, large-scale data mining and knowledge discovery have become a new challenge. In this paper, a hierarchical-coevolutionary-MapReduce-based knowledge reduction algorithm (HCMRKR) with robust ensemble Pareto dominance is proposed for big data analysis. Firstly, a novel hierarchical co-evolutionary MapReduce framework with niche neighborhood radius is designed to divide the entire population into N subpopulations. The layered niche neighborhood radius is constructed so that subpopulations can be self-adaptively decomposed into the attribute approximate space with uncovering the underlying interaction structure of decision attributes. The reduction performances of elitist leaders are surely guaranteed to be the same with those using the whole independent dataset. Secondly, a robust ensemble strategy of reduction Pareto equilibrium is adopted to help elitist leaders from the Pareto front region to conduct the ensemble game of reduction subsets in different niche conic subspaces. Thus, most reduction solutions will be enhanced to be more robust within the stable elitists region. Thirdly, various elitist leaders will use the parallel MapReduce mechanism to determine the best strategy to extract their knowledge reduction set, and the adaptive reduction evolution with incremental learning can be implemented well. Experimental results illustrate that the proposed HCMRKR algorithm can effectively and efficiently handle the knowledge reduction problem in the big data sets as well as keeping more robust and stable performances, compared with existing state-of-the-art algorithms. Furthermore, the validation performed on two kinds of neonatal brain MRIs further demonstrates the HCMRKR's promising advantage for the real-world applications.