A mapreduce fuzzy techniques of big data classification

Abstract

Due to the huge increase in the size of the data it becomes troublesome to perform efficient analysis using the current traditional techniques. Big data put forward a lot of challenges due to its several characteristics like volume, velocity, variety, variability, value and complexity. Today there is not only a necessity for efficient data mining techniques to process large volume of data but in addition a need for a means to meet the computational requirements to process such huge volume of data. The objective of this research is to implement a map reduce paradigm using fuzzy and crisp techniques, and to provide a comparative study between the results of the proposed systems and the methods reviewed in the literature. In this paper four proposed system is implemented using the map reduce paradigm to process on big data. First, in the mapper there are two techniques used; the fuzzy k-nearest neighbor method as a fuzzy technique and the support vector machine as non-fuzzy technique. Second, in the reducer there are three techniques used; the mode, the fuzzy soft labels and Gaussian fuzzy membership function. The first proposed system is using the fuzzy KNN in the mapper and the mode in the reducer, the second proposed system is using the SVM in the mapper and the mode in the reducer, the third proposed system is using the SVM in the mapper and the soft labels in the reducer, and the fourth proposed system is using the SVM in the mapper and fuzzy Gaussian membership function in the reducer. Results on different data sets show that the fuzzy proposed methods outperform a better performance than the crisp proposed method and the method reviewed in the literature.