Feature reduction based LDA with SVM classification on dimensionality reduction for Big Data

Abstract

Analytics of Big data research has been entering the latest processes of "fast-data", in which every second many Giga Bytes of data arriving towards a massive structure of data. Based on the number, speed, importance, variation, uncertainty and veracity of the collected data, current Big data applications gather dynamic data sources and thus create massive unstructured Big Data. Data sources that are decreased and significant are deemed more valuable than raw, repetitive, unreliable, and noisy data set. A further prospect for reducing the big data whereas the thousands of attributes in large data sets are the cause of the dimensionality which takes infinite computing resources to expose working patterns of information. Not each feature in the generated datasets is essential for the training of computer algorithms. Any characteristics do not influence the effects of the forecast and some may be negligible. The ignorance of this trivial or less important characteristics lowers the pressure on the algorithms of Machine Learning (ML). The MapReduce technology in existing has also been used to decrease dimensionality, but without decreasing irrelevant features it takes all data for a direct reduction, which contributes to lower classification precision.