Big Data Preprocessing Phase in Engendering Quality Data

Abstract

The change in the behavior of humans in the past decade has shown a tremendous generation in the data. The various researchers have given various definitions and discussed the different characteristics of big data. In the present study, we emphasize on the less focused areas of big data. One such zone is big data preprocessing. Extracting valuable information from big data has broadly three phases: first is acquisition and storage, second is data preprocessing, third is applying data mining and, at last, analysis of data. The contribution of this paper is that it shows generating the valuable information from big data not dependent on opting an advanced algorithm or novel algorithm but more than that it depends on acquisition of relevant data and preprocessing phase. The preprocessing phase plays a significant role in generating valuable data which serves as a great input in decision-making. At last, this paper gives a brief survey and analysis on big data preprocessing techniques used to handle imperfect data, reduction of data size and imbalanced data. It also theoretically discusses the different problems associated with the various phases and gives future directions where the researchers can work.