An Empirical Evaluation of Intelligent Machine Learning Algorithms under Big Data Processing Systems

Abstract

The rapid increase in the magnitude of data produced by industries that need to be processed using Machine Learning algorithms to generate business intelligence has created a dilemma for data scientists. This is due to the fact that traditional machine learning platforms such as Weka and R are not designed to handle data with such Volume, Velocity and Variety. Several machine learning algorithms and associated toolkits have been built specifically to work with big data; however, their performance is yet to be evaluated to allow researchers to get the most of these platforms. In this paper, the authors intend to provide an empirical evaluation of two emerging machine learning platforms under big data processing systems namely, H2O and Sparkling Water, by performing an experimental comparison between the two platforms in terms of performance over several generalization error metrics and model training time using the Santander Bank Dataset. Up to the authors’ knowledge, this is the first time such a study is conducted. The evaluation results showed that the H2O platform has significantly outperformed the Sparkling Water platform in terms of model training time almost by fifty percent, while achieving convergent results.