A Novel Multi-class Ensemble model based on feature selection using Hadoop framework for classifying imbalanced Biomedical Data

Abstract

Due to the exponential growth of biomedical repositories such as PubMed and Medline, an accurate predictive model is essential for knowledge discovery in Hadoop environment. Traditional decision tree models such as multivariate Bernoulli model, random forest and multinominal naive Bayesian tree use attribute selection measures to decide best split at each node of the decision tree. Also, the efficiency of document analysis in Hadoop framework is limited mainly due to the class imbalance problem and large candidate sets. In this paper, we proposed a two phase map-reduce framework with text preprocessor and classification model. In the first phase, mapper based preprocessing method was designed to eliminate irrelevant features, missing values and outliers from the biomedical data. In the second phase, a map-reduce based multi-class ensemble decision tree model was designed and implemented on the preprocessed mapper data to improve the true positive rate and computational time. The experimental results on the complex biomedical datasets show that the performance of our proposed Hadoop based multi-class ensemble model significantly outperforms state-of-the-art baselines.