Machine learning based feature selection and knowledge reasoning for CBR system under big data

Abstract

Under big data, large number of features as well as their complex data types makes traditional feature selection and knowledge reasoning in CBR system not adapt to new condition. To solve these problems, first, this paper proposes Weighted Relative Probability Change of Solution Parameters (WRPCSP) algorithm to execute feature selection. Then, this paper integrates Bayesian network (BN) with CBR system for knowledge reasoning. Based on probability calculation and reasoning, WRPCSP algorithm together with BN allows the proposed CBR system to well work under big data. In addition, to overcome the efficiency problem caused by large number of features, this paper also proposes Group-Outside (GO) algorithm to assign the computing task of big data for parallel data processing. GO algorithm can make the computing capacity of Hadoop fully utilized to gain the least time costing for parallel data processing. Finally, lots of experiments are performed to validate the proposed method.