Bayesian Attribute Bagging-Based Extreme Learning Machine for High-Dimensional Classification and Regression

Abstract

This article presents a Bayesian attributebagging-based extreme learning machine (BAB-ELM)to handle high-dimensional classification and regression problems. First, thedecision-making degree (DMD)of a condition attribute is calculated based on the Bayesian decision theory, i.e., the conditional probability of the condition attribute given the decision attribute. Second, the condition attribute with the highest DMD is put into thecondition attribute group (CAG)corresponding to the specific decision attribute. Third, thebagging attribute groups (BAGs)are used to train an ensemble learning model ofextreme learning machines (ELMs).Each base ELM is trained on a BAG which is composed of condition attributes that are randomly selected from the CAGs. Fourth, the information amount ratios of bagging condition attributes to all condition attributes is used as the weights to fuse the predictions of base ELMs in BAB-ELM. Exhaustive experiments have been conducted to compare the feasibility and effectiveness of BAB-ELM with seven other ELM models, i.e., ELM, ensemble-based ELM (EN-ELM), voting-based ELM (V-ELM), ensemble ELM (E-ELM), ensemble ELM based on multi-activation functions (MAF-EELM), bagging ELM, and simple ensemble ELM. Experimental results show that BAB-ELM is convergent with the increase of base ELMs and also can yield higher classification accuracy and lower regression error for high-dimensional classification and regression problems.