Multiple Extreme Learning Machine Research Articles

BE-ELM: Biological ensemble Extreme Learning Machine without the need of explicit aggregation

Extreme learning machines (ELMs) are commonly adopted as base learners in ensemble methods due to unstable results and fast learning speed. However, most existing ensemble structures require explicit aggregation of ELM base learners’ results before the final decision. This study proposes a novel Biological Ensemble ELMs called BE-ELM from a biological perspective for the first time and exploit the superiority of assembling multiple ELM base learners in parallel without the need for explicit aggregation, and thus simplifying the learning procedure. BE-ELM’s structure is inspired by recent MIT neuroscience findings of brain learning mechanisms. The expression of the analytical solution of BE-ELM is similar to that of basic ELMs, which allows it to inherit their fast learning speed in the ensemble structure. Moreover, BE-ELM also has the added advantage of superior performance. Our theoretical analysis shows that BE-ELM consisting of multiple ELM base learners built on subsets of the original features is equivalent to a more complex ELM on the original feature space. We prove that BE-ELM, even without any explicit aggregation, can guarantee enhanced generalization capabilities. To confirm our findings, we conducted extensive experiments on various datasets. The results demonstrate that BE-ELM outperforms traditional ELMs and other state-of-the-art ensemble ELMs in terms of generalization performance on most datasets. These findings suggest that BE-ELM has the potential to improve prediction outcomes in practical applications.

A novel data-driven deep learning approach for wind turbine power curve modeling

Existing wind turbine power curve (WTPC) models have limited performance in capturing the complex relationship between wind speed and wind power due to their inadequate nonlinear fitting abilities. Deep learning (DL) excels at describing complex relationships. However, it is typically not applicable to WTPC modeling with a single wind speed input. This study proposes a novel data-driven DL approach mELM-CA-CNN to establish WTPCs based on multiple extreme learning machines (ELMs), channel attention (CA), convolutional neural network (CNN), and Huber loss (HL). First, multiple ELMs map a single wind speed to various high-dimensional feature spaces. Then, CA helps reduce redundant mappings of ELMs. Next, CNN extracts important features from all ELM mappings and models the complex relationship between wind speed and the corresponding power. Finally, the proposed model is trained with the differentiable and robust HL. To reduce the adverse impact of outliers on WTPC modeling, a segmented data cleaning approach based on 3 σ criterion and quartile algorithm is proposed. Comparisons with ten popular WTPC models demonstrate that mELM-CA-CNN obtains the most accurate WTPCs on four wind datasets, showing the superiority of the proposed DL approach. Moreover, the roles of the different modules of mELM-CA-CNN in improving model performance are verified.

Multiple Extreme Learning Machines Based Arrival Time Prediction for Public Bus Transport

Due to fast-growing urbanization, the traffic management system becomes a crucial problem owing to the rapid growth in the number of vehicles The research proposes an Intelligent public transportation system where information regarding all the buses connecting in a city will be gathered, processed and accurate bus arrival time prediction will be presented to the user. Various linear and time-varying parameters such as distance, waiting time at stops, red signal duration at a traffic signal, traffic density, turning density, rush hours, weather conditions, number of passengers on the bus, type of day, road type, average vehicle speed limit, current vehicle speed affecting traffic are used for the analysis. The proposed model exploits the feasibility and applicability of ELM in the travel time forecasting area. Multiple ELMs (MELM) for explicitly training dynamic, road and trajectory information are used in the proposed approach. A large-scale dataset (historical data) obtained from Kerala State Road Transport Corporation is used for training. Simulations are carried out by using MATLAB R2021a. The experiments revealed that the efficiency of MELM is independent of the time of day and day of the week. It can manage huge volumes of data with less human intervention at greater learning speeds. It is found MELM yields prediction with accuracy in the range of 96.7% to 99.08%. The MAE value is between 0.28 to 1.74 minutes with the proposed approach. The study revealed that there could be regularity in bus usage and daily bus rides are predictable with a better degree of accuracy. The research has proved that MELM is superior for arrival time predictions in terms of accuracy and error, compared with other approaches.

Forecasting High-Flow Discharges in a Flashy Catchment Using Multiple Precipitation Estimates as Predictors in Machine Learning Models

The use of machine learning (ML) for predicting high river flow events is gaining prominence and among its non-trivial design decisions is the definition of the quantitative precipitation estimate (QPE) product included in the input dataset. This study proposes and evaluates the use of multiple concurrent QPEs to improve the performance of a ML model towards the forecasting of the discharge in a flashy urban catchment. Multiple extreme learning machine (ELM) models were trained with distinct combinations of QPEs from radar, reanalysis, and gauge datasets. Their performance was then assessed in terms of goodness of fit and contingency analysis for the prediction of high flows. It was found that multi-QPEs models overperformed the best of its single-QPE counterparts, with gains in Kling-Gupta efficiency (KGE) values up to 4.76% and increase of precision in detecting high flows up to 7.27% for the lead times in which forecasts were considered “useful”. The novelty of these results suggests that the implementation of ML models could achieve better performance if the predictive features related to rainfall data were more diverse in terms of data sources when compared with the currently predominant use of a single QPE product.

Composite fault diagnosis of analog circuit system using chaotic game optimization-assisted deep ELM-AE

When multiple electronic components in an analog circuit fail simultaneously, since its high complexity, it is arduous to establish an accurate fault model and diagnose the composite faults precisely. Traditional approach for composite fault diagnosis is severely constrained since it excessively relies on domain expertise and signal processing technology. The rapid development of deep learning provides a new idea for it. In this study, an improved strategy based on deep extreme learning machine with autoencoder (DELM-AE) is proposed, which possesses more prominent capability on feature extraction and representation. The proposed method first utilizes multiple extreme learning machine with autoencoder (ELM-AE) to adaptively extract the hidden features from original collected signals layer by layer, which does not need to design representative features manually. Subsequently, efficient classification will be conducted based on extreme learning machine algorithm. All parameter values of the proposed network are determined at one time, eliminating the iterative updating process. To further enhance the performance of DELM-AE on feature extraction and classification, a novel chaos game optimization algorithm (CGO) is introduced to optimize the initial network parameters. The experimental results on two typical test circuits demonstrate that the proposed method, CGO-DELM-AE, can effectively diagnose the composite faults of analog circuit.

Monthly and seasonal hydrological drought forecasting using multiple extreme learning machine models

ABSTRACT Hydrological drought forecasting is a key component in water resources modeling as it relates directly to water availability. It is crucial in managing and operating dams, which are constructed in rivers. In this study, multiple extreme learning machines (ELMs) are utilized to forecast hydrological drought. For this purpose, the standardized hydrological drought index (SHDI) and standardized precipitation index (SPI) are computed for 1 and 3 aggregated months. Two scenarios are considered, namely, using SHDI in previous months as the input, and using SHDI and SPI in previous months as the input. Considering these scenarios and two timescales (1 and 3 months), 12 input–output combinations are generated. Then, five different ELMs and support vector machine models are used to predict the SHDI on both timescales. For preprocessing of the data, the wavelet is hybridized with the models, leading to 144 different models. The results indicate that ELMs are capable of forecasting SHDI with high precision. The self-adaptive differential evolution ELM outperforms the other models and the wavelet has a highly positive effect on the model performance, especially in error reduction. In general, using ELMs in hydrological drought forecasting is promising and this model can feasibly be used for this purpose.

Detection of illegal traffic pattern using Hybrid Improved CART and Multiple Extreme Learning Machine Approach

In the proposed hybrid intrusion detection process, misuse detection and anomaly detection model is integrated to detect the attack in traffic pattern. In misuse detection model, the traffic pattern is classified into known attack and not known attack. Each extracted normal data set does not have known attack and it contains small amount of varied connection patterns than overall normal data set. Anomaly detection model classifies the not known attack as normal data set and unknown attack thus improving the performance of normal traffic behavior. Experiment is carried out using NSL –KDD dataset and performance of proposed approach is compared with traditional learning approaches in terms of training time, testing time, false positive ratio and detection ratio. The proposed method detects the known attacks and unknown attacks with ratio of 99.8 % and 52% respectively.

Expensive Multiobjective Optimization Algorithm Based on Equivariate Component Analysis

In order to reduce the cost of candidate solution evaluation in the process of solving expensive optimization problems, an expensive multi-objective optimization algorithm based on equivalence component analysis was proposed to study the influence of decision space equivalence components on the prediction accuracy of agent models. Based on the analysis of the equivalence of decision space attributes, a limit learning network based on the equivalence components was constructed for Pareto dominance prediction among candidate solutions. A multi-objective test problem with equivalent components was selected in Pareto dominance prediction experiments, the results of which showed that the algorithm can effectively improve the accuracy of Pareto dominance prediction among candidate solutions. Successively the candidate solutions were scored with multiple ELM (Extreme Learning Machine) models, selected for evaluation and updated, and integrated into the Pareto-based multi-objective evolutionary algorithm. Through comparative experiments on the test problem, the method could achieve a better Pareto approximation solution under the limitation of a limited number of evaluations, and the goal of reducing the cost of expensive multi-objective optimization calculations.

Novel hybrid machine learning optimizer algorithms to prediction of fracture density by petrophysical data

One of the challenges in reservoir management is determining the fracture density (FVDC) in reservoir rock. Given the high cost of coring operations and image logs, the ability to predict FVDC from various petrophysical input variables using a supervised learning basis calibrated to the standard well is extremely useful. In this study, a novel machine learning approach is developed to predict FVDC from 12-input variable well-log based on feature selection. To predict the FVDC, combination of two networks of multiple extreme learning machines (MELM) and multi-layer perceptron (MLP) hybrid algorithm with a combination of genetic algorithm (GA) and particle swarm optimizer (PSO) has been used. We use a novel MELM-PSO/GA combination that has never been used before, and the best comparison result between MELM-PSO-related models with performance test data is RMSE = 0.0047 1/m; R2 = 0.9931. According to the performance accuracy analysis, the models are MLP-PSO < MLP-GA < MELM-GA < MELM-PSO. This method can be used in other fields, but it must be recalibrated with at least one well. Furthermore, the developed method provides insights for the use of machine learning to reduce errors and avoid data overfitting in order to create the best possible prediction performance for FVDC prediction.

Extreme Learning Machine for Accurate Indoor Localization Using RSSI Fingerprints in Multifloor Environments

A new extreme learning machine (ELM) localization technique that uses received signal strength indicator fingerprints only is proposed for multifloor environments. This structured scheme forms multiple individual ELMs for the floors as well as for the geographically formed data clusters of each floor. Multifloor environments often have huge amount of training and online measurement data. To maximize efficiency, we develop a data preprocessing algorithm, aiming to: 1) efficiently extract out only the essential information from the vast amount of data sets and reduce the data dimension and 2) transform the floor-level data sets and positioning data sets of each floor into a proper structure that is suitable for the proposed ensemble ELM technique. The proposed solution is unique in that its offline phase exploits multiple individual ELMs for all floors to generate a set of floor-level classification functions with the preprocessed training data sets, and for each floor, it exploits multiple ELMs for the data clusters to generate a set of position regression functions. The online phase executes a coarse localization step to estimate the floor by using the floor-level classification functions and a refined step to estimate the position on the floor by using the position regression functions. The proposed algorithm and several existing algorithms are implemented to perform localization using the same measured datasets in a multistory building. For both floor estimation and localization on the floor, it outperforms existing schemes. For most cases, the performance gap is substantial.

Large Scale Identification of Osteosarcoma Pathogenic Genes by Multiple Extreme Learning Machine.

At present, the main treatment methods of osteosarcoma are chemotherapy and surgery. Its 5-year survival rate has not been significantly improved in the past decades. Osteosarcoma has extremely complex multigenomic heterogeneity and lacks universally applicable signal blocking targets. Osteosarcoma is often found in adolescents or children under the age of 20, so it is very important to explore its genetic pathogenic factors. We used known osteosarcoma-related genes and computer algorithms to find more osteosarcoma pathogenic genes, laying the foundation for the treatment of osteosarcoma immune microenvironment-related treatments, so as to carry out further explorations on these genes. It is a traditional method to identify osteosarcoma related genes by collecting clinical samples, measuring gene expressions by RNA-seq technology and comparing differentially expressed gene. The high cost and time consumption make it difficult to carry out research on a large scale. In this paper, we developed a novel method “RELM” which fuses multiple extreme learning machines (ELM) to identify osteosarcoma pathogenic genes. The AUC and AUPR of RELM are 0.91 and 0.88, respectively, in 10-cross validation, which illustrates the reliability of RELM.

Robust hybrid machine learning algorithms for gas flow rates prediction through wellhead chokes in gas condensate fields

Condensate reservoirs are the most challenging hydrocarbon reservoirs in the world. The behavior of condensate gas reservoirs regarding pressure and temperature variation is unique. Adjusting fluid flow rate through wellhead chokes of condensate gas wells is critical and challenging for reservoir management. Predicting this vital parameter is a big step for the development of condensate gas fields. In this study, a novel machine learning approach is developed to predict gas flow rate (Qg) from six input variables: temperature (T); upstream pressure (Pu); downstream pressure (Pd); gas gravity (γg); choke diameter (D64) and gas–liquid ratio (GLR). Due to the absence of accurate recombination methods for determining Qg, machine learning methods offer a functional alternative approach. Four hybrid machine learning (HML) algorithms are developed by integrating multiple extreme learning machine (MELM) and least squares support vector machine (LSSVM) with two optimization algorithms, the genetic algorithm (GA) and the particle swarm optimizer (PSO). The evaluation conducted on prediction performance and accuracy of the four HML models developed indicates that the MELM-PSO model has the highest Qg prediction accuracy achieving a root mean squared error (RMSE) of 2.8639 Mscf/d and a coefficient of determination (R2) of 0.9778 for a dataset of 1009 data records compiled from gas-condensate fields around Iran. Comparison of the prediction performance of the HML models developed with those of the previous empirical equations and artificial intelligence models reveals that the novel MELM-PSO model presents superior prediction efficiency and higher computational accuracy. Moreover, the Spearman correlation coefficient analysis performed demonstrates that D64 and GLR are the most influential variables in the gas flow rate for the large dataset evaluated in this study.

Hybrid machine learning algorithms to predict condensate viscosity in the near wellbore regions of gas condensate reservoirs

Gas condensate reservoirs display unique phase behavior and are highly sensitive to reservoir pressure changes. This makes it difficult to determine their PVT characteristics, including their condensate viscosity, which is a key variable in determining their flow behavior. In this study, a novel machine learning approach is developed to predict condensate viscosity in the near wellbore regions (μc) from five input variables: pressure (P), temperature (T), initial gas to condensate ratio (RS), gas specific gravity (γg), and condensate gravity (API). Due to the absence of accurate recombination methods for determining μc machine learning methods offer a useful alternative approach. Nine machine learning and hybrid machine learning algorithms are evaluated including novel multiple extreme learning machine (MELM), least squares support vector machine (LSSVM) and multi-layer perceptron (MLP) and each hybridized with a particle swarm optimizer (PSO) and genetic algorithm (GA). The new MELM algorithm has some advantages including 1) rapid execution, 2) high accuracy, 3) simple training, 4) avoidance of overfitting, 5) non-linear conversion during training, 6) no trapping at local optima, 6) fewer optimization steps than SVM and LSSVM. Combining MELM with PSO, to find the best control parameters, further improves its performance. Analysis indicates that the MELM-PSO model provides the highest μc prediction accuracy achieving a root mean squared error (RMSE) of 0.0035 cP and a coefficient of determination (R2) of 0.9931 for a dataset of 2269 data records compiled from gas-condensate fields around the world. The MELM-PSO algorithm generates no outlying data predictions. Spearman correlation coefficient analysis identifies that P, γg and Rs are the most influential variables in terms of condensate viscosity based on the large dataset studied.

Variable-weighted FDA combined with t-SNE and multiple extreme learning machines for visual industrial process monitoring

The visualization of an operating state of industrial processes allows operators to identify and diagnose faults intuitively and quickly. The identification and diagnosis of faults are important for ensuring industrial production safety. A method that combines variable-weighted Fisher discriminant analysis (VWFDA), t-distributed stochastic neighbor embedding (t-SNE), and multiple extreme learning machines (ELMs) is proposed for visual process monitoring. First, the VWFDA weighs variables on the basis of their contribution to the fault, thereby amplifying the fault information. The VWFDA is used to extract feature vectors from industrial data, and normal state and various fault states can be separated from each other in the space formed by these feature vectors. Second, t-SNE is used to visualize these feature vectors. Third, given that t-SNE lacks a transformation matrix during dimension reduction, one ELM is used for each class data of t-SNE to obtain the mapping relation from its input data to its mapping points. Finally, the VWFDA and multiple trained ELMs are combined for online process monitoring. The performance of the proposed approach is compared with that of FDA–t-SNE and other methods on the basis of the Tennessee Eastman process, thereby confirming that the proposed approach is advantageous for visual industrial process monitoring.

Hybrid Ensemble Pruning Using Coevolution Binary Glowworm Swarm Optimization and Reduce-Error

Ensemble pruning has been widely applied to improve the capacity of multiple learner system. Both diversity and classification accuracy of learners are considered as two key factors for achieving an ensemble with competitive classification ability. Considering that extreme learning machine (ELM) is characterized by excellent training rate and generalization capability, it is employed as the base classifier. For a multiple ELM system, when we increase its constituents’ diversity, the mean accuracy of the whole members must be decreased. Therefore, a compromise between them can ensure that the ELMs remain good diversity and high precision, but finding the compromise brings a heavy computational burden. It is hard to look for the exact result via the searching of intelligent algorithms or pruning of diversity measures. On the basis, we propose a hybrid ensemble pruning approach employing coevolution binary glowworm swarm optimization and reduce-error (HEPCBR). Considering the good performance of reduce-error (RE) in selecting ELMs with high diversity and precision, we try to employ RE to choose the satisfactory ELMs from the generated ELMs. In addition, the constituents are further selected via the proposed coevolution binary glowworm swarm optimization, which are utilized to construct the promising ensemble. Experimental results indicate that, compared to other frequently used methods, the proposed HEPCBR achieves significantly superior performance in classification.

Bayesian robust multi-extreme learning machine

Outliers usually exist in the collected data due to human or instrumentation errors. Their existence makes the regression error obey a non-Gaussian distribution. So, people assume that the regression error obeys finite mixture distributions, such as mixture of Gaussians and mixture of Student’s t-distributions. This paper proposed two Bayesian robust regression models based on multiple extreme learning machines. First, the finite mixture error distributions are extended to their infinite counterparts according to the theory of stick-breaking construction, which can avoid selecting the optimal number of components in mixture models. Second, a multi-extreme learning machine, which combines many single extreme learning machines with different numbers of hidden nodes, is constructed, which can avoid the effects of different numbers of hidden nodes on the final regression performance. Besides, sparse Bayesian learning has also been derived to perform the sparse model weight and output weight inference automatically. The experimental results on artificial datasets and eight benchmark datasets show that the proposed two models outperform the other compared models under different rates of outliers. Also, they generate better multi-step ahead wind speed and traffic speed forecasts in real applications.

KAEA: A Novel Three-stage Ensemble Model for Software Defect Prediction

Software defect prediction is a research hotspot in the field of software engineering. However, due to the limitations of current machine learning algorithms, we can’t achieve good effect for defect prediction by only using machine learning algorithms. In previous studies, some researchers used extreme learning machine (ELM) to conduct defect prediction. However, the initial weights and biases of the ELM are determined randomly, which reduces the prediction performance of ELM. Motivated by the idea of search based software engineering, we propose a novel software defect prediction model named KAEA based on kernel principal component analysis (KPCA), adaptive genetic algorithm, extreme learning machine and Adaboost algorithm, which has three main advantages: (1) KPCA can extract optimal representative features by leveraging a nonlinear mapping function; (2) We leverage adaptive genetic algorithm to optimize the initial weights and biases of ELM, so as to improve the generalization ability and prediction capacity of ELM; (3) We use the Adaboost algorithm to integrate multiple ELM basic predictors optimized by adaptive genetic algorithm into a strong predictor, which can further improve the effect of defect prediction. To effectively evaluate the performance of KAEA, we use eleven datasets from large open source projects, and compare the KAEA with four machine learning basic classifiers, ELM and its three variants. The experimental results show that KAEA is superior to these baseline models in most cases.

Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling

In a human–machine cooperation system, assessing the mental workload (MW) of the human operator is quite crucial to maintaining safe operation conditions. Among various MW indicators, electroencephalography (EEG) signals are particularly attractive because of their high temporal resolution and sensitivity to the occupation of working memory. However, the individual difference of the EEG feature distribution may impair the machine-learning based MW classifier. In this paper, we employed a fast-training neural network, extreme learning machine (ELM), as the basis to build an individual-specific classifier ensemble to recognize binary MW. To improve the diversity of the classification committee, heterogeneous member classifiers were adopted by fusing multiple ELMs and Bayesian models. Specifically, a deep network structure was applied in each weak model aiming at finding informative EEG feature representations. The structure of hyper-parameters of the proposed heterogeneous ensemble ELM (HE-ELM) was then identified and then its performance was compared against several competitive MW classifiers. We found that the HE-ELM model was superior for improving the individual-specific accuracy of MW assessments.

Seemingly unrelated extreme learning machine

A seemingly unrelated regression (SUR) system simultaneously studies the groups of samples that are related with each other through the covariance of decision attributes. Each group of samples is studied by a regression equation, and the error terms of the regression equations are correlated in SUR system. Extreme Learning Machine (ELM) is a training method for single-hidden layer feedforward neural network, it is widely used in many machine learning domains due to the good generalization capability and fast training speed. Since a single ELM ignores the correlated information among different groups of samples, it fails to solve the SUR problem effectively. This ineffectiveness becomes more obvious with the correlation among equations going up. In order to overcome this problem, an extended ELM model is proposed in this paper, described as Seemingly Unrelated ELM (SUELM). SUELM simultaneously learns multiple ELMs by sufficiently using the correlated information among different groups of samples, thus it can solve the SUR problem effectively. In comparison with a single ELM, SUELM significantly improves the performance. Simulation results show that SUELM performs better than the single ELM with respect to mean square error and generalization ability, especially when significant correlations exist among different groups of samples. This paper provides an effective way for solving the SUR problem by adopting ELM as the learning model.

A Stratified Method for Large-Scale Power System Transient Stability Assessment Based on Maximum Relevance Minimum Redundancy Arithmetic

A stratified method to transient stability assessment in large-scale power systems based on mutual information theory and artificial intelligence algorithm is proposed in this paper. A set of inter-complementary dynamic stability features are picked up one by one through the maximum-relevance minimum-redundancy (MRMR) algorithm. Besides, multiple extreme learning machines (ELMs) are trained based on the generated feature datasets. Because of the high requirement of evaluation speed in practical application, in order to balance the contradiction between assessment speed and accuracy, a hierarchical assessment structure is adopted in the final assessment process. Different ensemble classifiers with different response times are trained to construct different layers. The performance of the proposed technique is tested in the IEEE-39 bus system and a practical 1648 bus system provided by PSS/E. The experimental results indicate that, compared to other traditional methods, the proposed hierarchical method can give a more accurate result in a shorter period of time. As an efficient method, it is suitable for on-line transient stability assessment.