Ensemble Extreme Learning Machine Research Articles

Hyperspectral image classification using ensemble extreme learning machine based on fuzzy entropy weights and auto-adapted spatial-spectral features

Hyperspectral image classification using ensemble extreme learning machine based on fuzzy entropy weights and auto-adapted spatial-spectral features

Hierarchical Cat and Mouse based ensemble extreme learning machine for spectrum sensing data falsification attack detection in cognitive radio network

While cognitive radio networks (CRNs) provide a promising solution to mitigate the scarcity of the radio spectrum, they are still susceptible to several security threats. Until now, only a few researchers considered the usage of intrusion detection systems (IDSs) to combat the threads against CRNs. In CRN, spectrum sensing is considered as a significant function and collaborative spectrum sensing (CSS) has known to result in better sensing accuracy. However, CSS is vulnerable to Spectrum Sensing Data Falsification (SSDF) attack wherein a node maliciously falsifies the sensing report prior to sending it to the Fusion Centre (FC), with the aim of disrupting the spectrum sensing process. Thus, a new novel machine learning model is significant for detecting of attacks in a CRN. In this paper, a Hierarchical Cat and Mouse Based Ensemble Extreme Learning Machine (HCM-EELM) model has proposed to analyse the security threats in CRN for enhancing the network performance by minimizing the attacks. Also, it is attempted at investigating the viability of machine learning classification for detecting SSDF attack in a binary reporting CRN. History of the sensing reports accumulated at the FC are applied to acquire the temporal characteristics of SU, and thereby establishing the training and testing data-sets. Here, three attacks namely Random Yes (RY) attack, Random No (RN) attack, and Random False (RF) attack have considered. Moreover, the proposed model is implemented in MATLAB software and the stimulation results are compared with existing models such as Extreme Learning Machine (ELM), Deep Neural Network (DNN), Recurrent Neural Network (RNN), and Support Vector Machine (SVM). The result has determined in terms of performance metrics such as accuracy, precision, recall, F1-score and FAR. As a result, better accuracy for 99.7% has achieved than the existing models by detecting the attacks efficiently.

ENIC: Ensemble and Nature Inclined Classification with Sparse Depiction based Deep and Transfer Learning for Biosignal Classification

The electrical activities of the brain are recorded and measured with Electroencephalography (EEG) by means of placing the electrodes on the scalp of the brain. It is quite a famous and versatile methodology utilized in both clinical and academic research activities. In this work, sparse depiction is initially incorporated to the EEG signals by means of using K-Singular Value Decomposition (K-SVD) algorithm and the features are extracted by means of using Self-Organizing Map (SOM) technique. The extracted features are initially classified with Extreme Learning Machine (ELM) and the proposed classification versions of ELM such as Ensemble ELM model and Nature Inclined ELM Model. The proposed ensemble ELM model makes use of the combination of Modified Adaboost. RT based on wavelet thresholding with ELM. The proposed Nature Inclined ELM makes use of the combination of some famous swarm intelligence algorithms such as Genetic Algorithm based ELM (GA-ELM), Particle Swarm Optimization based ELM (PSO-ELM), Ant Colony Optimization based ELM (ACO-ELM), Artificial Bee Colony based ELM (ABC-ELM) and Glowworm Swarm Optimization based ELM (GSO-ELM). The​ extracted features are also classified with deep learning methodology by means of utilizing an incidental Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN). Another famous methodology using Non-negative Matrix Factorization (NMF) and Affinity Propagation Congregation based Mutual Information (APCMI) with transfer learning techniques is also proposed and implemented once the sparse modelling is done and the results are analysed. The proposed methodology is implemented for two EEG datasets such as epilepsy dataset and schizophrenia dataset and a comprehensive analysis is done with very promising results.

Rotation transformation-based selective ensemble of one-class extreme learning machines

Extreme learning machine (ELM) possesses merits of rapid learning speed and good generalization ability. However, due to the random initialization of connection weights, the network outputs of ELM are usually unstable. Similar to ELM, one-class ELM (OCELM) also has the disadvantage of output instability. To enhance the stability and generalization performance of OCELM, a selective ensemble of OCELMs based on rotation transformation is proposed. First, principal component analysis (PCA)-based rotation transformation is utilized to construct different transformed training sets. Furthermore, several component OCELMs are trained independently on these training sets. Second, a dissimilarity measure based on angle cosine is used to evaluate the dissimilarity between each pair of OCELMs. The diversity of each component OCELM in the obtained ensemble can be further achieved. Thereafter, the component OCELMs with lower value of diversity are removed from the original ensemble. Finally, the voting strategy is utilized to determine that testing samples belong to the target class or the non-target class. Experimental results on 15 UCI benchmark data sets and one handwritten digit data set show that the proposed method is superior to its related approaches.

Energy Consumption Forecasting for Smart Meters Using Extreme Learning Machine Ensemble.

The employment of smart meters for energy consumption monitoring is essential for planning and management of power generation systems. In this context, forecasting energy consumption is a valuable asset for decision making, since it can improve the predictability of forthcoming demand to energy providers. In this work, we propose a data-driven ensemble that combines five single well-known models in the forecasting literature: a statistical linear autoregressive model and four artificial neural networks: (radial basis function, multilayer perceptron, extreme learning machines, and echo state networks). The proposed ensemble employs extreme learning machines as the combination model due to its simplicity, learning speed, and greater ability of generalization in comparison to other artificial neural networks. The experiments were conducted on real consumption data collected from a smart meter in a one-step-ahead forecasting scenario. The results using five different performance metrics demonstrate that our solution outperforms other statistical, machine learning, and ensembles models proposed in the literature.

Detection of adulterants in medicinal products by infrared spectroscopy and ensemble of window extreme learning machine

Motivated by the superiority of extreme learning machines (ELM), a so-called simple ensemble version, abbreviated as EWELM (ensemble window ELM), and was devised for multivariate calibration. In such an algorithm, the self-contained random operation of ELM provides an underlying way of generating diversity for designing ensemble algorithms. Fast learning speed of ELM makes the algorithm to be of practical value. A total of 41 samples of Jiangtangning capsules adulterated with metformin hydrochloride are prepared. The adulteration was performed within a concentration range of 0–29%. Partial least squares (PLS) and the full-spectrum ELM (EFELM) are selected as the reference algorithms. The experimental results proved the superiority of the EWELM algorithm to the references, indicating that it is a feasible screening tool to detect synthetic drug adulterants in Chinese patent medicine (TCM) on the market.

A study of extreme learning machine on small sample-sized classification problems

Extreme learning machine (ELM) is a special type of single hidden layer feedforward neural network that emphasizes training speed and optimal generalization. The ELM model proposes that the weights of hidden neurons need not be tuned, and the weights of output neurons can be calculated by finding the Moore-Penrose generalized inverse method. Thus, the ELM classifier is suitable to use in a homogeneous ensemble model due to the untuned random hidden weights which promote diversity even with the same training data. This paper studies the effectiveness of the ELM ensemble models in solving small sample-sized classification problems. The research involves two variants of the ensemble model: the normal ELM ensemble with majority voting (ELE), and the random subspace method (RS-ELM). To simulate the small sample cases, only 30% of the total data will be used as the training data. Experiment results show that the RS-ELM model can outperform a multi-layer perceptron (MLP) model under the assumptions of a Friedman test. Furthermore, the ELE model has similar performance as an MLP model under the same assumptions.

Extreme Learning Machine for Accurate Indoor Localization Using RSSI Fingerprints in Multifloor Environments

A new extreme learning machine (ELM) localization technique that uses received signal strength indicator fingerprints only is proposed for multifloor environments. This structured scheme forms multiple individual ELMs for the floors as well as for the geographically formed data clusters of each floor. Multifloor environments often have huge amount of training and online measurement data. To maximize efficiency, we develop a data preprocessing algorithm, aiming to: 1) efficiently extract out only the essential information from the vast amount of data sets and reduce the data dimension and 2) transform the floor-level data sets and positioning data sets of each floor into a proper structure that is suitable for the proposed ensemble ELM technique. The proposed solution is unique in that its offline phase exploits multiple individual ELMs for all floors to generate a set of floor-level classification functions with the preprocessed training data sets, and for each floor, it exploits multiple ELMs for the data clusters to generate a set of position regression functions. The online phase executes a coarse localization step to estimate the floor by using the floor-level classification functions and a refined step to estimate the position on the floor by using the position regression functions. The proposed algorithm and several existing algorithms are implemented to perform localization using the same measured datasets in a multistory building. For both floor estimation and localization on the floor, it outperforms existing schemes. For most cases, the performance gap is substantial.

Rapid determination of hemoglobin concentration by a novel ensemble extreme learning machine method combined with near-infrared spectroscopy

A novel ensemble extreme learning machine (ELM) approach that combines Monte Carlo (MC) sampling and least absolute shrinkage and selection operator (LASSO), named as MC-LASSO-ELM, is proposed to determine hemoglobin concentration of blood. It employs MC sampling to randomly select samples from the training set and LASSO further to choose variables from selected samples to establish plenty of ELM sub-models. The final prediction is obtained by combining the predictions of these sub-models. Combined with near-infrared spectroscopy, MC-LASSO-ELM is used to determine the hemoglobin concentration of blood. Compared with ELM, MC-ELM and LASSO-ELM, MC-LASSO-ELM can obtain the best stability and highest accuracy.

PolSAR Data-Based Land Cover Classification Using Dual-Channel Watershed Region-Merging Segmentation and Bagging-ELM

Land cover classification based on spaceborne synthetic aperture radar (SAR) imagery is an emerging technology with many applications. Machine learning and related ensemble models have attracted much attention to this technology. This letter presents a new object-oriented land cover classification based on Polarimetric SAR data that integrate the dual-channel (DC) Watershed region-merging segmentation algorithm and a Bagging-based ensemble extreme learning machine (Bagging-ELM) image object classifier. The advantage of the DC segmentation algorithm is verified by comparing it with the well-known simple linear iterative clustering (SLIC) superpixel algorithm. Following the DC segmentation, the Bagging-ELM is implemented to perform the classification of image objects. Besides, two other classification methods (single ELM and ensembled-tree algorithms) are used for performance comparison. The quantitative analysis with respect to ground-truth information available for the test sites shows that the proposed approach can achieve noticeable improvements in classification accuracy and outperforms the other two algorithms, showing the potential for spaceborne-SAR-based land cover classification task in real applications.

Suspended sediment load prediction in consecutive stations of river based on ensemble pre-post-processing kernel based approaches

Abstract Sediment transportation and accurate estimation of its rate is a significant issue for river engineers and researchers. In this study, the capability of kernel based approaches including Kernel Extreme Learning Machine (KELM) and Gaussian Process Regression (GPR) was assessed for predicting the river daily Suspended Sediment Discharge (SSD). For this aim, the Mississippi River, with three consecutive hydrometric stations, was selected as the case study. Based on the sediment and flow characteristics during the period of 2005–2008, several models were developed and tested under two scenarios (i.e. modeling based on each station's own data or the previous stations' data). Two post-processing techniques, namely Wavelet Transform (WT) and Ensemble Empirical Mode Decomposition (EEMD), were used for enhancing the SSD modeling capability. Also, data post-proceeding was done using Simple Linear Averaging (SLAM) and Nonlinear Kernel Extreme Learning Machine Ensemble (NKELME) methods. Obtained results indicated that the integrated models resulted in more accurate outcomes. Data processing enhanced the models' capability up to 35%. It was found that SSD modeling based on the station's own data led to better results; however, using the integrated approaches, the previous station's data could be applied successfully for the SSD modeling when a station's own data were not available.

Using optimal choice of parameters for meta-extreme learning machine method in wind energy application

PurposeThis paper aims to deal with the optimal choice of a novel extreme learning machine (ELM) architecture based on an ensemble of classic ELM called Meta-ELM structural parameters by using a forecasting process.Design/methodology/approachThe modelling performance of the Meta-ELM architecture varies depending on the network parameters it contains. The choice of Meta-ELM parameters is important for the accuracy of the models. For this reason, the optimal choice of Meta-ELM parameters is investigated on the problem of wind speed forecasting in this paper. The hourly wind-speed data obtained from Bilecik and Bozcaada stations in Turkey are used. The different number of ELM groups (M) and nodes (Nh) are analysed for determining the best modelling performance of Meta-ELM. Also, the optimal Meta-ELM architecture forecasting results are compared with four different learning algorithms and a hybrid meta-heuristic approach. Finally, the linear model based on correlation between the parameters was given as three dimensions (3D) and calculated.FindingsIt is observed that the analysis has better performance for parameters of Meta-ELM, M = 15 − 20 and Nh = 5 − 10. Also considering the performance metric, the Meta-ELM model provides the best results in all regions and the Levenberg–Marquardt algorithm -feed forward neural network and adaptive neuro fuzzy inference system -particle swarm optimization show competitive results for forecasting process. In addition, the Meta-ELM provides much better results in terms of elapsed time.Originality/valueThe original contribution of the study is to investigate of determination Meta-ELM parameters based on forecasting process.

Active learning for road lane landmark inventory with V-ELM in highly uncontrolled image capture conditions

Road landmark inventory is becoming an important data product for the maintenance of transport infrastructures. Several commercial sensors are available which include synchronized optical cameras that allowto build 360° panoramic images of the surroundings of the vehicle used for road inspection. This paper is devoted to the analysis of such panorama images,specifically the area that contains themost relevant information. Road lane landmark detection is posed as a two class classification problem that may be solved bymachine learningapproaches, such as Random Forest (RF) and ensembles of Extreme Learning Machines (V-ELM). Besides model parameter selection, a central problem is the construction of a labeled training and validation datasetto cope with the highly uncontrolled conditions of image capture. Besides, human labor cost makes image data labeling a very expensive process. This paper proposes an open ended Active Learning (AL) approach involving a human oraclein the loop who provides the data labeling and can trigger the AL process when detection quality is degraded by the change in imaging conditions. The paper reports encouraging results over a collection of sample images selected from an industrial road landmark inventory operation. As an additional contribution, this paper assesses the ability of AL to overcomesome of the issues raised by highly class imbalanced datasets.

Subpixel Temperature Estimation by Information Transfer With Adaptive Ensemble Extreme Learning Machine (IT-AEELM)

The retrieval of land surface temperature (LST) using thermal infrared (TIR) data is important in many applications. However, TIR data usually suffer from low spatial resolution. We introduce a novel subpixel LST estimation model using the information-transfer-based adaptive ensemble extreme learning machine (IT-AEELM). The proposed method constructs a reliable relationship between subpixel LST and the input high-resolution visible and near-infrared (VNIR) data, short-wave infrared (SWIR) data, and low-resolution TIR data. Based on a detailed analysis of different ground objects, we divide the input data into multiple subsets. Instead of using consistent land surface parameters (LSPs), we utilize different LSPs to characterize the land surface properties in each subset. The VNIR-SWIR-LSPs data and the low-resolution LST are used to train a novel IT-AEELM network, where a feedback ensemble learning scheme is introduced to effectively remove inaccurate estimates. The main difference of the model against existing methods is that it builds a robust architecture at different spatial scales, which provides benefits including lower demand for training data, more rapid and accurate acquisition of subpixel LST, and better adaption to heterogeneous land surface. Numerical experiments demonstrate that the proposed method significantly improves the accuracy of subpixel LST compared with the state-of-the-art algorithms.

FPGA‐based favourite skin colour restoration using improved histogram equalization with variable enhancement degree and ensemble extreme learning machine

This paper presents skin color enhancement based on favorite skin color to agree with user-defined favorite skin color using improved histogram equalization with variable enhancement degree (IHEwVED) and machine learning methods. The skin color to be adjusted in the input image is shifted to favorite skin color by using novel control parameters of the proposed IHEwVED method. Three different novel display device-dependent color image processing methods are introduced based on hsv and yiq color space to obtain the desired enhanced output images. A reduced convolutional neural network and the novel ensemble extreme learning machine (EELM) architectures are developed and implemented in a field-programmable gate array to test, synthesize, and validate the recognition capability of the user-defined favorite skin color. The less computational complex proposed IHEwVED-EELM method recognizes 45 to 50 favorite skin color per second of test images by consuming 0.035 second training time with training root mean square error (RMSE) of 0.0048 and testing RMSE of 0.01208. Finally, a stand-alone favorite skin color restoration system is developed using the high-speed video processor NI-PXI-1031 based on the IHEwVED-EELM method in the Python-OpenCV environment. The laboratory experimental performances ascertain the real-time ability of the proposed favorite skin color restoration method.

Extreme ensemble of extreme learning machines

AbstractExtreme learning machine (ELM) has attracted attentions in pattern classification problems due to its preferences in low computations and high generalization. To overcome its drawbacks, caused by the randomness of input weights and biases, the ensemble of ELMs was proposed. The diversity of ELMs forming the ensemble was studied broadly in the literature, via using different activation functions and/or different number of hidden neurons. However, less attention was paid to aggregation mechanism in ensemble of ELMs. To speed up this aggregation process, we propose an ensemble framework for ELMs, called extreme ensemble of ELMs (EEoELMs) because of its extreme speed in ensemble process. In this framework, the input weights of each ELM are randomly pre‐assigned as usual. The ELMs make use of the same/distinct activation functions to increase the diversity of classifiers and so the generalization of ensemble. The output weights of each ELM are set using Moore–Penrose inverse method. However, the aggregation mechanism in EEoELM is novel. Instead of using majority/weighted voting on the prediction results of ELMs, their output neurons are combined in a new decision/ensemble layer. The output of this layer determines the ensemble output. As the weights of this decision layer are also computed using Moore–Penrose inverse method, the ensemble is extremely fast. Experimental results on synthetic and real‐world datasets indicate the acceptable classification performance of EEoELM in much less computational efforts.

Extreme learning machine ensemble model for time series forecasting boosted by PSO: Application to an electric consumption problem

Ensemble Model is a tool that has attracted attention due to its capability to improve the outcome performance of emerging techniques to solve the modelling and classifying problem. However, the feasibility of applying intelligent algorithms to build the Ensemble Model presents a challenge of its own. In this work, an Extreme Learning Machine ensemble is applied to the Time Series modelling problem. We develop a thorough study of the models that will be candidates to compose the ensemble, obtaining statistical results of optimal topologies to solve each Time Series problem. The proposed method for the ensemble is the weighted averaging method, where the parameters (weights) are tuned with the Particle Swarm Optimization algorithm. Lastly, the ensemble is tested first in the well known Santa Fe Time Series Competition benchmark. Given the obtained satisfactory results, the ensemble is secondly tested in a real problem of Spain’s electric consumption forecasting. It is demonstrated that the PSO is a suitable algorithm to optimize Extreme Learning Machine ensemble and that the proposed strategy can obtain good results in both Time Series problems.

Regularized ELM bagging model for Tropical Cyclone Tracks prediction in South China Sea

This paper aims to improve the prediction accuracy of Tropical Cyclone Tracks (TCTs) over the South China Sea (SCS) with 24 h lead time. The model proposed in this paper is a regularized extreme learning machine (ELM) ensemble using bagging. The method which turns the original problem into quadratic programming (QP) problem is proposed in this paper to solve lasso and elastic net problem in ELM. The forecast error of TCTs data set is the distance between real position and forecast position. Compared with the stepwise regression method widely used in TCTs, 8.26 km accuracy improvement is obtained by our model based on the dataset with 70/1680 testing/training records. By contrast, the improvement using this model is 16.49 km based on a smaller dataset with 30/720 testing/training records. Results show that the regularized ELM bagging has a general better generalization capacity on TCTs data set.

Ensemble of kernel extreme learning machine based random forest classifiers for automatic heartbeat classification

Automatic heartbeat classification technology based on the ECG plays an important role in assisting doctors with arrhythmia diagnosis. While many heartbeat classification studies can achieve good performance under the intra-patient paradigm, they still cannot offer acceptable classification results under the inter-patient paradigm. Additionally, the available ECG datasets are highly class imbalanced since normal heartbeats appear much more frequently than abnormal heartbeats, resulting in most methods having low sensitives and positive predictive values on minority class ectopic heartbeats. To solve the above problems, this study proposes an automatic ECG heartbeat classification method based on ensemble learning and multi-kernel learning. First, we use a linear combination of the radial basis function kernel and the polynomial kernel to produce a mixed-kernel-based extreme learning machine (MKELM). Then, a MKELM-based random forest binary classifier (MKELM-RF) is constructed. Finally, an ensemble multiclass classifier MKELM-RF-OVO is proposed based on one-vs.-one (OVO) reduction and MKELM-RF. We evaluated the proposed method on the public MIT-BIH-AR benchmarks database, under the inter-patient paradigm, classifying four types of heartbeats, namely, normal beats (N), supraventricular ectopic beats (S), ventricular ectopic beats (V) and the fusion of ventricular and normal (F). The obtained overall accuracy and the average positive predictive value are 98.1% and 93.9%, respectively, which are higher than the current studies by approximately 4% and 6%, respectively. The sensitivities for classes S and V are 1 and 94.4%, respectively, which outperforms most methods. The evaluation results show that our proposed method achieves a superior classification performance compared to the state-of-the-art methods.

Multi-objective ensembles of echo state networks and extreme learning machines for streamflow series forecasting

Streamflow series forecasting composes a fundamental step in planning electric energy production for hydroelectric plants. In Brazil, such plants produce almost 70% of the total energy. Therefore, it is of great importance to improve the quality of streamflow series forecasting by investigating state-of-the-art time series forecasting algorithms. To this end, this work proposes the development of ensembles of unorganized machines, namely Extreme Learning Machines (ELMs) and Echo State Networks (ESNs). Two primary contributions are proposed: (1) a new training logic for ESNs that enables the application of bootstrap aggregation (bagging); and (2) the employment of multi-objective optimization to select and adjust the weights of the ensemble’s base models, taking into account the trade-off between bias and variance. Experiments are conducted on streamflow series data from five real-world Brazilian hydroelectric plants, namely those in Sobradinho, Serra da Mesa, Jiraú, Furnas and Água Vermelha. The statistical results for four different prediction horizons (1, 3, 6, and 12 months ahead) indicate that the ensembles of unorganized machines achieve better results than autoregressive (AR) models in terms of the Nash–Sutcliffe model efficiency coefficient (NSE), root mean squared error (RMSE), coefficient of determination (R2), and RMSE-observations standard deviation ratio (RSR). In such results, the ensembles with ESNs and the multi-objective optimization design procedure achieve the best scores.