Original Extreme Learning Machine Research Articles

Short-Term Electrical Load Forecasting Using an Enhanced Extreme Learning Machine Based on the Improved Dwarf Mongoose Optimization Algorithm

Accurate short-term electrical load forecasting is crucial for the stable operation of power systems. Given the nonlinear, periodic, and rapidly changing characteristics of short-term power load forecasts, this paper introduces a novel forecasting method employing an Extreme Learning Machine (ELM) enhanced by an improved Dwarf Mongoose Optimization Algorithm (Local escape Dwarf Mongoose Optimization Algorithm, LDMOA). This method addresses the significant prediction errors of conventional ELM models and enhances prediction accuracy. The enhancements to the Dwarf Mongoose Optimization Algorithm include three key modifications: initially, a dynamic backward learning strategy is integrated at the early stages of the algorithm to augment its global search capabilities. Subsequently, a cosine algorithm is employed to locate new food sources, thereby expanding the search scope and avoiding local optima. Lastly, a “madness factor” is added when identifying new sleeping burrows to further widen the search area and effectively circumvent local optima. Comparative analyses using benchmark functions demonstrate the improved algorithm’s superior convergence and stability. In this study, the LDMOA algorithm optimizes the weights and thresholds of the ELM to establish the LDMOA-ELM prediction model. Experimental forecasts utilizing data from China’s 2016 “The Electrician Mathematical Contest in Modeling” demonstrate that the LDMOA-ELM model significantly outperforms the original ELM model in terms of prediction error and accuracy.

Stacking integrated learning model via ELM and GRU with mixture correntropy loss for robust state of health estimation of lithium-ion batteries

Accurate estimation of the state of health (SOH) is crucial for the safe and stable operation of lithium-ion batteries. However, since the labeled values may contain non-Gaussian noise which may lead to a decrease in the effectiveness of traditional data-driven based estimation methods. Therefore, a novel robust stacking integrated learning model (ILM) is proposed to enable accurate SOH estimation under non-Gaussian noises (or outliers) conditions. The mixture correntropy loss (MCL) is used in original extreme learning machine (ELM) and the gated recurrent unit (GRU) frameworks to develop novel robust learning models, i.e MCL-ELM and MCL-GRU, and they are used as the sub-models of the proposed ILM, which takes into account the generalization performance of the model and the overall trend of SOH over the time series. In addition, the entropy weighting method that can well reflect the differences between the sub-models is utilized to reduce the complexity of the stacked ILM. Furthermore, the local tangent space alignment is used to capture the local relationships between different loops to enhance the effectiveness of the extracted health features. Several numerical experiments are performed under different cases to validate the estimation effect of the proposed model by using two publicly available datasets, and the experimental results demonstrate that the maximum RMSE, MAE, and MAPE values of the proposed model are 1.391%, 0.932%, and 1.480%, respectively, under non-Gaussian noises conditions.

Twin extreme learning machine based on heteroskedastic Gaussian noise model and its application in short-term wind-speed forecasting

Extreme learning machine (ELM) has received increasingly more attention because of its high efficiency and ease of implementation. However, the existing ELM algorithms generally suffer from the drawbacks of noise sensitivity and poor robustness. Therefore, we combine the advantages of twin hyperplanes with the fast speed of ELM, and then introduce the characteristics of heteroscedastic Gaussian noise. In this paper, a new regressor is proposed, which is called twin extreme learning machine based on heteroskedastic Gaussian noise (TELM-HGN). In addition, the augmented Lagrange multiplier method is introduced to optimize and solve the presented model. Finally, a significant number of experiments were conducted on different data-sets including real wind-speed data, Boston housing price dataset and stock dataset. Experimental results show that the proposed algorithms not only inherits most of the merits of the original ELM, but also has more stable and reliable generalization performance and more accurate prediction results. These applications demonstrate the correctness and effectiveness of the proposed model.

FORECASTING OF CURRENCY CIRCULATION IN INDONESIA USING HYBRID EXTREME LEARNING MACHINE

Forecasting currency circulation, including inflow and outflow, is one of Bank Indonesia's strategies to maintain the Rupiah value's stability. The characteristic of inflow and outflow data is that they have seasonal variations. This study proposes a hybrid model by combining decomposition techniques and Extreme Learning Machine to overcome data that has seasonal variations. The forecasting results of the proposed model are compared with the original Extreme Learning Machine. The comparison results show that the forecasting results with the hybrid model have the smallest errors. Thus, the hybrid model can predict data with seasonal variations better than the original Extreme Learning Machine.

Fully interpretable neural network for locating resonance frequency bands for machine condition monitoring

In recent years, various neural networks have been developed to process vibration signals for machine condition monitoring. Nevertheless, the physical interpretation of neural networks is still on-going and not fully explored. This paper aims to design a fully interpretable neural network for machine condition monitoring from the aspects of signal processing and physical feature extraction. The main idea of the fully interpretable neural network is to extend the uninterpretable structure of extreme learning machine (ELM) to an interpretable structure for machine condition monitoring. From the aspect of signal processing, wavelet transform, square envelope and Fourier transform are incorporated into the input layer of the original ELM to extract repetitive transients, localize informative frequency bands for an enhancement of a signal-to-noise ratio, and realize square envelope spectra for exhibiting cyclo-stationarity of repetitive transients. Hence, the first to four layers of the proposed network are physically interpretable. From the aspect of physical feature extraction, popular sparsity measures are innovatively incorporated into all random nodes in the single-hidden layer of the original ELM to interpret the use of all hidden nodes in the fifth layer of the proposed network to characterize cyclo-stationarity of repetitive transients. The significance of this paper is to show that signal processing algorithms and physical feature extraction can be reformulated as the architecture of an interpretable neural network to automatically localize informative frequency bands for machine condition monitoring. This paper attempts to inspire researchers in the field of signal processing and machine learning to think about the design of more advanced interpretable neural networks for machine condition monitoring.

Multilayer Extreme Learning Machine as Equalizer in OFDM-based Radio-over-fiber Systems

Mobile/wireless networks aim to support diverse services with numerous and sophisticated requirements, such as energy efficiency, spectral efficiency, negligible latency, robustness against time and frequency selective channels, low hardware complexity, among others. From the central station to the base stations, radio-over-fiber orthogonal frequency division multiplexing (RoF-OFDM) schemes with direct-detection are then implemented. Unfortunately, laser phase noise, chromatic fiber dispersion, and carrier frequency offset impair the orthogonality of the subcarriers; hence, deteriorating the performance of the RoF-OFDM system. In order to take all the processing tasks to the cognitive level (the last goal in the telecommunication industry), various extreme learning machines (ELMs), composed by only a single hidden layer, have been recently adopted as equalizers. The reason behind this trend comes from the lower computational complexity, higher detection accuracy, and minimum human intervention of the ELM algorithms. In this article, we introduce a multilayer ELM-based receiver for RoF schemes transmitting phase-correlated OFDM signals affected by phase and frequency errors. Results report that by appropriately setting the hyper-parameters of the multilayer ELMs, the ELM with 3 hidden layers outperforms most of the ELMs reported in the literature (the ELM with 2 hidden layers, original ELM, regularized ELM, and 2 fully-independent ELMs defined in the real domain), as well as the benchmark pilot-assisted equalizer in terms of bit error rate. Nevertheless, this benefit comes with excessive computational cost. Finally, we show that the fully-complex ELM is still the best equalizer taking into account several key metrics.

An Expert Artificial Intelligence Model for Discriminating Microseismic Events and Mine Blasts

To reduce the workload and misjudgment of manually discriminating microseismic events and blasts in mines, an artificial intelligence model called PSO-ELM, based on the extreme learning machine (ELM) optimized by the particle swarm optimization (PSO) algorithm, was applied in this study. Firstly, based on the difference between microseismic events and mine blasts and previous research results, 22 seismic parameters were selected as the discrimination feature parameters and their correlation was analyzed. Secondly, 1600 events were randomly selected from the database of the microseismic monitoring system in Fankou Lead-Zinc Mine to form a sample dataset. Then, the optimal discrimination model was established by investigating the model parameters. Finally, the performance of the model was tested using the sample dataset, and it was compared with the performance of the original ELM model and other commonly used intelligent discrimination models. The results indicate that the discrimination performance of PSO-ELM is the best. The values of the six evaluation indicators are close to the optimal value, which shows that PSO-ELM has great potential for discriminating microseismic events and blasts. The research results obtained can provide a new method for discriminating microseismic events and blasts, and it is of great significance to ensure the safe and smooth operation of mines.

Negative Correlation Hidden Layer for the Extreme Learning Machine

Extreme Learning Machine (ELM) algorithms have achieved unprecedented performance in supervised machine learning tasks. However, the preconfiguration of the nodes in the hidden layer in ELM models through randomness does not always lead to a suitable transformation of the original features. Consequently, the performance of these models relies on broad exploration of these feature mappings, generally using a large number of nodes in the hidden layer. In this paper, a novel ELM architecture is presented, called Negative Correlation Hidden Layer ELM (NCHL-ELM), based on the Negative Correlation Learning (NCL) framework. This model incorporates a parameter into each node in the original ELM hidden layer, and these parameters are optimized by reducing the error in the training set and promoting the diversity among them in order to improve the generalization results. Mathematically, the ELM minimization problem is perturbed by a penalty term, which represents a measure of diversity among the parameters. A variety of regression and classification benchmark datasets have been selected in order to compare NCHL-ELM with other state-of-the-art ELM models. Statistical tests indicate the superiority of our method in both regression and classification problems.

A soft measurement approach of wastewater treatment process by lion swarm optimizer-based extreme learning machine

Some key variables in the wastewater treatment process, such as the biochemical oxygen demand (BOD5) and the chemical oxygen demand (COD), are difficult to perform timely and accurate measurement by hardware means. In view of this issue, we introduce a novel soft measurement approach. In the proposed approach, the sparse principal component analysis (SPCA) is used for dimensionality reduction of datasets, and construction of the soft measurement model is done by an improved extreme learning machine (ELM) algorithm, the lion swarm optimizer-based extreme learning machine (LSO-ELM). In the LSO-ELM, the connection weight matrix from the input layer to the hidden layer and the bias vector of the hidden layer are optimized by the LSO. The optimization of these two parameters has greatly improved the predictive ability of original ELM. We illustrate the LSO-ELM approach on Benchmark Simulation Model 1 (BSM1) and the results show that it has excellent performance on the prediction of BOD5 and COD. The proposed approach will greatly improve the quality of soft measurement in wastewater treatment process.

Extreme Learning Machine Based on Calculating the Output Weight of Partial Robust M-regression

In order to improve the nonlinear mapping capability and learning performance of extreme learning machine (ELM), a new learning algorithm called extreme learning machine based on calculating the output weight of partial robust M-regression is proposed. This algorithm introduces the partial robust M-regression into the extreme learning machine algorithm. Firstly, the hidden layer output matrix H is calculated by extreme learning machine. Secondly, the matrix H and vector Y are weighted by weighted strategy. Then, PRM algorithm is used to establish the regression model between the weighted matrix Hw and vector Yw, and calculate its regression coefficient, namely output weight of the ELM Algorithm. The proposed method predicts the Mackay’s robot arm regression and sediment concentration of the Yellow River Basin to verify the effectiveness of the method. The simulation results show that the proposed PRMELM algorithm is superior to the original extreme learning machine algorithm in prediction accuracy and generalization performance.

Constrained voting extreme learning machine and its application

Extreme learning machine (ELM) has been proved to be an effective pattern classification and regression learning mechanism by researchers. However, its good performance is based on a large number of hidden layer nodes. With the increase of the nodes in the hidden layers, the computation cost is greatly increased. In this paper, we propose a novel algorithm, named constrained voting extreme learning machine (CV-ELM). Compared with the traditional ELM, the CV-ELM determines the input weight and bias based on the differences of between-class samples. At the same time, to improve the accuracy of the proposed method, the voting selection is introduced. The proposed method is evaluated on public benchmark datasets. The experimental results show that the proposed algorithm is superior to the original ELM algorithm. Further, we apply the CV-ELM to the classification of superheat degree (SD) state in the aluminum electrolysis industry, and the recognition accuracy rate reaches 87.4%, and the experimental results demonstrate that the proposed method is more robust than the existing state-of-the-art identification methods.

Robust multi-layer extreme learning machine using bias-variance tradeoff

As a new neural network model, extreme learning machine (ELM) has a good learning rate and generalization ability. However, ELM with a single hidden layer structure often fails to achieve good results when faced with large-scale multi-featured problems. To resolve this problem, we propose a multi-layer framework for the ELM learning algorithm to improve the model’s generalization ability. Moreover, noises or abnormal points often exist in practical applications, and they result in the inability to obtain clean training data. The generalization ability of the original ELM decreases under such circumstances. To address this issue, we add model bias and variance to the loss function so that the model gains the ability to minimize model bias and model variance, thus reducing the influence of noise signals. A new robust multi-layer algorithm called ML-RELM is proposed to enhance outlier robustness in complex datasets. Simulation results show that the method has high generalization ability and strong robustness to noise.

A Hybrid Method Based on Extreme Learning Machine and Self Organizing Map for Pattern Classification.

Extreme learning machine is a fast learning algorithm for single hidden layer feedforward neural network. However, an improper number of hidden neurons and random parameters have a great effect on the performance of the extreme learning machine. In order to select a suitable number of hidden neurons, this paper proposes a novel hybrid learning based on a two-step process. First, the parameters of hidden layer are adjusted by a self-organized learning algorithm. Next, the weights matrix of the output layer is determined using the Moore–Penrose inverse method. Nine classification datasets are considered to demonstrate the efficiency of the proposed approach compared with original extreme learning machine, Tikhonov regularization optimally pruned extreme learning machine, and backpropagation algorithms. The results show that the proposed method is fast and produces better accuracy and generalization performances.

Smoothing L0 Regularization for Extreme Learning Machine

Extreme learning machine (ELM) has been put forward for single hidden layer feedforward networks. Because of its powerful modeling ability and it needs less human intervention, the ELM algorithm has been used widely in both regression and classification experiments. However, in order to achieve required accuracy, it needs many more hidden nodes than is typically needed by the conventional neural networks. This paper considers a new efficient learning algorithm for ELM with smoothing L0 regularization. A novel algorithm updates weights in the direction along which the overall square error is reduced the most and then this new algorithm can sparse network structure very efficiently. The numerical experiments show that the ELM algorithm with smoothing L0 regularization has less hidden nodes but better generalization performance than original ELM and ELM with L1 regularization algorithms.

Hybrid particle swarm optimization with extreme learning machine for daily reference evapotranspiration prediction from limited climatic data

Accurate prediction of reference evapotranspiration (ETo) is pivotal to the determination of crop water requirement and irrigation scheduling in agriculture as well as water resources management in hydrology. In the present study, the particle swarm optimization (PSO) algorithm was utilized to optimally determine the parameters of the extreme learning machine (ELM) model, and a novel hybrid PSO-ELM model was thus proposed for estimating daily ETo in the arid region of Northwest China with limited input data. The PSO-ELM model was compared with the original ELM, artificial neural networks (ANN) and random forests (RF) models as along with six empirical models (including radiation-, temperature- and mass transfer-based empirical models). Three input combinations were utilized to develop the data-driven models, which corresponded to the radiation-, temperature- and mass transfer-based models, respectively. The results indicated that machine learning models provided more accurate ETo estimates, compared with the corresponding empirical models with the same inputs. The hybrid PSO-ELM model exhibited better performance than the other models for daily ETo estimation as indicated by the statistical results. Although radiation-based machine learning models outperformed temperature- and mass transfer-based machine learning models, the temperature-based PSO-ELM model obtained reasonable results when only air temperature data were available, which was considered as a promising model for forecasting future ETo with temperature data. Overall, the PSO-ELM model was superior to the other machine learning and empirical models, which was thus recommended to predict daily ETo with limited inputs in the arid region of Northwest China.

Chaotic emperor penguin optimised extreme learning machine for microarray cancer classification.

Microarray technology plays a significant role in cancer classification, where a large number of genes and samples are simultaneously analysed. For the efficient analysis of the microarray data, there is a great demand for the development of intelligent techniques. In this article, the authors propose a novel hybrid technique employing Fisher criterion, ReliefF, and extreme learning machine (ELM) based on the principle of chaotic emperor penguin optimisation algorithm (CEPO). EPO is a recently developed metaheuristic method. In the proposed method, initially, Fisher score and ReliefF are independently used as filters for relevant gene selection. Further, a novel population-based metaheuristic, namely, CEPO was proposed to pre-train the ELM by selecting the optimal input weights and hidden biases. The authors have successfully conducted experiments on seven well-known data sets. To evaluate the effectiveness, the proposed method is compared with original EPO, genetic algorithm, and particle swarm optimisation-based ELM along with other state-of-the-art techniques. The experimental results show that the proposed framework achieves better accuracy as compared to the state-of-the-art schemes. The efficacy of the proposed method is demonstrated in terms of accuracy, sensitivity, specificity, and F-measure.

A Hybrid Model for Financial Time Series Forecasting—Integration of EWT, ARIMA With The Improved ABC Optimized ELM

The practical significances and complexities of financial time series analysis induce highly demand more reliable hybrid model that denoised the data efficiently, handled with both linear and nonlinear patterns in the data, to achieve more accurate results. This paper suggests a new forecasting hybrid model for financial time series data combined Empirical Wavelet Transform (EWT) technique with improved Artificial Bee Colony (ABC) algorithm, Extreme Learning Machine (ELM) neural network, and Auto-Regressive Integrated Moving Average (ARIMA) linear analysis algorithm. The EWT is used to decompose and denoise the data to reconstruct the data more suitable for forecast. The improvement of the ABC algorithm is according to the Good Point Sets (GPS) theory and adaptive Elite-based Opposition (EO) strategy (GPS-EO-ABC) to overcome the drawbacks of the original algorithm and enhance the optimization performance. The optimized ELM with GPS-EO-ABC, as well as the ARIMA, are utilized independently to generated different forecasting results and combined by the weight-based procession. We testify the performance of the proposed improved ABC algorithm by ten benchmark functions, simulating the proposed forecasting models by three financial time-series datasets. The results indicate that: (1) The proposed algorithm shows outstanding capacities in parameter optimization. The optimized ELM generated more stable and precise results compared with original ELM, ABC-ELM, single LSTM, and ANN; (2) The proposed hybrid model has not only effectiveness but also efficiencies in denoising data, correcting outliers, coordinating both linear and nonlinear patterns, its performance in financial time series forecasting is more excellent than existing hybrid models.

Inverse-Free Incremental Learning Algorithms With Reduced Complexity for Regularized Extreme Learning Machine

The existing inverse-free incremental learning algorithm for the regularized extreme learning machine (ELM) was based on an inverse-free algorithm to update the regularized pseudo-inverse, which was deduced from an inverse-free recursive algorithm to update the inverse of a Hermitian matrix. Before that recursive algorithm was applied in the existing inverse-free ELM, its improved version had been utilized in previous literatures. Then from the improved recursive algorithm to update the inverse, we deduce a more efficient inverse-free algorithm to update the regularized pseudo-inverse, from which we propose the inverse-free incremental ELM algorithm based on regularized pseudo-inverse. Usually the above-mentioned inverse is smaller than the pseudo-inverse, while in the processor units with limited precision, the recursive algorithm to update the inverse may introduce numerical instabilities. Then to further reduce the computational complexity, we also propose the inverse-free incremental ELM algorithm based on the ${\mathrm {LDL}}^{T}$ factors of the inverse, where the ${\mathrm {LDL}}^{T}$ factors are updated iteratively by the inverse ${\mathrm {LDL}}^{T}$ factorization. With respect to the existing inverse-free ELM, the proposed ELM based on regularized pseudo-inverse and that based on ${\mathrm {LDL}}^{T}$ factors are expected to require only $\frac {3}{8+M}$ and $\frac {1}{8+M}$ of complexities, respectively, where $M$ is the output node number. The numerical experiments show that both the proposed ELM algorithms significantly accelerate the existing inverse-free ELM, and the speedup in training time is not less than 1.41. On the Modified National Institute of Standards and Technology (MNIST) Dataset, usually the proposed algorithm based on ${\mathrm {LDL}}^{T}$ factors is much faster than that based on regularized pseudo-inverse. On the other hand, in the numerical experiments, the original ELM, the existing inverse-free ELM and the proposed two ELM algorithms achieve the same performance in regression and classification, and result in the same solutions, which include the output weights and the output sequence for the same input sequence.

Two noise tolerant incremental learning algorithms for single layer feed-forward neural networks

This paper focuses on noise resistant incremental learning algorithms for single layer feed-forward neural networks (SLFNNs). In a physical implementation of a well trained neural network, faults or noise are unavoidable. As biological neural networks have ability to tolerate noise, we would like to have a trained neural network that has certain ability to tolerate noise too. This paper first develops a noise tolerant objective function that can handle multiplicative weight noise. We assume that multiplicative weight noise exist in the weights between the input layer and the hidden layer, and in the weights between the hidden layer and the output layer. Based on the developed objective function, we propose two noise tolerant incremental extreme learning machine algorithms, namely weight deviation incremental extreme learning machine (WDT-IELM) and weight deviation convex incremental extreme learning machine (WDTC-IELM). Compared to the original extreme learning machine algorithms, the two proposed algorithms have much better ability to tolerate the multiplicative weight noise. Several simulations are carried out to demonstrate the superiority of the two proposed algorithms.

Adaptive neural tracking control for automotive engine idle speed regulation using extreme learning machine

The automotive engine idle speed control problem is a compromise among low engine speed for fuel saving, minimum emissions and disturbance rejection ability to prevent engine stall. However, idle speed regulation is very challenging due to the presence of high nonlinearity and aging-caused uncertainties in the engine dynamics. Therefore, the engine idle speed system is a typical uncertain nonlinear system. To address the problems of inherent nonlinearity and uncertainties in idle speed regulation, an extreme learning machine (ELM)-based adaptive neural control algorithm is proposed for tracking the target idle speed adaptively. The purpose of ELM is to rapidly deal with the uncertain nonlinear engine system. Since the original ELM is not designed for adaptive control, a new adaptation law is designed to update the weights of ELM in the sense of Lyapunov stability. Experiment is conducted to validate the performance of the proposed control method. Experimental result indicates that the ELM-based adaptive neural control outperforms the classical proportional–integral–derivative (PID), fuzzy-PID and back-propagation-neural-network-based controllers in terms of tracking performance under the variation of engine load.