Extreme Learning Machine Neural Network Research Articles

Factor analysis and forecasting of CO2 emissions in Hebei, using extreme learning machine based on particle swarm optimization

In the prevailing low-carbon economy, China is under enormous pressure to control CO2 emissions, therefore, of great significance is the study to analyze what influential factors mainly contribute to emissions, so as to forecast emissions accurately and harness the growth from the source. In this paper, basing on 22 influencing factors identified by bivariate correlation analysis, factor analysis is then adopted to extract the latent factors which essentially affect emissions and 8 special factors transformed by scoring coefficients are acquired. Extreme learning machine (ELM) whose input weights and bias threshold were optimized by particle swarm optimization (PSO), hereafter referred as PSO-ELM, is established to predict CO2 emissions and testify the availability of the factor analysis. Case studies reveal that the factor analysis which generates 8 factors as input can highly improve prediction accuracy. And the simulation results demonstrate that the built model PSO-ELM outperforms the compared ELM and back propagation neural network in forecasting CO2 emissions. Eventually, the analysis made in this study can provide valuable policy implications for Hebei’s CO2 emissions reduction and strategic low-carbon development.

Joint multiple fully connected convolutional neural network with extreme learning machine for hepatocellular carcinoma nuclei grading

Accurate cell grading of cancerous tissue pathological image is of great importance in medical diagnosis and treatment. This paper proposes a joint multiple fully connected convolutional neural network with extreme learning machine (MFC-CNN-ELM) architecture for hepatocellular carcinoma (HCC) nuclei grading. First, in preprocessing stage, each grayscale image patch with the fixed size is obtained using center-proliferation segmentation (CPS) method and the corresponding labels are marked under the guidance of three pathologists. Next, a multiple fully connected convolutional neural network (MFC-CNN) is designed to extract the multi-form feature vectors of each input image automatically, which considers multi-scale contextual information of deep layer maps sufficiently. After that, a convolutional neural network extreme learning machine (CNN-ELM) model is proposed to grade HCC nuclei. Finally, a back propagation (BP) algorithm, which contains a new up-sample method, is utilized to train MFC-CNN-ELM architecture. The experiment comparison results demonstrate that our proposed MFC-CNN-ELM has superior performance compared with related works for HCC nuclei grading. Meanwhile, external validation using ICPR 2014 HEp-2 cell dataset shows the good generalization of our MFC-CNN-ELM architecture.

Remote Sensing Image Classification Based on Ensemble Extreme Learning Machine With Stacked Autoencoder

Classification is one of the most popular topics in remote sensing. Consider the problems that the remote sensing data are complicated and few labeled training samples limit the performance and efficiency in the classification of remote sensing image. For these problems, a huge number of methods were proposed in the last two decades. However, most of them do not yield good performance. In this paper, a remote sensing image classification algorithm based on the ensemble of extreme learning machine (ELM) neural network, namely, stacked autoencoder (SAE)-ELM, is proposed. First, due to improve the ensemble classification accuracy, we adopt feature segmentation and SAE in the sample data to create high diversity among the base classifiers. Furthermore, ELM neural network is chosen as a base classifier to improve the learning speed of the algorithm. Finally, to determine the final ensemble-based classifier, Q-statistics is adopted. The experiment compares the proposed algorithm with Bagging, Adaboost, Random Forest et al., which results show that the proposed algorithm not only gets high classification accuracy on low resolution, medium resolution, high resolution and hyperspectral remote sensing images, but also has strong stability and generalization on UCI data.

A Dissolved Oxygen Prediction Method Based on K-Means Clustering and the ELM Neural Network: A Case Study of the Changdang Lake, China

According to mass data from the internet, prediction is an important application of information processing. Predicting the amount of dissolved oxygen (DO) of a water environment is vital and can improve the aquaculture production efficiency. Using water quality and meteorological data for the Changdang Lake (Jiangsu, China) from 2014 to 2016, this study proposes a new strategy for predicting short-term DO based on K-means clustering and extreme learning machine (ELM) neural networks. First, the weights of the environmental factors on the DO are obtained using Pearson correlation analysis, and similar days are defined. Subsequently, according to the defined similarity, the K-means clustering algorithm divides the historical data into several clusters and identifies similar sample sets that have characteristics that are similar to the forecast day. Finally, after the ELM neural network model establishes the training and testing data, the DO is predicted using the similar sample set and the real-time environmental factors of the forecast days as input data. The prediction efficiency of our model was compared to that of others in terms of the mean absolute percentage error and the mean square error. The experimental results showed that our approach had higher forecasting accuracy and faster computation speed, which is beneficial for water management. Keywords: Prediction method, Similar day, K-means clustering, ELM neural network, Dissolved oxygen.

Pure intelligent monitoring system for steam economizer trips

Steam economizer represents one of the main equipment in the power plant. Some steam economizer's behavior lead to failure and shutdown in the entire power plant. This will lead to increase in operating and maintenance cost. By detecting the cause in the early stages maintain normal and safe operational conditions of power plant. However, these methodologies are hard to be achieved due to certain boundaries such as system learning ability and the weakness of the system beyond its domain of expertise. The best solution for these problems, an intelligent modeling system specialized in steam economizer trips have been proposed and coded within MATLAB environment to be as a potential solution to insure a fault detection and diagnosis system (FDD). An integrated plant data preparation framework for 10 trips was studied as framework variables. The most influential operational variables have been trained and validated by adopting Artificial Neural Network (ANN). The Extreme Learning Machine (ELM) neural network methodology has been proposed as a major computational intelligent tool in the system. It is shown that ANN can be implemented for monitoring any process faults in thermal power plants. Better speed of learning algorithms by using the Extreme Learning Machine has been approved as well.

Optimal Control of Complex Systems Based on Improved Dual Heuristic Dynamic Programming Algorithm

When applied to solving the data modeling and optimal control problems of complex systems, the dual heuristic dynamic programming (DHP) technique, which is based on the BP neural network algorithm (BP-DHP), has difficulty in prediction accuracy, slow convergence speed, poor stability, and so forth. In this paper, a dual DHP technique based on Extreme Learning Machine (ELM) algorithm (ELM-DHP) was proposed. Through constructing three kinds of network structures, the paper gives the detailed realization process of the DHP technique in the ELM. The controller designed upon the ELM-DHP algorithm controlled a molecular distillation system with complex features, such as multivariability, strong coupling, and nonlinearity. Finally, the effectiveness of the algorithm is verified by the simulation that compares DHP and HDP algorithms based on ELM and BP neural network. The algorithm can also be applied to solve the data modeling and optimal control problems of similar complex systems.

Application of ELM to predict the coagulant dosing in water treatment plants

Predicting the coagulant dosage is especially crucial to the purification process in water treatment plants, directly affecting the quality of the purified water. Nowadays, several mathematical methods have been adopted for the purification process, but their predictive precision and speed still need to be improved. This study applies a novel neural network called the extreme learning machine (ELM) to predict the coagulant dosage based on certain signification factors of the raw water. Performances are compared between ELM and back-propagation neural networks in this paper. The results show that both neural network algorithms perform well in this application and ELM can realize online prediction due to its short time consumption.

A Hybrid Machine Learning Method for Intrusion Detection

Data security is an important area of concern for every computer system owner. An intrusion detection system is a device or software application that monitors a network or systems for malicious activity or policy violations. Already various techniques of artificial intelligence have been used for intrusion detection. The main challenge in this area is the running speed of the available implementations. In this research work, we present a hybrid approach which is based on the “linear discernment analysis” and the “extreme learning machine” to build a tool for intrusion detection. In the proposed method, the linear discernment analysis is used to reduce the dimensions of data and the extreme learning machine neural network is used for data classification. This idea allowed us to benefit from the advantages of both methods. We implemented the proposed method on a microcomputer with core i5 1.6 GHz processor by using machine learning toolbox. In order to evaluate the performance of the proposed method, we run it on a comprehensive data set concerning intrusion detection. The data set is called KDD, which is a version of the data set DARPA presented by MIT Lincoln Labs. The experimental results were organized in related tables and charts. Analysis of the results show meaningful improvements in intrusion detection. In general, compared to the existing methods, the proposed approach works faster with higher accuracy.

Adaptive control of nonlinear system using online error minimum neural networks

In this paper, a new learning algorithm named OEM-ELM (Online Error Minimized-ELM) is proposed based on ELM (Extreme Learning Machine) neural network algorithm and the spreading of its main structure. The core idea of this OEM-ELM algorithm is: online learning, evaluation of network performance, and increasing of the number of hidden nodes. It combines the advantages of OS-ELM and EM-ELM, which can improve the capability of identification and avoid the redundancy of networks. The adaptive control based on the proposed algorithm OEM-ELM is set up which has stronger adaptive capability to the change of environment. The adaptive control of chemical process Continuous Stirred Tank Reactor (CSTR) is also given for application. The simulation results show that the proposed algorithm with respect to the traditional ELM algorithm can avoid network redundancy and improve the control performance greatly.

A changing forgetting factor RLS for online identification of nonlinear systems based on ELM–Hammerstein model

In this paper, an online identification method is proposed for nonlinear system identification based on extreme learning machine (ELM)–Hammerstein model. The ELM–Hammerstein model comprises an ELM neural network followed by a linear dynamic subsystem. This model is linear in parameters and nonlinear in the input. To speed up the convergence and meanwhile improve identification accuracy, a changing forgetting factor recursive least squares (CFF-RLS) method is proposed as online learning algorithm. The algorithm can identify the parameters of linear dynamic subsystem and the weights of ELM neural network simultaneously. Simulation results verify the effectiveness of the proposed method.

Bio-inspired neural network with application to license plate recognition: hysteretic ELM approach

Purpose This paper aims to present a bio-inspired neural network for improvement of information processing capability of the existing artificial neural networks. Design/methodology/approach In the network, the authors introduce a property often found in biological neural system – hysteresis – as the neuron activation function and a bionic algorithm – extreme learning machine (ELM) – as the learning scheme. The authors give the gradient descent procedure to optimize parameters of the hysteretic function and develop an algorithm to online select ELM parameters, including number of the hidden-layer nodes and hidden-layer parameters. The algorithm combines the idea of the cross validation and random assignment in original ELM. Finally, the authors demonstrate the advantages of the hysteretic ELM neural network by applying it to automatic license plate recognition. Findings Experiments on automatic license plate recognition show that the bio-inspired learning system has better classification accuracy and generalization capability with consideration to efficiency. Originality/value Comparing with the conventional sigmoid function, hysteresis as the activation function enables has two advantages: the neuron’s output not only depends on its input but also on derivative information, which provides the neuron with memory; the hysteretic function can switch between the two segments, thus avoiding the neuron falling into local minima and having a quicker learning rate. The improved ELM algorithm in some extent makes up for declining performance because of original ELM’s complete randomness with the cost of a litter slower than before.

Identification and control of nonlinear system based on Laguerre-ELM Wiener model

In this paper, a new Wiener model is presented for identification and control of single-input single-output (SISO) nonlinear systems. The proposed Wiener model consists of a linear Laguerre filter in cascaded with an extreme learning machine (ELM) neural network (called Laguerre-ELM Wiener model). Laguerre filter can approximate a stable linear system to any degree of accuracy with a small number of Laguerre filters, which provides a parsimony structure and high level accuracy simultaneously. To determine the appropriated number of Laguerre filters in Laguerre-ELM Wiener model, Lipschitz quotient criterion is adapted to determine the order of linear part. A generalized ELM algorithm is proposed to estimate the parameters of Laguerre-ELM Wiener model. Once the unknown nonlinear system is identified using Laguerre-ELM Wiener model, a generalized predictive control (GPC) algorithm is designed for control of nonlinear system. The advantage of the proposed control method is that it transfers a nonlinear control problem to a linear one by inserting the inverse of static nonlinear section. Simulation results demonstrate the effectiveness of the proposed identification and control algorithms.

The Features Extraction of Infants Cries by Using Discrete Wavelet Transform Techniques

This article proposes the study and experiment of infant sound features extraction by using discrete wavelet transform (DWT) techniques. The DWTs using in this research are Haar, Symlet2 and Coiflet1 mother wavelets. In this classification, the Dunstan baby language is the infant sound data. The extracted features from the infant sound by DWT are learned by using the extreme learning machine (ELM) neural network. The results of this learning are compared in term of learning accuracy. From the experimental result, it is found that the average result of the ELM with Haar wavelet features extraction at number node of 30 is better than results of ELM with other wavelets in term of learning accuracy. However, there are insignificant differences in learning accuracy when the number of nodes is increased from 20 to 30 nodes.

An adaptive ensemble of on-line Extreme Learning Machines with variable forgetting factor for dynamic system prediction

A demand for predictive models for on-line estimation of variables is increasing in industry. As industrial processes are time-varying, on-line learning algorithms should be adaptive to capture process changes. On-line ensemble methods have been shown to provide better generalization performance than single models in changing environments. However, most on-line ensembles do not include and exclude models during on-line operation. As a result, the ensembles have limited adaptation capability. Moreover, a higher performance can be obtained by combining a selected set of most relevant models of the ensemble for the current situation, rather than combining all the models. This paper proposes a new on-line learning ensemble of regressor models using an ordered aggregation (OA) technique which is able to provide on-line predictions of variables in changing environments. OA dynamically selects an optimal size and composition of a subset of models based on the minimization of the ensemble error on the newest sample. The proposed strategy overcomes the problem of defining the optimal ensemble size, and in most cases it obtains better performance than aggregating all the models. Models are added or removed for assuring adaptation of the ensemble in changing environments. Furthermore, this paper proposes and integrates a new on-line Extreme Learning Machine (ELM) neural network model with variable forgetting factor (FF) using the directional FF method which shows superior performance in industrial applications when compared to the well-known On-line Sequential ELM (OS-ELM) algorithm. Experiments are reported to demonstrate the performance and effectiveness of the proposed methods.

Self-adaptive extreme learning machine

In order to overcome the disadvantage of the traditional algorithm for SLFN (single-hidden layer feedforward neural network), an improved algorithm for SLFN, called extreme learning machine (ELM), is proposed by Huang et al. However, ELM is sensitive to the neuron number in hidden layer and its selection is a difficult-to-solve problem. In this paper, a self-adaptive mechanism is introduced into the ELM. Herein, a new variant of ELM, called self-adaptive extreme learning machine (SaELM), is proposed. SaELM is a self-adaptive learning algorithm that can always select the best neuron number in hidden layer to form the neural networks. There is no need to adjust any parameters in the training process. In order to prove the performance of the SaELM, it is used to solve the Italian wine and iris classification problems. Through the comparisons between SaELM and the traditional back propagation, basic ELM and general regression neural network, the results have proven that SaELM has a faster learning speed and better generalization performance when solving the classification problem.

An ELM-Based Approach for Estimating Train Dwell Time in Urban Rail Traffic

Dwell time estimation plays an important role in the operation of urban rail system. On this specific problem, a range of models based on either polynomial regression or microsimulation have been proposed. However, the generalization performance of polynomial regression models is limited and the accuracy of existing microsimulation models is unstable. In this paper, a new dwell time estimation model based on extreme learning machine (ELM) is proposed. The underlying factors that may affect urban rail dwell time are analyzed first. Then, the relationships among different factors are extracted and modeled by ELM neural networks, on basis of which an overall estimation model is proposed. At last, a set of observed data from Beijing subway is used to illustrate the proposed method and verify its overall performance.

Lung disease detection using feature extraction and extreme learning machine

INTRODUCTION: The World Health Organization estimates that by 2030 the Chronic Obstructive Pulmonary Disease (COPD) will be the third leading cause of death worldwide. Computerized Tomography (CT) images of lungs comprise a number of structures that are relevant for pulmonary disease diagnosis and analysis. METHODS: In this paper, we employ the Adaptive Crisp Active Contour Models (ACACM) for lung structure segmentation. And we propose a novel method for lung disease detection based on feature extraction of ACACM segmented images within the cooccurrence statistics framework. The spatial interdependence matrix (SIM) synthesizes the structural information of lung image structures in terms of three attributes. Finally, we perform a classification experiment on this set of attributes to discriminate two types of lung diseases and health lungs. We evaluate the discrimination ability of the proposed lung image descriptors using an extreme learning machine neural network (ELMNN) comprising 4-10 neurons in the hidden layer and 3 neurons in the output layer to map each pulmonary condition. This network was trained and validated by applying a holdout procedure. RESULTS: The experimental results achieved 96% accuracy demonstrating the effectiveness of the proposed method on identifying normal lungs and diseases as COPD and fibrosis. CONCLUSION: Our results lead to conclude that the method is suitable to integrate clinical decision support systems for pulmonary screening and diagnosis.

Breast Cancer Diagnosis using Artificial Neural Networks with Extreme Learning Techniques

Breast cancer is the second cause of dead among women. Early detection followed by appropriate cancer treatment can reduce the deadly risk. Medical professionals can make mistakes while identifying a disease. The help of technology such as data mining and machine learning can substantially improve the diagnosis accuracy. Artificial Neural Networks (ANN) has been widely used in intelligent breast cancer diagnosis. However, the standard Gradient-Based Back Propagation Artificial Neural Networks (BP ANN) has some limitations. There are parameters to be set in the beginning, long time for training process, and possibility to be trapped in local minima. In this research, we implemented ANN with extreme learning techniques for diagnosing breast cancer based on Breast Cancer Wisconsin Dataset. Results showed that Extreme Learning Machine Neural Networks (ELM ANN) has better generalization classifier model than BP ANN. The development of this technique is promising as intelligent component in medical decision support systems.

Adaptive Control of Nonlinear Discrete-Time Systems by Using OS-ELM Neural Networks

As a kind of novel feedforward neural network with single hidden layer, ELM (extreme learning machine) neural networks are studied for the identification and control of nonlinear dynamic systems. The property of simple structure and fast convergence of ELM can be shown clearly. In this paper, we are interested in adaptive control of nonlinear dynamic plants by using OS-ELM (online sequential extreme learning machine) neural networks. Based on data scope division, the problem that training process of ELM neural network is sensitive to the initial training data is also solved. According to the output range of the controlled plant, the data corresponding to this range will be used to initialize ELM. Furthermore, due to the drawback of conventional adaptive control, when the OS-ELM neural network is used for adaptive control of the system with jumping parameters, the topological structure of the neural network can be adjusted dynamically by using multiple model switching strategy, and an MMAC (multiple model adaptive control) will be used to improve the control performance. Simulation results are included to complement the theoretical results.

Identification of nonlinear system using extreme learning machine based Hammerstein model

In this paper, a new method for nonlinear system identification via extreme learning machine neural network based Hammerstein model (ELM-Hammerstein) is proposed. The ELM-Hammerstein model consists of static ELM neural network followed by a linear dynamic subsystem. The identification of nonlinear system is achieved by determining the structure of ELM-Hammerstein model and estimating its parameters. Lipschitz quotient criterion is adopted to determine the structure of ELM-Hammerstein model from input–output data. A generalized ELM algorithm is proposed to estimate the parameters of ELM-Hammerstein model, where the parameters of linear dynamic part and the output weights of ELM neural network are estimated simultaneously. The proposed method can obtain more accurate identification results with less computation complexity. Three simulation examples demonstrate its effectiveness.