Multilayer Extreme Learning Machine Research Articles

Coal Exploration Based on a Multilayer Extreme Learning Machine and Satellite Images

The demand for coal has been on the rise in modern society. With the number of opencast coal mines decreasing, it has become increasingly difficult to find coal. Low efficiencies and high casualty rates have always been problems in the process of coal exploration due to complicated geological structures in coal mining areas. Therefore, we propose a new exploration technology for coal that uses satellite images to explore and monitor opencast coal mining areas. First, we collected bituminous coal and lignite from the Shenhua opencast coal mine in China in addition to non-coal objects, including sandstones, soils, shales, marls, vegetation, coal gangues, water, and buildings. Second, we measured the spectral data of these objects through a spectrometer. Third, we proposed a multilayer extreme learning machine algorithm and constructed a coal classification model based on that algorithm and the spectral data. The model can assist in the classification of bituminous coal, lignite, and non-coal objects. Fourth, we collected Landsat 8 satellite images for the coal mining areas. We divided the image of the coal mine using the constructed model and correctly described the distributions of bituminous coal and lignite. Compared with the traditional coal exploration method, our method manifested an unparalleled advantage and application value in terms of its economy, speed, and accuracy.

Fabric Weave Pattern and Yarn Color Recognition and Classification Using a Deep ELM Network

Usually, a fabric weave pattern is recognized using methods which identify the warp floats and weft floats. Although these methods perform well for uniform or repetitive weave patterns, in the case of complex weave patterns, these methods become computationally complex and the classification error rates are comparatively higher. Furthermore, the fault-tolerance (invariance) and stability (selectivity) of the existing methods are still to be enhanced. We present a novel biologically-inspired method to invariantly recognize the fabric weave pattern (fabric texture) and yarn color from the color image input. We proposed a model in which the fabric weave pattern descriptor is based on the HMAX model for computer vision inspired by the hierarchy in the visual cortex, the color descriptor is based on the opponent color channel inspired by the classical opponent color theory of human vision, and the classification stage is composed of a multi-layer (deep) extreme learning machine. Since the weave pattern descriptor, yarn color descriptor, and the classification stage are all biologically inspired, we propose a method which is completely biologically plausible. The classification performance of the proposed algorithm indicates that the biologically-inspired computer-aided-vision models might provide accurate, fast, reliable and cost-effective solution to industrial automation.

Prediction model of permeability index for blast furnace based on the improved multi-layer extreme learning machine and wavelet transform

The permeability index of the blast furnace is a significant symbol to measure the smooth operation of the blast furnace. This paper proposes a novel prediction model for permeability index of the blast furnace based on the multi-layer extreme learning machine (ML-ELM), the principal component analysis (PCA) method and wavelet transform (called as W-PCA-ML-ELM prediction model). This modified ML-ELM algorithm is based on the ML-ELM algorithm and the PCA method (named as PCA-ML-ELM). The PCA method is applied on the ML-ELM algorithm to improve the algebraic property of the last hidden layer output matrix which deteriorates its generalization performance due to the high multicollinearity. Because the production data of the blast furnace field contain noises, this paper applies the wavelet transform to remove the noise. Comparing with other prediction models which are based on the ML-ELM, the ELM, the BP and the SVM, simulation results illustrate that the better generalization performance and stability of the proposed W-PCA-ML-ELM prediction model.

Extended Hierarchical Extreme Learning Machine with Multilayer Perceptron

The Deep Learning approach provides a high performance of classification, especially when invoking image classification problems. However, a shortcoming of the traditional Deep Learning method is the large time scale of training. The hierarchical extreme learning machine (H-ELM) framework was based on the hierarchical learning architecture of multilayer perceptron to address the problem. H-ELM is composed of two parts; the first entails unsupervised multilayer encoding, and the second is the supervised feature classification. H-ELM can give a higher accuracy rate than the traditional ELM. However, there still remains room to enhance its classification performance. This paper therefore proposes a new method termed the extending hierarchical extreme learning machine (EH-ELM), which extends the number of layers in the supervised portion of the H-ELM from a single layer to multiple layers. To evaluate the performance of the EH-ELM, the various classification datasets were studied and compared with the H-ELM and the multilayer ELM, as well as various state-of-the-art such deep architecture methods. The experimental results show that the EH-ELM improved the accuracy rates over most other methods.

A Multiple Hidden Layers Extreme Learning Machine Method and Its Application

Extreme learning machine (ELM) is a rapid learning algorithm of the single‐hidden‐layer feedforward neural network, which randomly initializes the weights between the input layer and the hidden layer and the bias of hidden layer neurons and finally uses the least‐squares method to calculate the weights between the hidden layer and the output layer. This paper proposes a multiple hidden layers ELM (MELM for short) which inherits the characteristics of parameters of the first hidden layer. The parameters of the remaining hidden layers are obtained by introducing a method (make the actual output zero error approach the expected hidden layer output). Based on the MELM algorithm, many experiments on regression and classification show that the MELM can achieve the satisfactory results based on average precision and good generalization performance compared to the two‐hidden‐layer ELM (TELM), the ELM, and some other multilayer ELM.

Kernel-Based Multilayer Extreme Learning Machines for Representation Learning.

Recently, multilayer extreme learning machine (ML-ELM) was applied to stacked autoencoder (SAE) for representation learning. In contrast to traditional SAE, the training time of ML-ELM is significantly reduced from hours to seconds with high accuracy. However, ML-ELM suffers from several drawbacks: 1) manual tuning on the number of hidden nodes in every layer is an uncertain factor to training time and generalization; 2) random projection of input weights and bias in every layer of ML-ELM leads to suboptimal model generalization; 3) the pseudoinverse solution for output weights in every layer incurs relatively large reconstruction error; and 4) the storage and execution time for transformation matrices in representation learning are proportional to the number of hidden layers. Inspired by kernel learning, a kernel version of ML-ELM is developed, namely, multilayer kernel ELM (ML-KELM), whose contributions are: 1) elimination of manual tuning on the number of hidden nodes in every layer; 2) no random projection mechanism so as to obtain optimal model generalization; 3) exact inverse solution for output weights is guaranteed under invertible kernel matrix, resulting to smaller reconstruction error; and 4) all transformation matrices are unified into two matrices only, so that storage can be reduced and may shorten model execution time. Benchmark data sets of different sizes have been employed for the evaluation of ML-KELM. Experimental results have verified the contributions of the proposed ML-KELM. The improvement in accuracy over benchmark data sets is up to 7%.

Generalized extreme learning machine autoencoder and a new deep neural network

Extreme learning machine (ELM) is an efficient learning algorithm of training single layer feed-forward neural networks (SLFNs). With the development of unsupervised learning in recent years, integrating ELM with autoencoder has become a new perspective for extracting feature using unlabeled data. In this paper, we propose a new variant of extreme learning machine autoencoder (ELM-AE) called generalized extreme learning machine autoencoder (GELM-AE) which adds the manifold regularization to the objective of ELM-AE. Some experiments carried out on real-world data sets show that GELM-AE outperforms some state-of-the-art unsupervised learning algorithms, including k-means, laplacian embedding (LE), spectral clustering (SC) and ELM-AE. Furthermore, we also propose a new deep neural network called multilayer generalized extreme learning machine autoencoder (ML-GELM) by stacking several GELM-AE to detect more abstract representations. The experiments results show that ML-GELM outperforms ELM and many other deep models, such as multilayer ELM autoencoder (ML-ELM), deep belief network (DBN) and stacked autoencoder (SAE). Due to the utilization of ELM, ML-GELM is also faster than DBN and SAE.

Study on suitability and importance of multilayer extreme learning machine for classification of text data

The dynamic Web, which contains huge number of digital documents, is expanding day by day. Thus, it has become a tough challenge to search for a particular document from such a large volume of collections. Text classification is a technique which can speed up the search and retrieval tasks and hence is the need of the hour. Aiming in this direction, this study proposes an efficient technique that uses the concept of connected component (CC) of a graph and Wordnet along with four established feature selection techniques [e.g., TF-IDF, Chi-square, Bi-Normal Separation (BNS) and Information Gain (IG)] to select the best features from a given input dataset in order to prepare an efficient training feature vector. Next, multilayer extreme learning machine (ML-ELM) (which is based on the architecture of deep learning) and other state-of-the-art classifiers are trained on this efficient training feature vector for classification of text data. The experimental work has been carried out on DMOZ and 20-Newsgroups datasets. We have studied the behavior and compared the results of different classifiers using these four important feature selection techniques used for classification process and observed that ML-ELM achieved the maximum overall F-measure of 72.28 % on DMOZ dataset using TF-IDF as the feature selection technique and 81.53 % on 20-Newsgroups dataset using BNS as the feature selection technique compared to other state-of-the-art classifiers which signifies the usefulness of deep learning used by ML-ELM for classifying the text data. Experimental results on these benchmark datasets show the stability and effectiveness of our approach over other competing approaches.

Encrypted image classification based on multilayer extreme learning machine

Nowadays, numerous corporations (such as Google, Baidu, etc.) require an efficient and effective search algorithm to crawl out the images with queried objects from databases. Moreover, privacy protection is a significant issue such that confidential images must be encrypted in corporations. Nevertheless, decrypting and then classifying millions of encrypted images becomes a heavy burden to computation. In this paper, we proposed an encrypted image classification framework based on multi-layer extreme learning machine that is able to directly classify encrypted images without decryption. Experiments were conducted on popular handwritten digits and letters databases. Results demonstrate that the proposed framework is secure, efficient and accurate for classifying encrypted images.

Classification Based on Multilayer Extreme Learning Machine for Motor Imagery Task from EEG Signals

Classification of motor imagery electroencephalogram (EEG) is one of the most important technologies for BCI. To improve the accuracy, this paper introduces a classification system based on Multilayer Extreme Learning Machine (ML-ELM). In the system, the combination of PCA and LDA is chosen as the method of feature extraction and the ML-ELM is used to classify. The ML-ELM has not only the advantage which ELM has but also better performance than ELM. In the experiment, our method is compared with the methods based on ELM, such as kernel-ELM, Constrained-ELM and V-ELM, and some state-of–the-art methods on the same dataset. The experimental results show that ML-ELM is much more suitable for motor imagery EEG data and has better performance than the others.

Two-hidden-layer extreme learning machine for regression and classification

As a single-hidden-layer feedforward neural network, an extreme learning machine (ELM) randomizes the weights between the input layer and the hidden layer as well as the bias of hidden neurons, and analytically determines the weights between the hidden layer and the output layer using the least-squares method. This paper proposes a two-hidden-layer ELM (denoted TELM) by introducing a novel method for obtaining the parameters of the second hidden layer (connection weights between the first and second hidden layer and the bias of the second hidden layer), hence bringing the actual hidden layer output closer to the expected hidden layer output in the two-hidden-layer feedforward network. Simultaneously, the TELM method inherits the randomness of the ELM technique for the first hidden layer (connection weights between the input weights and the first hidden layer and the bias of the first hidden layer). Experiments on several regression problems and some popular classification datasets demonstrate that the proposed TELM can consistently outperform the original ELM, as well as some existing multilayer ELM variants, in terms of average accuracy and the number of hidden neurons.

Multilayer Extreme Learning Machine With Subnetwork Nodes for Representation Learning.

The extreme learning machine (ELM), which was originally proposed for "generalized" single-hidden layer feedforward neural networks, provides efficient unified learning solutions for the applications of clustering, regression, and classification. It presents competitive accuracy with superb efficiency in many applications. However, ELM with subnetwork nodes architecture has not attracted much research attentions. Recently, many methods have been proposed for supervised/unsupervised dimension reduction or representation learning, but these methods normally only work for one type of problem. This paper studies the general architecture of multilayer ELM (ML-ELM) with subnetwork nodes, showing that: 1) the proposed method provides a representation learning platform with unsupervised/supervised and compressed/sparse representation learning and 2) experimental results on ten image datasets and 16 classification datasets show that, compared to other conventional feature learning methods, the proposed ML-ELM with subnetwork nodes performs competitively or much better than other feature learning methods.

Denoising Laplacian multi-layer extreme learning machine

Most of semi-supervised learning algorithms based on manifold regularization framework are surface learning algorithms, such as semi-supervised ELM (SS-ELM) and Laplacian smooth twin support vector machine (Lap-STSVM). Multi-layer extreme learning machine (ML-ELM) stacks extreme learning machine based auto encoder (ELM-AE) to create a multi-layer neural network. ML-ELM not only approximates the complicated function but also achieves fast training time. The outputs of ELM-AE are the same as inputs, which cannot guarantee the effectiveness of the learning feature representations. We put forward extreme learning machine based denoising auto encoder (ELM-DAE) which introduces local denoising criterion into ELM-AE and is used as the basic component for Denoising ML-ELM. Resembling ML-ELM, Denoising ML-ELM stacks ELM-DAE to create a deep network. And then we introduce manifold regularization into the model of Denoising ML-ELM and propose denoising Laplacian ML-ELM (Denoising Lap-ML-ELM). Denoising Lap-ML-ELM is more efficient than SS-ELM in classification and does not need to spend too much time. Experimental results show that Denoising ML-ELM and Denoising Lap-ML-ELM are effective learning algorithms.

Deep Extreme Learning Machine and Its Application in EEG Classification

Recently, deep learning has aroused wide interest in machine learning fields. Deep learning is a multilayer perceptron artificial neural network algorithm. Deep learning has the advantage of approximating the complicated function and alleviating the optimization difficulty associated with deep models. Multilayer extreme learning machine (MLELM) is a learning algorithm of an artificial neural network which takes advantages of deep learning and extreme learning machine. Not only does MLELM approximate the complicated function but it also does not need to iterate during the training process. We combining with MLELM and extreme learning machine with kernel (KELM) put forward deep extreme learning machine (DELM) and apply it to EEG classification in this paper. This paper focuses on the application of DELM in the classification of the visual feedback experiment, using MATLAB and the second brain-computer interface (BCI) competition datasets. By simulating and analyzing the results of the experiments, effectiveness of the application of DELM in EEG classification is confirmed.