Extreme Learning Machine Autoencoder Research Articles

Many-objective coevolutionary learning algorithm with extreme learning machine auto-encoder for ensemble classifier of feedforward neural networks

In artificial neural network (ANN) learning, empirical risk can be expressed by training error, while structural risk can be expressed by network complexity. Learning from data is often seen as a tradeoff between these two risks. Additionally, balancing training error and validation error can overcome the overfitting problem to some extent. It is known that the network performance is also related to regularization term. In order to consider four factors, i.e. training error, validation error, network complexity and regularization term simultaneously in the training process of a single-hidden layer feedforword neural network (SLFN), a many-objective coevolutionary learning algorithm (MOCELA) integrated with extreme learning machine auto-encoder (ELMAE), called MOCELA-ELMAE is presented. In MOCELA, the non-dominated sorting genetic algorithm III (NSGA-III) is improved for handling the many-objective optimization model with hybrid variables, where binary coding is used for structure learning and real coding is utilized for representing input parameters, referring to all input weights and hidden biases of the AE network. Output parameters of AE, i.e. output weights are analytically calculated by the non-iterative learning rule. The network structure and connection parameters of SLFN are determined based on those of AE. A set of Pareto optimal solutions are eventually collected by the MOCELA-ELMAE, which represents multiple optimal SLFNs. To make the final decision, three best SLFNs with minimum validation errors are selected as the base classifiers for selective ensemble learning. Extensive experiments are implemented on the benchmark classification data sets of UCI machine learning repository, and obvious improvements have been observed when the proposed MOCELA-ELMAE is compared with the NSGA-III based on hybrid coding and completely non-iterative learning of SLFN respectively. The experimental results also illustrate that the MOCELA-ELMAE performs much better than other state-of-the-art learning algorithms on many data sets.

Dynamic identification of coupler yaw angle of heavy haul locomotive: An optimal multi-task ELM-based method

The lateral instability of the couplers seriously threaten the operation safety of 20,000-tonne heavy-haul trains. It is essential to dynamically monitor the coupler yaw angle (CYA) in order to evaluate the operation stability of the train. However, the traditional direct measurement methods for CYA are hindered by the harsh service environment, making it difficult to maintain long-term stability in monitoring. To address these issues, within this paper a multi-task deep multiple kernel extreme learning machine (MT-DMKELM) is developed to quantitatively identify CYA based on the multi-channel relative displacements between the car body and frame (RDCFs). Firstly, the correlation of multi-channel RDCFs is analyzed based on the dynamics model, and the data fusion of strong correlation channels is achieved through an extreme learning machine auto-encoder. Then, a deep network for feature extraction is created by stacking numerous multiple kernel extreme learning machine auto-encoders. Further, the high-level abstract features are input to multiple kernel extreme learning machine for regression estimation of CYA. In addition, the optimal model parameters are obtained using the genetic mutation particle swarm optimization for MT-DMKELM to improve the CYA identification performance. Finally, both dynamics simulation and field test are conducted to confirm the capability of the proposed methods in CYA identification.

Prediction of compressive strength of nano-silica modified engineering cementitious composites exposed to high temperatures using hybrid deep learning models

This study estimated the compressive strength of nano-silica-modified engineering cementitious composites subjected to high temperatures using innovative hybrid deep learning models. The innovative hybrid models in this study were designed using autoencoder (AE)-decision tree (DT) and autoencoder (AE)-extreme learning machine (ELM). Additionally, ELM, DT, and deep AE models in this study were designed to compare the results of innovative hybrid deep learning models. The sensitivity analysis was used for the statistical assessment of the experimental results. The input variables of the models were selected as the cement amount, fly ash amount, sand amount, water amount, high-range water reducer amount, PVA (polyvinylalcohol) fiber amount, nano-silica amount, and the degree of exposure to high temperatures. The compressive strength was used as the output variable of the models. The mixture ratio in the experimental study was 583 kg/m3 cement, 467 kg/m3 sand, 700 kg/m3 fly ash, 187 kg/ m3 water, PVA fiber (0.5 %, 1 %, 1.5 % and 2 %) and nano silica (0 %, 1 %, 2 %, 3 % and 4 %) were used. The ELM, DT, and deep AE models estimated the compressive strength of nano-silica-modified engineering cementitious composites subjected to high temperatures with 93.86 %, 77.35 %, and 86.5 % accuracy, respectively. Also, the same compressive strength was estimated with 94.28 % and 98 % accuracy using the hybrid deep AE-DT and AE-ELM models. This study found that the innovative hybrid deep AE-ELM model predicted compressive strength with higher accuracy than the deep AE-DT, DT, ELM, and deep AE models. Additionally, the deep AE-DT model predicted compressive strength with higher accuracy than non-hybrid models. Thus, it can be stated that innovative hybrid deep models are more advantageous than other models in estimating the compressive strength of ECC. The sensitivity analysis obtained that the PVA fiber was the most significant variable affecting the compressive strength results of nano-silica-modified engineering cementitious composites subjected to high temperatures.

UCAV situation assessment based on SAE-ELM

A Stacked Autoencoder Extreme learning machine (SAE-ELM) assessment model is proposed to study the weight of an Unmanned Combat Air Vehicle (UCAV) situation assessment. On the basis of the existing air combat situation description parameters and their optimized situation advantage function weights, the mapping relationship between the air combat situation description parameters and the situation advantage function weights is established by using the SAE-ELM to achieve a more accurate evaluation. From the four error indicators of MAE, RMSE, MAPE, and R-Squared, the simulation experiment shows that the SAE-ELM evaluation error is the smallest and can effectively achieve accurate situation assessment.

Novel Hybrid Calibration Transfer Method Based on Nonlinear Dimensionality Reduction for Robust Standardization in Near-Infrared Spectroscopy

To correct the spectral changes measured on different instruments in near-infrared (NIR) spectroscopy, a novel hybrid calibration transfer method with nonlinear dimensionality reduction based on direct standardization (DS) and autoencoder (AE), named DS-AE, is reported. First, DS was employed to preliminarily eliminate the spectral difference from the master and slave instruments. Next, the spectral features were extracted by AE to construct the partial least squares (PLS) calibration model. Compared with the linear dimensionality reduction methods, AE learns more latent features of the input spectra that are beneficial to reflect the chemical information of samples. Two NIR experiment datasets, including wheat and corn samples measured on different spectrometers, were employed to evaluate the performance of the DS-AE method. DS, piecewise direct standardization (PDS), canonical correlation analysis (CCA), principal components canonical correlation analysis (PC-CCA), and transfer via an extreme learning machine auto-encoder (TEAM) were introduced for comparative analysis with the proposed method. The results showed that DS-AE provided the lowest root mean squared error of prediction (RMSEP) for the wheat dataset; For the corn dataset, DS-AE provided lower RMSEP than DS, PDS, and CCA, and comparable with PC-CCA and TEAM. The score of the PLS principal components (PCs) describes the spectral differences of different instruments. The results indicated that the hybrid DS-AE method effectively corrected for the spectral variations. In summary, the proposed hybrid DS-AE method provided an alternative for robust standardization of near-infrared spectra measured on different instruments.

Autoencoder Extreme Learning Machine for Fingerprint-Based Positioning: A Good Weight Initialization is Decisive

Autoencoder Extreme Learning Machine for Fingerprint-Based Positioning: A Good Weight Initialization is Decisive

Multilayer-graph-embedded extreme learning machine for performance degradation prognosis of bearing

As a key component in electromechanical systems, the health condition monitoring of rolling bearings is crucial for the safe operation of the whole system. For this purpose, the prediction of rolling bearing performance degradation is indispensable. To improve the accuracy of Extreme Learning Machine (ELM) based algorithms for the bearing performance degradation prediction, a novel Graph embedded ELM autoencoder (GEELM-AE) is first constructed via combining the graph embedding framework and then a performance degradation prediction method of Multilayer-Graph-embedded ELM (MGEELM) with a deep framework is developed by stacking multiple GEELM-AEs. The MGEELM algorithm not only extracts the abstract features in the data by virtue of its deep structure, but also maintains the local structural information and non-local structural information of the data during feature extraction. Accordingly, the performance degradation trend of rolling bearings can be accurately predicted with the proposed MGEELM. Moreover, the MGEELM algorithm does not require reverse fine-tuning, which greatly reduces the training time and improves the prediction efficiency. The advantages of the proposed method are validated by rolling bearing life-cycle vibration data. In the experimental analysis, the four performance indicators of the proposed method obviously outperform the comparative methods, which indicates the superior ability of the proposed method in tracking the bearing operating condition evolution.

Deep hypergraph autoencoder embedding: An efficient intelligent approach for rotating machinery fault diagnosis

Intelligent fault diagnosis based on deep learning (DL) has been widely used in various engineering practices. However, when confronting massive unlabeled industrial data, traditional data-driven intelligent fault diagnosis approaches cannot fully mine the correlation geometric structure information between data, and therefore cannot obtain good fault diagnosis results. To overcome this difficulty, an efficient fault diagnosis method based on deep hypergraph autoencoder embedding (DHAEE) is presented in this study. First, unlabeled vibration signals are converted into hypergraphs by applying the designed hypergraph construction method. Second, a hypergraph convolutional extreme learning machine autoencoder (HCELM-AE) is designed, which can mine the higher-order structural information and subspace structural information of the original unlabeled data by designing hypergraph convolutional and self-representation layers. Furthermore, by stacking multiple HCELM-AE modules in a DL framework, the DHAEE and its corresponding fault diagnosis method is constructed, which not only has the advantage of high computational efficiency of ELM-AE, but also has strong representational learning ability of DL methods. Finally, the effectiveness of the DHAEE based fault diagnosis method is verified by rolling bearing fault data and rotor fault data. Experimental results show that the presented fault diagnosis method has higher accuracy and lower computational complexity than other comparison methods, thus proving that DHAEE is an efficient intelligent massive unlabeled data processing approach for rotating machinery fault diagnosis.

Homogeneous ensemble extreme learning machine autoencoder with mutual representation learning and manifold regularization for medical datasets

Homogeneous ensemble extreme learning machine autoencoder with mutual representation learning and manifold regularization for medical datasets

Multilayer Fisher extreme learning machine for classification

As a special deep learning algorithm, the multilayer extreme learning machine (ML-ELM) has been extensively studied to solve practical problems in recent years. The ML-ELM is constructed from the extreme learning machine autoencoder (ELM-AE), and its generalization performance is affected by the representation learning of the ELM-AE. However, given label information, the unsupervised learning of the ELM-AE is difficult to build the discriminative feature space for classification tasks. To address this problem, a novel Fisher extreme learning machine autoencoder (FELM-AE) is proposed and is used as the component for the multilayer Fisher extreme leaning machine (ML-FELM). The FELM-AE introduces the Fisher criterion into the ELM-AE by adding the Fisher regularization term to the objective function, aiming to maximize the between-class distance and minimize the within-class distance of abstract feature. Different from the ELM-AE, the FELM-AE requires class labels to calculate the Fisher regularization loss, so that the learned abstract feature contains sufficient category information to complete classification tasks. The ML-FELM stacks the FELM-AE to extract feature and adopts the extreme leaning machine (ELM) to classify samples. Experiments on benchmark datasets show that the abstract feature extracted by the FELM-AE is more discriminative than the ELM-AE, and the classification results of the ML-FELM are more competitive and robust in comparison with the ELM, one-dimensional convolutional neural network (1D-CNN), ML-ELM, denoising multilayer extreme learning machine (D-ML-ELM), multilayer generalized extreme learning machine (ML-GELM), and hierarchical extreme learning machine with L21‑norm loss and regularization (H-LR21-ELM).

EA-CNN: A smart indoor 3D positioning scheme based on Wi-Fi fingerprinting and deep learning

Accurate indoor location information in multi-building/floor environments is essential for establishing many indoor location-based services (LBS). Wi-Fi fingerprinting with received signal strength indicators (RSSI) has become one of the most practical techniques to localize users indoors. However, the fluctuation of wireless signals caused by fading, the multipath effect, and device heterogeneity leads to considerable variations in RSSIs, which poses a challenge to accurate localization. This paper proposes an indoor positioning method, which is constructed based on a convolutional neural network (CNN) framework. Specifically, a novel model by combining an extreme learning machine autoencoder (ELM-AE) with a two-dimensional CNN is proposed. The ELM-AE extracts critical features by reducing input dimensions, while the CNN is trained to effectively achieve significant performance in the positioning phase. To increase positioning accuracy and deal with data shortages, a data augmentation strategy by frequently adding noisy data to the original fingerprint map is devised. The statistical properties namely, the RSS values of each MAC address, are utilized to adjust input noise. The performance of the proposed system is evaluated on two Tampere and UJIIndoorLoc datasets. The experimental results show that EA-CNN achieves better performance than CNN by decreasing average positioning error up to 40.95% and 43.74% in Tampere and UJIIndoorLoc datasets, respectively. Compared to state-of-the-art deep learning-based methods, the positioning performance improves up to 68.36% in Tampere and 67.56% in the UJIIndoorLoc dataset by exploiting only 25% of the training samples.Compared to several state-of-the-art deep learning models, EA-CNN achieves higher accuracy in both positioning and floor estimation.

DC-DC Buck circuit fault diagnosis with insufficient state data based on deep model and transfer strategy

As the common power supply of electronic systems, the fault of DC-DC Buck circuit will trigger the faulty power supply which eventually cause serious problems in system tasks. Therefore, it is of great significance to accurately identify the faults of DC-DC Buck circuit. Due to the high cost of collecting circuit fault data set, the lack of effective fault data will lead to the inability to establish an intelligent fault diagnosis model. In this paper, the model of source circuit with sufficient data is used to solve the problem of fault diagnosis of target circuit with lack of data. A deep transfer kernel extreme learning machine auto encoder (DKEA) model is designed. The activation function of Gaussian error linear units (GELU) is used to describe the probability of neuron input, and the kernel extreme learning machine is employed as the classifier to complete the diagnosis task. First, only the output voltage signal of the circuit is collected to extract its primary features in time and frequency domain, and then the primary features are mined and classified by using the deep extreme learning machine auto encoder (ELM-AE) model. Sufficient source domain data is used to train the deep ELM-AE model, and the parameters obtained are used as the initialization parameters of the target domain diagnosis model. With the help of transfer learning, a small number of training samples in the target domain are used to fine-tune the deep ELM-AE model to adapt to the remaining testing samples in the target domain. The experimental results proves that the proposed method can effectively achieve the transfer diagnosis between different DC-DC Buck circuits based on the source domain data.

Training of an Extreme Learning Machine Autoencoder Based on an Iterative Shrinkage-Thresholding Optimization Algorithm

Orthogonal transformations, proper decomposition, and the Moore–Penrose inverse are traditional methods of obtaining the output layer weights for an extreme learning machine autoencoder. However, an increase in the number of hidden neurons causes higher convergence times and computational complexity, whereas the generalization capability is low when the number of neurons is small. One way to address this issue is to use the fast iterative shrinkage-thresholding algorithm (FISTA) to minimize the output weights of the extreme learning machine. In this work, we aim to improve the convergence speed of FISTA by using two fast algorithms of the shrinkage-thresholding class, called greedy FISTA (G-FISTA) and linearly convergent FISTA (LC-FISTA). Our method is an exciting proposal for decision-making involving the resolution of many application problems, especially those requiring longer computational times. In our experiments, we adopt six public datasets that are frequently used in machine learning: MNIST, NORB, CIFAR10, UMist, Caltech256, and Stanford Cars. We apply several metrics to evaluate the performance of our method, and the object of comparison is the FISTA algorithm due to its popularity for neural network training. The experimental results show that G-FISTA and LC-FISTA achieve higher convergence speeds in the autoencoder training process; for example, in the Stanford Cars dataset, G-FISTA and LC-FISTA are faster than FISTA by 48.42% and 47.32%, respectively. Overall, all three algorithms maintain good values of the performance metrics on all databases.

Broad Learning Based Dynamic Fuzzy Inference System With Adaptive Structure and Interpretable Fuzzy Rules

This article investigates the feasibility of applying the broad learning system (BLS) to realize a novel Takagi–Sugeno–Kang (TSK) neuro-fuzzy model, namely a broad learning based dynamic fuzzy inference system (BL-DFIS). It not only improves the accuracy and interpretability of neuro-fuzzy models but also solves the challenging problem that models are incapable of determining the optimal architecture autonomously. BL-DFIS first accomplishes a TSK fuzzy system under the framework of BLS, in which an extreme learning machine auto-encoder is employed to obtain feature representation in a fast and analytical way, and an interpretable linguistic fuzzy rule is integrated into the enhancement node to ensure the high interpretability of the system. Meanwhile, the extended-enhancement unit is designed to achieve the first-order TSK fuzzy system. In addition, a dynamic incremental learning algorithm with internal pruning and updating mechanism is developed for the learning of BL-DFIS, which enables the system to automatically assemble the optimal structure to obtain a compact rule base and an excellent classification performance. Experiments on benchmark datasets demonstrate that the proposed BL-DFIS can achieve a better classification performance than some state-of-the-art nonfuzzy and neuro-fuzzy methods, simultaneously using the most parsimonious model structure.

Wi-Fi fingerprinting-based floor detection using adaptive scaling and weighted autoencoder extreme learning machine

In practical applications, accurate floor determination in multi-building/floor environments is particularly useful and plays an increasingly crucial role in the performance of location-based services. An accurate and robust building and floor detection can reduce the location search space and ameliorate the positioning and wayfinding accuracy. As an efficient solution, this paper proposes a floor identification method that exploits statistical properties of wireless access point propagated signals to exponent received signal strength (RSS) in the radio map. Then, using single-layer extreme learning machine-weighted autoencoder (ELM-WAE) main feature extraction and dimensional reduction is implemented. Finally, ELM based classifier is trained over a new feature space to determine floor level. For the efficiency evaluation of our proposed model, we utilized three different datasets captured in the real scenarios. The evaluation result shows that the proposed model can achieve state-of-art performance and improve the accuracy of floor detection compared with multiple recent techniques. In this way, the floor level can be identified with 97.30%, 95.32%, and 96.39% on UJIIndoorLoc, Tampere, and UTSIndoorLoc datasets, respectively.

Wi-Fi fingerprinting-based floor detection using adaptive scaling and weighted autoencoder extreme learning machine

In practical applications, accurate floor determination in multi-building/floor environments is particularly useful and plays an increasingly crucial role in the performance of location-based services. An accurate and robust building and floor detection can reduce the location search space and ameliorate the positioning and wayfinding accuracy. As an efficient solution, this paper proposes a floor identification method that exploits statistical properties of wireless access point propagated signals to exponent received signal strength (RSS) in the radio map. Then, using single-layer extreme learning machine-weighted autoencoder (ELM-WAE) main feature extraction and dimensional reduction is implemented. Finally, ELM based classifier is trained over a new feature space to determine floor level. For the efficiency evaluation of our proposed model, we utilized three different datasets captured in the real scenarios. The evaluation result shows that the proposed model can achieve state-of-art performance and improve the accuracy of floor detection compared with multiple recent techniques. In this way, the floor level can be identified with 97.30%, 95.32%, and 96.39% on UJIIndoorLoc, Tampere, and UTSIndoorLoc datasets, respectively.

Remote Sensing Image Recognition Based on LOG-T-SSA-LSSVM and AE-ELM Network.

Aiming at the influence of different working conditions on recognition accuracy in remote sensing image recognition, this paper adopts hierarchical strategy to construct a network. Firstly, in order to establish the classification relationship between different samples, labeled samples are used for classification. A Logistic-T-distribution-Sparrow Search Algorithm-Least Squares Support Vector Machines (LOG-T-SSA-LSSVM) classification network is proposed. LOG-T-SSA algorithm is used to optimize parameters in LSSVM to establish a better network to achieve accurate classification between sample sets and then identify according to different categories. Through UCI dataset test, the accuracy of LOG-T-SSA-LSSVM network classification is significantly improved compared with that of contrast network. The autoencoder is integrated with Extreme Learning Machine, and the autoencoder is used to realize data compression. The advantages of Extreme Learning Machine (ELM) network, such as less training parameters, fast learning speed, and strong generalization ability, are fully utilized to realize efficient and supervised recognition. Experiments verify that the autoencoder-extreme learning machine (AE-ELM) network has a good recognition effect when the sigmoid activation function is selected and the number of hidden layer neurons are 2000. Finally, after image recognition under different working conditions, it is proved that the recognition accuracy of AE-ELM based on LOG-T-SSA-LSSVM classification is significantly improved compared with traditional ELM network and Particle Swarm Optimization-Extreme Learning Machine (PSO-ELM) network.

An Experimental Approach to Diagnose Covid-19 Using Optimized CNN

The outburst of novel corona viruses aggregated worldwide and has undergone severe trials to manage medical sector all over the world. A radiologist uses x-rays and Computed Tomography (CT) scans to analyze images through which the existence of corona virus is found. Therefore, imaging and visualization systems contribute a dominant part in diagnosing process and thereby assist the medical experts to take necessary precautions and to overcome these rigorous conditions. In this research, a Multi-Objective Black Widow Optimization based Convolutional Neural Network (MBWO-CNN) method is proposed to diagnose and classify covid-19 data. The proposed method comprises of four stages, preprocess the covid-19 data, attribute selection, tune parameters, and classify covid-19 data. Initially, images are fed to preprocess and features are selected using Convolutional Neural Network (CNN). Next, Multi-objective Black Widow Optimization (MBWO) method is imparted to finely tune the hyper parameters of CNN. Lastly, Extreme Learning Machine Auto Encoder (ELM-AE) is used to check the existence of corona virus and further classification is done to classify the covid-19 data into respective classes. The suggested MBWO-CNN model was evaluated for effectiveness by undergoing experiments and the outcomes attained were matched with the outcome stationed by prevailing methods. The outcomes confirmed the astonishing results of the ELM-AE model to classify covid-19 data by achieving maximum accuracy of 97.53%. The efficacy of the proposed method is validated and observed that it has yielded outstanding outcomes and is best suitable to diagnose and classify covid-19 data.

Compact Bat Algorithm with Deep Learning Model for Biomedical EEG EyeState Classification

Electroencephalography (EEG) eye state classification becomes an essential tool to identify the cognitive state of humans. It can be used in several fields such as motor imagery recognition, drug effect detection, emotion categorization, seizure detection, etc. With the latest advances in deep learning (DL) models, it is possible to design an accurate and prompt EEG EyeState classification problem. In this view, this study presents a novel compact bat algorithm with deep learning model for biomedical EEG EyeState classification (CBADL-BEESC) model. The major intention of the CBADL-BEESC technique aims to categorize the presence of EEG EyeState. The CBADL-BEESC model performs feature extraction using the ALexNet model which helps to produce useful feature vectors. In addition, extreme learning machine autoencoder (ELM-AE) model is applied to classify the EEG signals and the parameter tuning of the ELM-AE model is performed using CBA. The experimental result analysis of the CBADL-BEESC model is carried out on benchmark results and the comparative outcome reported the supremacy of the CBADL-BEESC model over the recent methods.

Binary imbalanced data classification based on diversity oversampling by generative models

In many practical applications, the data are class imbalanced. Accordingly, it is very meaningful and valuable to investigate the classification of imbalanced data. In the framework of binary imbalanced data classification, the synthetic minority oversampling technique (SMOTE) is the best-known oversampling method. However, for each positive sample, SMOTE generates only k synthetic samples on the lines between the positive sample and its k-nearest neighbors, resulting in three drawbacks: (1) SMOTE cannot effectively extend the training field of positive samples; (2) the generated positive samples lack diversity; (3) SMOTE does not accurately approximate the probability distribution of the positive samples. Therefore, two binary imbalanced data classification methods named BIDC1 and BIDC2 based on diversity oversampling by generative models are proposed. The BIDC1 and BIDC2 conduct diversity oversampling using extreme learning machine autoencoder and generative adversarial network, respectively. Extensive experiments on 26 data sets are conducted to compare the two methods with 14 state-of-the-art methods using five metrics: F-measure, G-means, AUC-area, MMD-score, and Silhouette-score. The experimental results demonstrate that the two methods outperform the other 14 methods.