Sparse Extreme Learning Machine Research Articles

Enhanced Epileptic Seizure Identification using Sparse ELM-ABO Fusion with Feature Reduction and Multi-class Classification

The interpretation of the electroencephalogram (EEG) signal is one method that can be utilized to diagnose epilepsy, which is one of the most prevalent brain illnesses. The length of an EEG signal is typically quite long, making it difficult to interpret manually. Extreme Learning Machine (ELM) is used to detection of Epilepsy and Seizure. But in ELM Storage space and training time is high. In order to reduce training time and storage space African Buffalo Optimization (ABO) algorithm is used. ABO is combined with Sparse ELM to improve the speed, accuracy of detection and reduce the storage space. First, Wavelet transform is used to extract relevant features. Due to their high dimensionality, these features are then reduced by using linear discriminant analysis (LDA). The proposed Hybrid Sparse ELM technique is successfully implemented for diagnosing epileptic seizure disease. For classification, the Sparse ELM-ABO classifier is applied to the UCI Epileptic Seizure Recognition Data Set training dataset, and the experimental findings are compared to those of the SVM, Sparse ELM, and ELM classifiers applied to the same database. The proposed model was tested in two scenarios: binary classification and multi-label classification. Seizure identification is the only factor in binary classification. Seizure and epilepsy identification are part of multi-label classification. It is observed that the proposed method obtained high accuracy in classification with less execution time along with performance evaluation of parameters such as prediction accuracy, specificity, precision, recall and F-score. Binary classification scores 96.08%, while multi-label classification achieves 90.89%.

Probabilistic outlier detection for robust regression modeling of structural response for high-speed railway track monitoring

Outlier detection is an important procedure taken in structural health monitoring (SHM) to create clean and reliable data. A robust time series outlier detection method incorporating a Bayesian perspective and an extreme learning machine (ELM) neural network model is proposed, with application to long-term monitoring data of ballastless tracks for high-speed railway systems. A robust sparse Bayesian ELM (SBELM) model is first established by computing the posterior probability density function of the ELM weight parameters and then marginalizing over the prediction-error precision parameter to obtain a robust nonlinear regression model between the track temperature and structural response. Both the posterior mean and the associated uncertainties of the robust SBELM model are then taken into account to compute the outlier probability for each suspicious data point, which quantifies their degree of data “outlier-ness.” It effectively takes into account the prediction uncertainty of the SBELM regression model. The method is applied to long-term monitoring data for track temperatures, and track strain and relative displacement responses, from two high-speed rail track systems where there are both slight and serious outliers. The results demonstrate that the proposed method can reliably detect outliers by quantifying the outlier probability and that the final results are robust to the selection of the “thresholds.” It is also shown that our new algorithm produces significantly improved model prediction performance after the outliers are detected and removed.

Genetic Algorithm-Based Ensemble Hybrid Sparse ELM for Grasp Stability Recognition With Multimodal Tactile Signals

Grasp stability recognition based on tactile perception has attracted increasing attention in the robotics community. In this article, we extract tactile features from multimodal tactile signals and propose a novel ensemble approach named genetic algorithm-based ensemble hybrid sparse extreme learning machine (GA-EHSELM) for the grasp stability recognition task. In contrast to traditional ensemble extreme learning machines (ELMs), the proposed approach takes the sparsity of the random weights assigned to the connection between the input layer and the hidden layer for each base learner into consideration, in order to avoid the problem of overfitting. In addition, The diversity of base learners is increased by constructing two types of sparse ELM (SELM). The random weights of the base SELMs are sampled from two different distributions with a certain probability. Furthermore, we utilize the genetic algorithm (GA) for the optimization of tactile features and the base sparse ELMs. To be specific, GA is employed for feature selection as well as the optimization of the sparsity of the random weights associated with different types of base learners and the number of each type of base learner. The effectiveness of the proposed approach has been demonstrated on a public tactile grasp stability dataset.

Physics-informed multi-fidelity learning-driven imaging method for electrical capacitance tomography

The electrical capacitance tomography is considered to be a promising non-invasive visualization method for the measurement of multiphase flow parameters, but low-quality tomograms limit the reliability and accuracy of measurements. In order to overcome this limitation and bottleneck, the physics-informed data-dependent prior learned from the given samples is introduced and coupled with the measurement mechanism and the domain knowledge modeled by a new L1/L2 norm-based regularizer into a new optimization imaging model in this study. To improve the convergence and reduce the computational load, the built imaging model is solved by a new numerical scheme driven by the split Bregman method and the forward–backward splitting method. A new physics-informed robust sparse extreme learning machine method that not only obeys the measurement mechanism in training but also promotes the robustness and performance of the model is proposed and used to build a novel multi-fidelity learning method to infer the physics-informed data-dependent prior. The new imaging method achieves multi-source data fusion, leads to new changes in reconstruction algorithms, increases the complementarity and diversity of image priors, unifies the image reconstruction and multi-fidelity learning method, and improves robustness, numerical stability and flexibility of the model. The reconstruction results show that the new imaging method achieves more accurate reconstruction with reduced sensitivity to noise compared to important and popular imaging methods. This study leads to new changes in imaging algorithms and mining and use of image priors, and serves as a catalyst for the paradigm shift in image reconstruction.

Study on the detection of heavy metal lead (Pb) in mussels based on near-infrared spectroscopy technology and a REELM classifier

It is toxic to consume mussels polluted by heavy metal Pb for humans. A quick, accurate, and non-destructive method based on near-infrared reflection spectroscopy (NIRS) has been investigated for the detection of Pb-polluted mussels in this paper. The spectral data of non-polluted and Pb-polluted mussels in the range of 950–1700 nm is acquired. A wavelength selection algorithm based on information measures of neighborhood rough set is applied to select optimal wavelengths. A sparse extreme learning machine (ELM) based on residual errors (REELM) is established as a classifier to detect Pb-polluted mussels. The average accuracy of 50 randomly assigned test datasets reaches 99.27% for detecting Pb-polluted mussels. For the class imbalance problem and mislabeled samples in the presence of practical applications, the detection performance of the proposed model has been analysed. The experimental results show that, compared with the traditional ELM and randomly pruned ELM, the REELM model is proven to be superior. The results indicate that NIRS combined with pattern recognition method has great potential to detect Pb pollution in mussels. This research is of significant importance in terms of the evaluation of edible quality and safety of mussels.

VLSI design of multiclass classification using sparse extreme learning machine for epilepsy and seizure detection

VLSI design of multiclass classification using sparse extreme learning machine for epilepsy and seizure detection

Multimodal Sparse Classifier for Adolescent Brain Age Prediction.

The study of healthy brain development helps to better understand both brain transformation and connectivity patterns, which happen during childhood to adulthood. This study presents a sparse machine learning solution across whole-brain functional connectivity measures of three datasets, derived from resting state functional magnetic resonance imaging (rs-fMRI) and two task fMRI data including a working memory n-back task (nb-fMRI) and an emotion identification task (em-fMRI). The fMRI data are collected from the Philadelphia Neurodevelopmental Cohort (PNC) for the prediction of brain age in adolescents. Due to extremely large variable-to-instance ratio of PNC data, a high-dimensional matrix with several irrelevant and highly correlated features is generated, and hence a sparse learning approach is necessary to extract effective features from fMRI data. We propose a sparse learner based on the residual errors along the estimation of an inverse problem for extreme learning machine (ELM). Our proposed method is able to overcome the overlearning problem by pruning several redundant features and their corresponding output weights. The proposed multimodal sparse ELM classifier based on residual errors is highly competitive in terms of classification accuracy compared to its counterparts such as conventional ELM, and sparse Bayesian learning ELM.

Bundle Extreme Learning Machine for Power Quality Analysis in Transmission Networks

This paper presents a novel method for online power quality data analysis in transmission networks using a machine learning-based classifier. The proposed classifier has a bundle structure based on the enhanced version of the Extreme Learning Machine (ELM). Due to its fast response and easy-to-build architecture, the ELM is an appropriate machine learning model for power quality analysis. The sparse Bayesian ELM and weighted ELM have been embedded into the proposed bundle learning machine. The case study includes real field signals obtained from the Turkish electricity transmission system. Most actual events like voltage sag, voltage swell, interruption, and harmonics have been detected using the proposed algorithm. For validation purposes, the ELM algorithm is compared with state-of-the-art methods such as artificial neural network and least squares support vector machine.

Multimodal Emotion Recognition Using the Symmetric S-ELM-LUPI Paradigm

Multimodal emotion recognition has become one of the new research fields of human-machine interaction. This paper focuses on feature extraction and data fusion in audio-visual emotion recognition, aiming at improving recognition effect and saving storage space. A semi-serial fusion symmetric method is proposed to fuse the audio and visual patterns of emotional recognition, and a method of Symmetric S-ELM-LUPI is adopted (Symmetric Sparse Extreme Learning Machine-Learning Using Privileged Information). The method inherits the generalized high speed of the Extreme Learning Machine, and combines this with the acceleration in the recognition process by the Learning Using Privileged Information and the memory saving of the Sparse Extreme Learning Machine. It is a learning method, which improves the traditional learning methods of examples and targets only. It introduces the role of a teacher in providing additional information to enhance the recognition (test) without complicating the learning process. The proposed method is tested on publicly available datasets and yields promising results. This method regards one pattern as the standard information source, while the other pattern as the privileged information source. Each mode can be treated as privileged information for another mode. The results show that this method is appropriate for multi-modal emotion recognition. For hundreds of samples, the execution time is less than one percent seconds. The sparsity of the proposed method has the advantage of storing memory economy. Compared with other machine learning methods, this method is more accurate and stable.

Content-Insensitive Blind Image Blurriness Assessment Using Weibull Statistics and Sparse Extreme Learning Machine

Most of the existing image blurriness assessment algorithms are proposed based on measuring image edge width, gradient, high-frequency energy, or pixel intensity variation. However, these methods are content sensitive with little consideration of image content variations, which causes variant estimations for images with different contents but same blurriness degrees. In this paper, a content-insensitive blind image blurriness assessment metric is developed utilizing Weibull statistics. Inspired by the property that the statistics of image gradient magnitude (GM) follows Weibull distribution, we parameterize the GM using $\beta$ (scale parameter) and $\gamma$ (shape parameter) of Weibull distribution. We also adopt skewness ( $\eta$ ) to measure the asymmetry of the GM distribution. In order to reduce the influence of image content and achieve more robust performance, divisive normalization is then incorporated to moderate the $\beta$ , $\gamma$ , and $\eta$ . The final image quality is predicted using a sparse extreme learning machine. Performances evaluation on the blur image subsets in LIVE, CSIQ, TID2008, and TID2013 databases demonstrate that the proposed method is highly correlated with human perception and robust with image contents. In addition, our method has low computational complexity which is suitable for online applications.

Local Block Multilayer Sparse Extreme Learning Machine for Effective Feature Extraction and Classification of Hyperspectral Images

Although extreme learning machines (ELM) have been successfully applied for the classification of hyperspectral images (HSIs), they still suffer from three main drawbacks. These include: 1) ineffective feature extraction (FE) in HSIs due to a single hidden layer neuron network used; 2) ill-posed problems caused by the random input weights and biases; and 3) lack of spatial information for HSIs classification. To tackle the first problem, we construct a multilayer ELM for effective FE from HSIs. The sparse representation is adopted with the multilayer ELM to tackle the ill-posed problem of ELM, which can be solved by the alternative direction method of multipliers. This has resulted in the proposed multilayer sparse ELM (MSELM) model. Considering that the neighboring pixels are more likely from the same class, a local block extension is introduced for MSELM to extract the local spatial information, leading to the local block MSELM (LBMSELM). The loopy belief propagation is also applied to the proposed MSELM and LBMSELM approaches to further utilize the rich spectral and spatial information for improving the classification. Experimental results show that the proposed methods have outperformed the ELM and other state-of-the-art approaches.

Effective Crack Damage Detection Using Multilayer Sparse Feature Representation and Incremental Extreme Learning Machine

Detecting cracks within reinforced concrete is still a challenging problem, owing to the complex disturbances from the background noise. In this work, we advocate a new concrete crack damage detection model, based upon multilayer sparse feature representation and an incremental extreme learning machine (ELM), which has both favorable feature learning and classification capabilities. Specifically, by cropping and using a sliding window operation and image rotation, a large number of crack and non-crack patches are obtained from the collected concrete images. With the existing image patches, the defect region features can be quickly calculated by the multilayer sparse ELM autoencoder networks. Then, the online incremental ELM classified network is used to recognize the crack defect features. Unlike the commonly-used deep learning-based methods, the presented ELM-based crack detection model can be trained efficiently without tediously fine-tuning the entire-network parameters. Moreover, according to the ELM theory, the proposed crack detector works universally for defect feature extraction and detection. In the experiments, when compared with other recently developed crack detectors, the proposed concrete crack detection model can offer outstanding training efficiency and favorable crack detecting accuracy.

Extreme Learning Machine with Elastic Net Regularization

Compared with deep neural learning, the extreme learning machine (ELM) can be quickly converged without iteratively tuning hidden nodes. Inspired by this merit, an extreme learning machine with elastic net regularization (ELM-EN) is proposed in this paper. The elastic net is a regularization method that combines LASSO and ridge penalties. This regularization can keep a balance between system stability and solution's sparsity. Moreover, an excellent optimization method, i.e., accelerated proximal gradient, is used to find the minimum of the system optimization function. Various datasets from UCI repository and two facial expression image datasets are used to validate the efficiency of our system. Final experimental results indicate that our ELM-EN requires less training time than multi-layer perceptron, and can achieve higher recognition accuracy than ELM and sparse ELM.

A Douglas–Rachford method for sparse extreme learning machine

A Douglas–Rachford method for sparse extreme learning machine

<inline-formula> <tex-math notation="LaTeX">$L_{21}$ </tex-math> </inline-formula>-Norm Based Loss Function and Regularization Extreme Learning Machine

Extreme learning machine (ELM) has gained increasing interests from various research fields recently. Researchers have proposed various extensions to improve its stability, sparsity, and generalization performance. In this paper, we propose a robust and sparse ELM to exploit $L_{21} $ -norm minimization of both loss function and regularization (LR21-ELM). Our $L_{21} $ -norm-based loss function can diminish the undue influence of noises and outliers of data points compared with the $L_{2} $ -norm based loss function and make the learned ELM model more robust and stable. The powerful structural sparse-inducing $L_{21} $ -norm regularization is integrated into the ELM objective function to eliminate the potential redundant neurons of ELM adaptively and reduce the complexity of the learning model. We introduce an effective iterative optimization algorithm to solve the $L_{21} $ -norm minimization problem. Empirical tests on a number of benchmark datasets indicate that our proposed algorithm can generate a more compact, robust, and discriminative model compared with the original ELM algorithm.

EEG classification using sparse Bayesian extreme learning machine for brain–computer interface

Mu rhythm is a spontaneous neural response occurring during a motor imagery (MI) task and has been increasingly applied to the design of brain–computer interface (BCI). Accurate classification of MI is usually rather difficult to be achieved since mu rhythm is very weak and likely to be contaminated by other background noises. As an extension of the single layer feedforward network, extreme learning machine (ELM) has recently proven to be more efficient than support vector machine that is a benchmark for MI-related EEG classification. With probabilistic inference, this study introduces a sparse Bayesian ELM (SBELM)-based algorithm to improve the classification performance of MI. SBELM is able to automatically control the model complexity and exclude redundant hidden neurons by combining advantageous of both ELM and sparse Bayesian learning. The effectiveness of SBELM for MI-related EEG classification is validated on a public dataset from BCI Competition IV IIb in comparison with several other competing algorithms. Superior classification accuracy confirms that the proposed SBELM-based algorithm is a promising candidate for performance improvement of an MI BCI.

A multiobjective optimization-based sparse extreme learning machine algorithm

Abstract Extreme Learning Machine (ELM) is a popular machine learning method and has been widely applied to real-world problems due to its fast training speed and good generalization performance. However, in ELM, the randomly assigned input weights and hidden biases usually degrade the generalization performance. Furthermore, ELM is considered as an empirical risk minimization model and easily leads to overfitting when dataset exists some outliers. In this paper, we proposed a novel algorithm named Multiobjective Optimization-based Sparse Extreme Learning Machine (MO-SELM), where parameter optimization and structure learning are integrated into the learning process to simultaneously enhance the generalization performance and alleviate the overfitting problem. In MO-SELM, the training error and the connecting sparsity are taken as two conflicting objectives of the multiobjective model, which aims to find sparse connecting structures with optimal weights and biases. Then, a hybrid encoding-based MOEA/D is used to optimize the multiobjective model. In addition, ensemble learning is embedded into this algorithm to make decisions after multiobjective optimization. Experimental results of several classification and regression applications demonstrate the effectiveness of the proposed MO-SELM.

Computational Deep Intelligence Vision Sensing for Nutrient Content Estimation in Agricultural Automation

This paper presents a novel computational intelligence vision sensing approach to estimate nutrient content in wheat leaves by analyzing color features of the leaves images captured on field with various lighting conditions. We propose the development of deep sparse extreme learning machines (DSELM) fusion and genetic algorithm (GA) to normalize plant images as well as to reduce color variability due to a variation of sunlight intensities. We also apply the DSELM in image segmentation to differentiate wheat leaves from a complex background. In this paper, four moments of color distribution of the leaves images (mean, variance, skewness, and kurtosis) are extracted and utilized as predictors in the nutrient estimation. We combine a number of DSELMs with committee machine and optimize them using the GA to estimate nitrogen content in wheat leaves. The results have shown the superiority of the proposed method in the term of quality and processing speed in all steps, i.e., color normalization, image segmentation, and nutrient prediction, as compared with other existing methods. Note to Practitioners —In order to support precision farming technology and agricultural automation, it is very essential to estimate the nutrient content in plants to prevent over fertilizing that will harm environment. Furthermore, the existing method to determine nutrient content in leaves is destructive as well as time consuming and requires special expertise to operate expensive devices. A number of methods have been developed to estimate nutrient content in leaves nondestructively based on the color features of the leaves. Most of these methods, however, are conducted in a controlled environment with artificial lighting. Such methods are not practical and need various equipment. We propose a low-cost, simple, and accurate technique to estimate nitrogen content in wheat leaves by analyzing RGB color of the leaves images. In this paper, we found that the developed DSELMs fusion has enabled better performance in normalizing images and is faster than other neural network types, i.e., backpropagation-based multilayer perceptron and original extreme learning machine. In image segmentation step, the established DSELM shows good performance to recognize and distinguish wheat leaves from other undesired background images. Furthermore, the developed weighted DSELMs have demonstrated enhanced ability in estimating nutrient content.

Identification of Alzheimer's disease and mild cognitive impairment using multimodal sparse hierarchical extreme learning machine.

Different modalities such as structural MRI, FDG-PET, and CSF have complementary information, which is likely to be very useful for diagnosis of AD and MCI. Therefore, it is possible to develop a more effective and accurate AD/MCI automatic diagnosis method by integrating complementary information of different modalities. In this paper, we propose multi-modal sparse hierarchical extreme leaning machine (MSH-ELM). We used volume and mean intensity extracted from 93 regions of interest (ROIs) as features of MRI and FDG-PET, respectively, and used p-tau, t-tau, and as CSF features. In detail, high-level representation was individually extracted from each of MRI, FDG-PET, and CSF using a stacked sparse extreme learning machine auto-encoder (sELM-AE). Then, another stacked sELM-AE was devised to acquire a joint hierarchical feature representation by fusing the high-level representations obtained from each modality. Finally, we classified joint hierarchical feature representation using a kernel-based extreme learning machine (KELM). The results of MSH-ELM were compared with those of conventional ELM, single kernel support vector machine (SK-SVM), multiple kernel support vector machine (MK-SVM) and stacked auto-encoder (SAE). Performance was evaluated through 10-fold cross-validation. In the classification of AD vs. HC and MCI vs. HC problem, the proposed MSH-ELM method showed mean balanced accuracies of 96.10% and 86.46%, respectively, which is much better than those of competing methods. In summary, the proposed algorithm exhibits consistently better performance than SK-SVM, ELM, MK-SVM and SAE in the two binary classification problems (AD vs. HC and MCI vs. HC).

Fixed-Size Extreme Learning Machines Through Simulated Annealing

Extreme learning machines (ELMs) are an interesting alternative to multilayer perceptrons because ELMs, in practice, require the optimization only of the number of hidden neurons by grid-search with cross-validation. Nevertheless, a large number of hidden neurons obtained after training is a significant drawback for the effective usage of such classifiers. In order to overcome this drawback, we propose a new approach to prune hidden layer neurons and achieve a fixed-size hidden layer without affecting the classifier’s accuracy and even improving it. The main idea is to leave out non-relevant or very similar neurons to others that are not necessary to achieve a simplified, yet accurate, model. In our proposal, differently from previous works based on genetic algorithms and simulated annealing, we choose only a subset of those neurons generated at random to belong to the hidden layer without weight adjustment or increasing of the hidden neurons. In this work, we compare our proposal called simulated annealing for pruned ELM (SAP-ELM) with pruning methods named optimally pruned ELM, genetic algorithms for pruning ELM and sparse Bayesian ELM. On the basis of our experiments, we can state that SAP-ELM is a promising alternative for classification tasks. We highlight that as our proposal achieves fixed-size models, it can help to solve problems where the memory consuming is crucial, such as embeded systems, and helps to control the maximum size that the models must reach after the training process.