Kernel-based Extreme Learning Machine Research Articles

A Clustering-Based Nonlinear Ensemble Approach for Exchange Rates Forecasting

A clustering-based nonlinear ensemble (CNE) learning approach is proposed in this paper to forecast exchange rates. In the proposed CNE learning approach: 1) a self-organizing map neural network is introduced to cluster the in-sample component forecasts; 2) kernel-based extreme learning machine is employed to calculate the in-sample ensemble weights for each cluster; and 3) the corresponding clusters’ in-sample ensemble weights are used for out-of-sample component forecasts to obtain the ensemble forecasts. To illustrate and verify the effectiveness of our proposed model, we test its directional and level forecasting accuracy using four major exchange rates. The out-of-sample forecasting performance results show that the proposed CNE learning approach consistently outperforms the component models and other ensemble learning approaches in terms of the directional forecasting accuracy and the level forecasting accuracy.

A new approach for prediction of the wear loss of PTA surface coatings using artificial neural network and basic, kernel-based, and weighted extreme learning machine

Wear tests are essential in the design of parts intended to work in environments that subject a part to high wear. Wear tests involve high cost and lengthy experiments, and require special test equipment. The use of machine learning algorithms for wear loss quantity predictions is a potentially effective means to eliminate the disadvantages of experimental methods such as cost, labor, and time. In this study, wear loss data of AISI 1020 steel coated by using a plasma transfer arc welding (PTAW) method with FeCrC, FeW, and FeB powders mixed in different ratios were obtained experimentally by some of the researchers in our group. The mechanical properties of the coating layers were detected by microhardness measurements and dry sliding wear tests. The wear tests were performed at three different loads (19.62, 39.24, and 58.86 N) over a sliding distance of 900 m. In this study, models have been developed by using four different machine learning algorithms (an artificial neural network (ANN), extreme learning machine (ELM), kernel-based extreme learning machine (KELM), and weighted extreme learning machine (WELM)) on the data set obtained from the wear test experiments. The R2 value was calculated as 0.9729 in the model designed with WELM, which obtained the best performance [with 11among the models evaluated.

Non-Destructive Detection of Tea Leaf Chlorophyll Content Using Hyperspectral Reflectance and Machine Learning Algorithms.

Tea trees are kept in shaded locations to increase their chlorophyll content, which influences green tea quality. Therefore, monitoring change in chlorophyll content under low light conditions is important for managing tea trees and producing high-quality green tea. Hyperspectral remote sensing is one of the most frequently used methods for estimating chlorophyll content. Numerous studies based on data collected under relatively low-stress conditions and many hyperspectral indices and radiative transfer models show that shade-grown tea performs poorly. The performance of four machine learning algorithms—random forest, support vector machine, deep belief nets, and kernel-based extreme learning machine (KELM)—in evaluating data collected from tea leaves cultivated under different shade treatments was tested. KELM performed best with a root-mean-square error of 8.94 ± 3.05 μg cm−2 and performance to deviation values from 1.70 to 8.04 for the test data. These results suggest that a combination of hyperspectral reflectance and KELM has the potential to trace changes in the chlorophyll content of shaded tea leaves.

Multiclass covert speech classification using extreme learning machine.

The objective of the proposed research is to classify electroencephalography (EEG) data of covert speech words. Six subjects were asked to perform covert speech tasks i.e mental repetition of four different words i.e 'left', 'right', 'up' and 'down'. Fifty trials for each word recorded for every subject. Kernel-based Extreme Learning Machine (kernel ELM) was used for multiclass and binary classification of EEG signals of covert speech words. We achieved a maximum multiclass and binary classification accuracy of (49.77%) and (85.57%) respectively. The kernel ELM achieves significantly higher accuracy compared to some of the most commonly used classification algorithms in Brain-Computer Interfaces (BCIs). Our findings suggested that covert speech EEG signals could be successfully classified using kernel ELM. This research involving the classification of covert speech words potentially leading to real-time silent speech BCI research.

A Flexible High-Level Fusion for an Accurate Human Action Recognition System

Action recognition is a very effective method of computer vision areas. In the last few years, there has been a growing interest in Deep learning networks as the Long Short–Term Memory (LSTM) architectures due to their efficiency in long-term time sequence processing. In the light of these recent events in deep neural networks, there is now considerable concern about the development of an accurate action recognition approach with low complexity. This paper aims to introduce a method for learning depth activity videos based on the LSTM and the classification fusion. The first step consists in extracting compact depth video features. We start with the calculation of Depth Motion Maps (DMM) from each sequence. Then we encode and concatenate contour and texture DMM characteristics using the histogram-of-oriented-gradient and local-binary-patterns descriptors. The second step is the depth video classification based on the naive Bayes fusion approach. Training three classifiers, which are the collaborative representation classifier, the kernel-based extreme learning machine and the LSTM, is done separately to get classification scores. Finally, we fuse the classification score outputs of all classifiers with the naive Bayesian method to get a final predicted label. Our proposed method achieves a significant improvement in the recognition rate compared to previous work that has used Kinect v2 and UTD-MHAD human action datasets.

Identifying groundwater contaminant sources based on a KELM surrogate model together with four heuristic optimization algorithms

Identifying groundwater contaminant sources involves a reverse determination of the source characteristics by monitoring contaminant concentrations in a few observation wells. However, due to the ill-posed nature and high time consumption of identification, an efficient identification process with accurate estimated results is particularly important. To improve the efficiency of identifying groundwater contaminant sources, a kernel-based extreme learning machine was used as a surrogate for the time-consuming simulation model. Four heuristic search algorithms were used to improve the accuracy of the identification results. The proposed approach was tested in both hypothetical and actual cases. The conclusions are: 1. By forward and backward calculation of the surrogate model, the time cost of identifying groundwater sources can be reduced significantly; 2. When a traditional genetic algorithm and a particle swarm optimization algorithm are combined with quantum computing, computational efficiency and accuracy are both improved; and 3. By using various search algorithms to identify unknown contaminant sources in the actual case, the range of release histories of each contaminant source can be obtained, decreasing the ill-posed nature of the identification result obtained by a single algorithm and improving the reliability of the identification results.

A hybrid framework for carbon trading price forecasting: The role of multiple influence factor

Current carbon trading price forecasting research ignores the significance of multiple influence factors, feature selection, preprocessing of the carbon price and its exogenous variables, multi-objective intelligence optimization and kernel-based models in terms of improving prediction validity, which may lead to an undesirable forecasting performance. As a result, a novel hybrid forecasting framework that considers multiple influence factors is developed for carbon price forecasting, which enjoys the merits of some new algorithms and successfully overcomes the challenges of multiple influence factor-based carbon price forecasting. Specifically, both the original carbon price and its exogenous variables benefit from advanced data preprocessing technology. Moreover, feature selection is proposed to determine the optimal feature for modeling the carbon price. Meanwhile, to overcome the limitations of the extreme learning machine and traditional artificial neural network models and obtain desirable, accurate and stable forecasts, the optimal kernel-based extreme learning machine model with good generalizability and stability is developed, grounded in a newly proposed optimizer named the multi-objective chaotic sine cosine algorithm. Experiments, analyses and discussion of the results prove that the developed framework outperforms all compared models which can be as an effective forecasting tool for forecasting and management of the carbon trading market.

A composite framework coupling multiple feature selection, compound prediction models and novel hybrid swarm optimizer-based synchronization optimization strategy for multi-step ahead short-term wind speed forecasting

Accurate wind speed prediction plays a vital role in power system in terms of rational dispatching and safe operation. For this purpose, a novel composite framework integrating time varying filter-based empirical mode decomposition (TVF-EMD), fuzzy entropy (FE) theory, singular spectrum analysis (SSA), phase space reconstruction (PSR), compound prediction models adopting kernel-based extreme learning machine (KELM) and convolutional long short-term memory network (ConvLSTM) as well as mutation and hierarchy-based hybrid optimization algorithm, is proposed in this paper. Among the supplementary strategies, TVF-EMD, FE and SSA are employed to achieve non-stationary raw series attenuation, aggregation for approximate IMFs as well as separation of dominant and residuary ingredients from the aggregated IMFs, respectively. Besides, parameters of PSR and KELM as well as wrapper method-based feature selection (FS) for input combination are synchronously optimized by the newly developed swarm optimizer integrating Harris hawks optimization (HHO) and grey wolf optimizer (GWO) with mutation operator and hierarchy strategy. Meanwhile, such hybrid structure is adopted to predict the preprocessed high-frequency components, while the remaining component is predicted by ConvLSTM cells-based deep learning network. Subsequently, the ultimate forecasting results of the raw wind speed are calculated by superimposing the predicted values of all components. Four datasets collected from various sites with two different time intervals and nine relevant contrastive models are carried out to evaluate the proposed approach, where the corresponding results demonstrate that: (1) data preprocessing strategy applying TVF-EMD and FE theory can significantly reduce the time consumption of the entire model without decreasing forecasting performance; (2) SSA-based dominant ingredients extraction can further improve the forecasting capability of combined model; (3) the proposed MHHOGWO can synchronously accomplish parameters optimization and FS effectively, thus improving the forecasting effectiveness of the entire model significantly; (4) the proposed compound prediction models based on KELM and ConvLSTM can exert the capabilities of each model adequately as well as ulteriorly reducing computational requirements.

Soil Moisture Inversion Via Semiempirical and Machine Learning Methods With Full-Polarization Radarsat-2 and Polarimetric Target Decomposition Data: A Comparative Study

In this article, surface soil moisture was retrieved from Radarsat-2 and polarimetric target decomposition data by using semiempirical models and machine learning methods. The semiempirical models and machine learning techniques employed were Oh (1992), Dubois (1995), Oh (2004) and Generalized Regression Neural Network (GRNN), Least Squares - Support Vector Machine (LS-SVM), Extreme Learning Machine (ELM), Kernel based Extreme Learning Machine (KELM), Adaptive Network based Fuzzy Inference System (ANFIS), respectively. In addition, Yamaguchi, van Zyl, Freeman-Durden, H/A/α and Cloude polarimetric target decomposition methods were used in this study. For soil moisture inversion, firstly, preprocessing was applied to the Radarsat-2 image of two different dates with bare and moderately vegetated soil. Then, sigma nought coefficients and the polarimetric decomposition components were extracted as feature vector from preprocessed SAR image pixels corresponding to ground measured points. Lastly, sigma nought coefficients were used in semiempirical inversion models, and sigma nought coefficients and polarimetric decomposition components were used as input to machine learning methods. The best accuracy results for semiempirical models were 13.01 vol. % and 17.91 vol. % Root Mean Square Error (RMSE) for bare and moderately vegetated soil, respectively. The best accuracy for machine learning techniques were 4.04 vol. % and 2.72 vol. % RMSE for two dates, respectively. The results indicated that the machine learning techniques performed much better than the semiempirical models.

A Novel Kernel-based Extreme Learning Machine with Incremental Hidden Layer Nodes

Extreme learning machine (ELM) is widely used in various fields because of its advantages such as short training time and good generalization performance. The input weights and bias of hidden layer of traditional ELM are generated randomly, and the number of hidden layer nodes is determined by artificial experience. Only by adjusting parameters manually can an appropriate network structure be found. This training method is complex and time-consuming, which increases the workload of workers. To solve this problem, the incremental extreme learning machine (I-ELM) is used to determine the appropriate number of hidden layer nodes and construct a compact network structure in this paper. At the same time, a new hidden layer activation function STR is proposed, which avoids the disadvantages of incomplete output information of hidden layer due to uneven distribution of sample data. The proposed algorithm is evaluated by public data sets and applied to the classification of superheat degree (SD) in aluminum electrolysis industry. The experimental results show that STR activation function has a good learning speed, and the proposed algorithm is superior to the existing SD identification algorithm in terms of accuracy and robustness.

Random Subspace Ensemble With Enhanced Feature for Hyperspectral Image Classification

In this letter, we propose a new hyperspectral image (HSI) classification approach, called the random subspace ensemble with enhanced feature (RSE-EF), which trains several individual classifiers with enhanced spatial information. The proposed approach aims to address two common issues: the curses of the imbalanced training samples and high feature-to-instance ratio. Specifically, we first propose a similar-neighboring-sample-search (SNSS) method to address the issue of imbalanced training samples. Afterward, we generate the enhanced random subspaces (ERSs) that possess relatively lower dimensionality and more distinctive information compared with the original random subspaces (RSs) so as to alleviate the curse of high feature-to-instance ratio more effectively. Furthermore, a shallow neural network kernel-based extreme learning machine (KELM) is applied to the RSE-EF to classify image pixels. Experimental results on two public hyperspectral data sets illustrate that the proposed RSE-EF approach outperforms the state-of-the-art HSI classification counterparts.

Improving Human Action Recognition Using Hierarchical Features And Multiple Classifier Ensembles

Abstract This paper presents a simple, fast and efficacious system to promote the human action classification outcome using the depth action sequences. Firstly, the motion history images (MHIs) and static history images (SHIs) are created from the front (XOY), side (YOZ) and top (XOZ) projected scenes of each depth sequence in a 3D Euclidean space through engaging the 3D Motion Trail Model (3DMTM). Then, the Local Binary Patterns (LBPs) algorithm is operated on the MHIs and SHIs to learn motion and static hierarchical features to represent the action sequence. The motion and static hierarchical feature vectors are then fed into a classifier ensemble to classify action classes, where the ensemble comprises of two classifiers. Thus, each ensemble includes a pair of Kernel-based Extreme Learning Machine (KELM) or ${\mathrm{l}}_{\mathrm{2}}$-regularized Collaborative Representation Classifier (${\mathrm{l}}_{\mathrm{2}}$-CRC) or Multi-class Support Vector Machine. To extensively assess the framework, we perform experiments on a couple of standard available datasets such as MSR-Action3D, UTD-MHAD and DHA. Experimental consequences demonstrate that the proposed approach gains a state-of-the-art recognition performance in comparison with other available approaches. Several statistical measurements on recognition results also indicate that the method achieves superiority when the hierarchical features are adopted with the KELM ensemble. In addition, to ensure real-time processing capability of the algorithm, the running time of major components is investigated. Based on machine dependency of the running time, the computational complexity of the system is also shown and compared with other methods. Experimental results and evaluation of the computational time and complexity reflect real-time compatibility and feasibility of the proposed system.

Spectral-Spatial Hyperspectral Image Classification with Superpixel Pattern and Extreme Learning Machine

Spectral-spatial classification of hyperspectral images (HSIs) has recently attracted great attention in the research domain of remote sensing. It is well-known that, in remote sensing applications, spectral features are the fundamental information and spatial patterns provide the complementary information. With both spectral features and spatial patterns, hyperspectral image (HSI) applications can be fully explored and the classification performance can be greatly improved. In reality, spatial patterns can be extracted to represent a line, a clustering of points or image texture, which denote the local or global spatial characteristic of HSIs. In this paper, we propose a spectral-spatial HSI classification model based on superpixel pattern (SP) and kernel based extreme learning machine (KELM), called SP-KELM, to identify the land covers of pixels in HSIs. In the proposed SP-KELM model, superpixel pattern features are extracted by an advanced principal component analysis (PCA), which is based on superpixel segmentation in HSIs and used to denote spatial information. The KELM method is then employed to be a classifier in the proposed spectral-spatial model with both the original spectral features and the extracted spatial pattern features. Experimental results on three publicly available HSI datasets verify the effectiveness of the proposed SP-KELM model, with the performance improvement of 10% over the spectral approaches.

Combining ASNARO-2 XSAR HH and Sentinel-1 C-SAR VH/VV Polarization Data for Improved Crop Mapping

The Advanced Satellite with New system ARchitecture for Observation-2 (ASNARO-2), which carries the X-band Synthetic Aperture Radar (XSAR), was launched on 17 January 2018 and is expected to be used to supplement data provided by larger satellites. Land cover classification is one of the most common applications of remote sensing, and the results provide a reliable resource for agricultural field management and estimating potential harvests. This paper describes the results of the first experiments in which ASNARO-2 XSAR data were applied for agricultural crop classification. In previous studies, Sentinel-1 C-SAR data have been widely utilized to identify crop types. Comparisons between ASNARO-2 XSAR and Sentinel-1 C-SAR using data obtained in June and August 2018 were conducted to identify five crop types (beans, beetroot, maize, potato, and winter wheat), and the combination of these data was also tested. To assess the potential for accurate crop classification, some radar vegetation indices were calculated from the backscattering coefficients for two dates. In addition, the potential of each type of SAR data was evaluated using four popular supervised learning models: Support vector machine (SVM), random forest (RF), multilayer feedforward neural network (FNN), and kernel-based extreme learning machine (KELM). The combination of ASNARO-2 XSAR and Sentinel-1 C-SAR data was effective, and overall classification accuracies of 85.4 ± 1.8% were achieved using SVM.

Deep motion templates and extreme learning machine for sign language recognition

Sign language is a visual language used by persons with hearing and speech impairment to communicate through fingerspellings and body gestures. This paper proposes a framework for automatic sign language recognition without the need of hand segmentation. The proposed method first generates three different types of motion templates: motion history image, dynamic image and our proposed RGB motion image. These motion templates are used to fine-tune three ConvNets trained on ImageNet dataset. Fine-tuning avoids learning all the parameters from scratch, leading to faster network convergence even with a small number of training samples. For combining the output of three ConvNets, we propose a fusion technique based on Kernel-based extreme learning machine (KELM). The features extracted from the last fully connected layer of trained ConvNets are used to train three KELMs, and the final class label is predicted by averaging their scores. The proposed approach is validated on a number of publicly available sign language as well as human action recognition datasets, and state-of-the-art results are achieved. Finally, an Indian sign language dataset is also collected using a thermal camera. The experimental results obtained show that our ConvNet-based deep features along with proposed KELM-based fusion are robust for any type of human motion recognition.

A Kernel-Based Extreme Learning Machine Framework for Classification of Hyperspectral Images Using Active Learning

The rapid development of advanced remote sensing technology with multichannel imaging sensors has increased its potential opportunity in the utilization of hyperspectral data for various applications. For supervised classification of hyperspectral data, obtaining suitable training set is essential for ensuring good performance. However, obtaining labeled training sample is often difficult, expensive, and time consuming in hyperspectral images (HSIs) and other image analysis applications. To overcome this problem, active learning (AL) technique plays a crucial role in the image analysis framework. As per literature, classification of HSI using AL has been focused in terms of accuracy, but learning rate in terms of computation time has not been focused yet. In this paper, multiview-based AL technique has been integrated with kernel-based extreme learning machine (KELM) classifier. Further, we have compared our approach with popularly used kernel-based support vector machine (KSVM). To validate our study, experiments were conducted on two Hyperspectral Images: Kennedy Space Centre (KSC) and Botswana (BOT) datasets. The proposed approach (KELM-AL) achieved the classification accuracy up to 91.15% in KSC dataset while 95.02% in case of BOT dataset with computation time of 149.78 s and 104.98 s, respectively. While KSVM-AL achieved the classification accuracy up to 91.59% in KSC dataset while 95.96% in case of BOT dataset with computation time of 7532.25 s and 6863.60 s, respectively. This shows that classification accuracy obtained by KELM-AL is comparable to KSVM-AL approach but significantly reduces the computational time. Thus, the proposed system shows the promising results with adequate classification accuracy while reducing the computation time drastically.

ELM and KELM based software defect prediction using feature selection techniques

Context: Software defect prediction (SDP) models help in delivering a dependable and a genuine product to the clients. However, the performance of these models is affected by the presence of irrelevant features in the datasets. This problem is addressed by feature selection techniques.Objectives: (1) To determine the performance of feature selection based classification models in the context of software defect prediction, and (2) To determine if the removal of insignificant features makes a significant difference in the performance of the SDP models.Method: SDP models are built using two classifiers – Extreme learning machine (ELM) and Kernel based extreme learning machine (KELM) based on five wrapper and seven filter based feature selection techniques. Experiments are performed using seven datasets from the PROMISE repository. Testing accuracy is used for performance comparison of the feature selection based ELM and KELM defect classification models.Results: (1) ELM based classifiers achieved a higher testing accuracy with wrapper based feature selection methods while KELM classifiers performed better with filter based methods. (2) It is also found that even after eliminating over 85 percent of the attributes from the original software project data, the classification performance of the models is comparable before and after removing the insignificant features in most of the cases and it improved in very few experiments.Conclusion: With respect to the feature selection based defect classification, the performance of ELM and KELM based models is better with wrapper and filter based methods, respectively. Overall, a dimensionally reduced space does not significantly affect the prediction performance of the SDP models. In a way, it is indicated that the feature subsets obtained after removing the insignificant software metrics provide more significance to the output class.

Final optics damage online inspection in high power laser facility*

For the laser-induced damage (LID) in large-aperture final optics, we present a novel approach of damage online inspection and its experimental system, which solves two problems: classification of true and false LID and size measurement of the LID. We first analyze the imaging principle of the experimental system for the true and false damage sites, then use kernel-based extreme learning machine (K-ELM) to distinguish them, and finally propose hierarchical kernel extreme learning machine (HK-ELM) to predict the damage size. The experimental results show that the classification accuracy is higher than 95%, the mean relative error of the predicted LID size is within 10%. So the proposed method meets the technical requirements for the damage online inspection.

Parcel-Based Crop Classification Using Multi-Temporal TerraSAR-X Dual Polarimetric Data

Cropland maps are useful for the management of agricultural fields and the estimation of harvest yield. Some local governments have documented field properties, including crop type and location, based on site investigations. This process, which is generally done manually, is labor-intensive, and remote-sensing techniques can be used as alternatives. In this study, eight crop types (beans, beetroot, grass, maize, potatoes, squash, winter wheat, and yams) were identified using gamma naught values and polarimetric parameters calculated from TerraSAR-X (or TanDEM-X) dual-polarimetric (HH/VV) data. Three indices (difference (D-type), simple ratio (SR), and normalized difference (ND)) were calculated using gamma naught values and m-chi decomposition parameters and were evaluated in terms of crop classification. We also evaluated the classification accuracy of four widely used machine-learning algorithms (kernel-based extreme learning machine, support vector machine, multilayer feedforward neural network (FNN), and random forest) and two multiple-kernel methods (multiple kernel extreme learning machine (MKELM) and multiple kernel learning (MKL)). MKL performed best, achieving an overall accuracy of 92.1%, and proved useful for the identification of crops with small sample sizes. The difference (raw or normalized) between double-bounce scattering and odd-bounce scattering helped to improve the identification of squash and yams fields.

Predicting the concentration of indoor culturable fungi using a kernel-based extreme learning machine (K-ELM)

ABSTRACTIndoor fungal is of great significance for human health. The kernel-based extreme learning machine is employed to determine the most important parameters for predicting the concentration of indoor culturable fungi (ICF). For model training and statistical analysis, parameters that contained indoor or outdoor PM10 and PM2.5, RH, Temperature, CO2 and ICF were measured in 85 residential buildings of Baoding, China, from November 2016 to March 2017. The variable selection process contains four different cases to identify the optimal input combination. The results indicate that root mean square error of the optimal input combinations can be improved 5.6% from 1 to 2 input variables, while that could be only improved 1.9% from 2 to 3 input variables. However, considering both precision and simplicity, the combination of indoor PM10 and RH provides a more suitable selection for predicting the ICF.