Reduced Kernel Extreme Learning Machine Research Articles

Multilayer Online Sequential Reduced Kernel Extreme Learning Machine-Based Modeling for Time-Varying Distributed Parameter Systems.

A significant number of industrial dynamic processes belong to time-varying distributed parameter systems (DPSs). To develop an accurate approximation model for these systems, it is critical to capture their time-varying behavior and strong nonlinearity. In this article, a multilayer online sequential reduced kernel extreme learning machine (ML-OSRKELM)-based online spatiotemporal modeling approach is developed for such DPSs. First, ML-OSRKELM stacks multiple online sequential reduced kernel extreme learning machine autoencoders (OSRKELM-AEs) to create a deep network, which can translate the spatiotemporal domain into a low-dimensional time domain. Then, an online sequential reduced kernel extreme learning machine (OS-RKELM) is employed to construct a dynamic temporal model. Finally, after obtaining time coefficients from the time domain, OS-RKELM is also used to reconstruct the original spatiotemporal domain. By using the kernel trick and the support vector selection strategy, the proposed method can remove redundant information while maintaining satisfactory nonlinear learning performance. Furthermore, the designed sequential update scheme can update the model parameters with real-time data, which makes it a promising method for capturing time-varying dynamics. Experiments and simulations on a lithium-ion battery's thermal process confirm the excellent performance and validity of the proposed model.

A data-driven method for SSCI identification using synchrophasor measurements

In recent years, subsynchronous control interaction (SSCI) frequently occurred due to the rapid development of inverter based resources. Online monitoring and identifying SSCI are thus of great importance for the safe operation of power systems. This paper proposes a data-driven method for SSCI identification using synchrophasors generated by phasor measurement units (PMUs). The challenges of this work mainly result from two factors: 1) SSCI may simultaneously involve a sub-synchronous oscillation (Sub-SO) mode and a super-synchronous oscillation (Sup-SO) mode. The spectra of these two modes would be aliased due to the limited reporting rate of PMUs. 2) The reporting filter in PMUs significantly weakens the oscillation signal as the frequency of SSCI is generally far away from the nominal frequency. To tackle these challenges, the influence of Sub-SO and Sup-SO on spectra of synchrophasors is analyzed first. It is revealed that the positive and negative spectra are linearly correlated with Sub-SO and Sup-SO modes. Several features are then extracted via the analysis of the spectra which can reflect the SSCI features and a data-driven identification method is further proposed through a fast reduced kernel extreme learning machine (RKELM) model. Compared with the traditional classification techniques, the RKELM algorithm based on kernel function has stronger generalization performance and higher accuracy while ensuring shorter training time. This significantly improves the accuracy and efficiency of SSCI identification, especially under noise conditions. Both simulations and field tests demonstrate the effectiveness and usefulness of the proposed method.

A Novel Reformed Reduced Kernel Extreme Learning Machine with RELIEF-F for Classification.

With the exponential growth of the Internet population, scientists and researchers face the large-scale data for processing. However, the traditional algorithms, due to their complex computation, are not suitable for the large-scale data, although they play a vital role in dealing with large-scale data for classification and regression. One of these variants, which is called Reduced Kernel Extreme Learning Machine (Reduced-KELM), is widely used in the classification task and attracts attention from researchers due to its superior performance. However, it still has limitations, such as instability of prediction because of the random selection and the redundant training samples and features because of large-scaled input data. This study proposes a novel model called Reformed Reduced Kernel Extreme Learning Machine with RELIEF-F (R-RKELM) for human activity recognition. RELIEF-F is applied to discard the attributes of samples with the negative values in the weights. A new sample selection approach, which is used to further reduce training samples and to replace the random selection part of Reduced-KELM, solves the unstable classification problem in the conventional Reduced-KELM and computation complexity problem. According to experimental results and statistical analysis, our proposed model obtains the best classification performances for human activity data sets than those of the baseline model, with an accuracy of 92.87 % for HAPT, 92.81 % for HARUS, and 86.92 % for Smartphone, respectively.

Radar HRRP Target Recognition Based on Dynamic Learning with Limited Training Data

For high-resolution range profile (HRRP)-based radar automatic target recognition (RATR), adequate training data are required to characterize a target signature effectively and get good recognition performance. However, collecting enough training data involving HRRP samples from each target orientation is hard. To tackle the HRRP-based RATR task with limited training data, a novel dynamic learning strategy is proposed based on the single-hidden layer feedforward network (SLFN) with an assistant classifier. In the offline training phase, the training data are used for pretraining the SLFN using a reduced kernel extreme learning machine (RKELM). In the online classification phase, the collected test data are first labeled by fusing the recognition results of the current SLFN and assistant classifier. Then the test samples with reliable pseudolabels are used as additional training data to update the parameters of SLFN with the online sequential RKELM (OS-RKELM). Moreover, to improve the accuracy of label estimation for test data, a novel semi-supervised learning method named constraint propagation-based label propagation (CPLP) was developed as an assistant classifier. The proposed method dynamically accumulates knowledge from training and test data through online learning, thereby reinforcing performance of the RATR system with limited training data. Experiments conducted on the simulated HRRP data from 10 civilian vehicles and real HRRP data from three military vehicles demonstrated the effectiveness of the proposed method when the training data are limited.

Group reduced kernel extreme learning machine for fault diagnosis of aircraft engine

The original kernel extreme learning machine (KELM) employs all training samples to construct hidden layer, thus avoiding the performance fluctuations caused by the ELM randomly assigning weights. However, excessive nodes will inevitably lead to structural redundancy, which hinders its application in systems with high real-time performance requirements but limited onboard storage and computing capacity. Considering the well interpretability of sparse learning, this study introduces the group sparse structure for KELM to resolve its limitation of structural redundancy. Specifically, the proposed novel method introduces a special norm to reformulate the dual optimization problem of KELM to realize group sparse structure in output weights. As a result, nodes with large weights can be selected as the significant nodes, while nodes with small weights will be regarded as the redundant nodes and neglected directly. In addition, we have also devised an alternating iterative optimization algorithm and deduced the complete proof of convergence to solve the non-smoothness optimization problem in proposed method. Then, the validity and feasibility of the proposed method are verified by extensive experiments on benchmark datasets. More importantly, tests of fault diagnosis for an aircraft engine show that the proposed approach can maintain the competitive recognition performance with much faster testing speed.

A clustering based ensemble of weighted kernelized extreme learning machine for class imbalance learning

Most of the classification problems in real word suffer from the problem of class skewness. Various methods are required to handle the classification of problems having class skewness. Several methods have been proposed for this purpose which includes some ensemble methods. Most of such ensemble methods targets on creating balanced sub-problems like UnderBagging based kernelized Extreme Learning Machine (UBKELM), UnderBagging based Reduced Kernel Extreme Learning Machine (UBRKELM), Random Undersampling Boost (RUSBoost), EasyEnsemble and BalanceCascade. This work suggests that apart from class skewness, there are many other factors like class overlapping, length of decision boundary and the number of probability distributions present in the problem which are responsible for performance degradation of a classifier. This work proposes an ensemble method which decomposes a complex imbalanced problem into simpler sub-problems, solves these sub-problems using cost sensitive classifiers and then combines the results of each classifier using voting methods. A classification problem can have mixture distribution, the complexity of the problem increases with increase in the number of probability distributions present in it. The proposed problem decomposition method try to create less complex sub-problems by decreasing the number of distributions present in the sub-problems than the original problem. The proposed method uses a clustering evaluation algorithm to find the optimal number of sub-problems. For decomposing an imbalanced classification problem into sub-problems this work uses fuzzy clustering algorithm (FCM) which is a soft clustering algorithm, i.e. the overlapping between the sub-problems depends on the selected value of threshold (TH) parameter in the FCM algorithm. The sub-problems created in this work may or may not be balanced, so Weighted Kernelized Extreme Learning Machine (WKELM) is used to create the classifiers for these sub-problems. The final prediction of the ensemble of these classifiers is determined using soft voting and majority voting. The proposed method is evaluated on 38 benchmark binary class imbalanced datasets downloaded from KEEL dataset repository. The obtained results show that the proposed method outperforms other state of the art methods of imbalance classification. The Wilcoxon signed rank test is performed to show the significant improvement in results.

A Hybrid Short-Term Wind Power Prediction Model Combining Data Processing, Multiple Parameters Optimization and Multi-Intelligent Models Apportion Strategy

As the wind power dates have strong volatility and intermittency, it is difficult to meet the safety and stability of power system operation. To make full use of the characteristics of wind power data and further improve wind power forecasting performance, an innovative hybrid framework is proposed to response to the above challenges, of which the non-stationary weakening technique, sample entropy (SE)-based prediction model allocation, optimized reduced kernel extreme learning machine (RKELM) and a novel deep learning network are integrated befittingly. Initially, the original wind power sequence is decomposed into multiple sub-sequences by empirical wavelet transform (EWT) adaptively, after which SE is employed to analyse the high and low components of sub-sequences. Then, a simplified gate recurrent unit network (SGRU) to achieve the prediction of the component with low SE-value. Besides, for the components with high SE-values, the multiple parameters optimization is implemented for RKELM based on slime mould algorithm (SMA), which possesses superior convergence speed and precision. Ultimately, the final prediction results can be obtained by accumulating the predicted values of the multiple models. In the experiment phase, two datasets are adopted to verify the predictive ability of the proposed hybrid EWT-SMA-RKELM-SGRU model, where the sufficient results indicate that the proposed model has a superior performance.

ELM-Kernel and reduced kernel ELM based watermarking scheme

With high popularity of Internet, digital data or information can be shared at much higher speed, with less expenses and efforts. This creates new opportunities like e-advertisement, web publishing, audio and video deliveries but these opportunities create many new threats also. An important issue is the copyright protection of the content. Digital watermarking is one of the approach gaining interest to achieve the security of the content. For copyright protection, the blind digital watermarking schemes with the new machine learning technique ELM-Kernel(KELM) and Reduced Kernel Extreme Learning Machine(RKELM) plus Integer Wavelet Transformation(IWT) and Singular Value Decomposition(SVD) have been proposed. The scheme is composed of three steps, namely, watermark embedding, machine learning phase and finally an extraction of watermark. In the embedding phase, the cover image is transformed into IWT domain, then the HL sub-band has been divided into 4 × 4 non-overlapping blocks. SVD is applied to each block to get their singular values then these values have been modified by using the Cox’s formula to embed one bit of watermark in one block and a watermarked image is then obtained. During the machine training phase, an ELM-Kernel(KELM) and Reduced Kernel Extreme Learning Machine(RKELM) is trained separately to memorize the relationship between the coefficients of watermarked image and the original cover image. During extraction phase, an oblivious phase, where the original image is not required and the trained machine is used to extract the watermark from the watermarked image. This algorithm aims to minimize the value of BER in order to assess the robustness of the algorithm. Results show that value of BER is very small even watermarked images are subject to attacks like noise, cropping, sharpening, rotation and blurring. The extracted watermarks are shown in the results which has great similarity with the original watermark. Performance of the proposed method is analyzed by the comparative study with some of the existing methods and it demonstrates that the proposed method outperforms these methods in terms of robustness.

Hyperspectral image classification based on multiple reduced kernel extreme learning machine

This paper presents an efficient hyperspectral images classification method based on multiple reduced kernel extreme learning machine (MRKELM). The MRKELM model is developed on the basis of the multiple kernel leaning method and the reduced kernel extreme learning machine method. In the presented MRKELM, the kernel function are not fixed anymore, multiple kernels are adaptively trained as a hybrid kernel and the optimal kernel combination weights are jointly optimized. Finally, two simulation examples, classification of benchmark datasets and classification of hyperspectral images including Indian Pines, University of Pavia, and Salinas respectively, are used testify the performance of the proposed MRKELM method.

Regularization incremental extreme learning machine with random reduced kernel for regression

Abstract For regression tasks, the existing extreme learning machine (ELM) and kernel extreme learning machine (KELM) algorithms exhibit singularity and over-fitting problems when the number of training samples is less than the number of hidden layer neurons. To overcome these shortcomings, this paper introduces random reduction kernel and regularization parameters, and the regularization incremental extreme learning machine with random reduced kernel (RKRIELM) algorithm is proposed. RKRIELM combines the kernel function and incremental extreme learning machine (I-ELM) to avoid randomness, thereby solving the singularity problem when the number of initial training samples of the ELM is less than the number of hidden layer neurons. Moreover, it uses the number of hidden layer neurons as the precondition for the loop ending of the training algorithm. Additionally, the regularization parameter is used to reduce the risk of over-fitting. Regression experiments were conducted for evaluating the proposed method, ELM, KELM, reduced kernel extreme learning machine, rotation forest selective ensemble extreme learning machine, reduced support vector regression, and gray wolf optimization support vector regression with standard data sets. The results indicate that the proposed method has a lower prediction error and better training efficiency than the other algorithms in most cases.

Multiclass power quality events classification using variational mode decomposition with fast reduced kernel extreme learning machine‐based feature selection

In this study, a modern adaptive signal processing technique called variational mode decomposition (VMD) has been used for power quality (PQ) events detection. Numerous single, as well as multiple PQ events, are simulated according to IEEE std. 1159-2009 and laboratory experimental signals are collected and passed through the VMD algorithm. VMD decomposes the signal into different modes and from these modes, different features have been extracted. To reduce the dimension of the feature set Fischer linear discriminant analysis (FDA) has been used. As a new contribution to the literature, VMD aided FDA-based feature selection with reduced kernel extreme learning machine technique has been used for detection and classification of multiple PQ disturbances. The performance of the proposed combined technique shows higher classification accuracy while classifying multiple PQ disturbances and the results are comparable with many existing methods.

Data decomposition based fast reduced kernel extreme learning machine for currency exchange rate forecasting and trend analysis

In this paper, we propose a hybrid forecasting model that combines Empirical Mode Decomposition (EMD) with fast reduced kernel Extreme Learning Machine (KELM) for day ahead foreign currency exchange rate forecasting. EMD is an efficient method for nonlinear data decomposition in such a noisy environment and the purpose is to find important components in terms of Intrinsic Mode Functions (IMFs) by which the nonlinear time series is converted into stationary time series by making the data smoother and simpler for analysis. The average IMFs decomposed from EMD (AEMD) are hybridized with fast KELM named as AEMD-KELM for producing a more accurate forecast. The experimental results using AEMD-KELM method for seven currency exchange rates like CAD/HKD, CAD/CNY, CAD/USD, CAD/BRL, CAD/JPY, EUR/USD, and GBP/USD provide superior prediction and trend analysis in comparison with EMD based ELM (EMD-ELM) approaches. Further currency exchange rate movement trends are used for generating trading signals like buy, sell or hold.

Dual reduced kernel extreme learning machine for aero-engine fault diagnosis

In order to improve the sparsity of kernel-based extreme learning machine (KELM), this paper proposed a novel method named dual reduced kernel extreme learning machine (DR-KELM). The proposed algorithm incorporates traditional greedy forward learning algorithm into backward learning algorithm to gain more sparsity and enhance testing time further. Compared to original KELM, the proposed method produces satisfactory performance of pattern recognition with fewer nodes, and reduces diagnostic consuming time from the tests on benchmark dataset. The DR-KELM application to aero-engine fault diagnosis also demonstrates its superior performance with more sparse structure.

Kernel fusion based extreme learning machine for cross-location activity recognition

Fixed placements of inertial sensors have been utilized by previous human activity recognition algorithms to train the classifier. However, the distribution of sensor data is seriously affected by the sensor placement. The performance will be degraded when the model trained on one placement is used in others. In order to tackle this problem, a fast and robust human activity recognition model called TransM-RKELM (Transfer learning mixed and reduced kernel Extreme Learning Machine) is proposed in this paper; It uses a kernel fusion method to reduce the influence by the choice of kernel function and the reduced kernel is utilized to reduce the computational cost. After realizing initial activity recognition model by mixed and reduced kernel extreme learning model (M-RKELM), in the online phase M-RKELM is utilized to classify the activity and adapt the model to new locations based on high confident recognition results in real time. Experimental results show that the proposed model can adapt the classifier to new sensor locations quickly and obtain good recognition performance.

A Fast Reduced Kernel Extreme Learning Machine

In this paper, we present a fast and accurate kernel-based supervised algorithm referred to as the Reduced Kernel Extreme Learning Machine (RKELM). In contrast to the work on Support Vector Machine (SVM) or Least Square SVM (LS-SVM), which identifies the support vectors or weight vectors iteratively, the proposed RKELM randomly selects a subset of the available data samples as support vectors (or mapping samples). By avoiding the iterative steps of SVM, significant cost savings in the training process can be readily attained, especially on Big datasets. RKELM is established based on the rigorous proof of universal learning involving reduced kernel-based SLFN. In particular, we prove that RKELM can approximate any nonlinear functions accurately under the condition of support vectors sufficiency. Experimental results on a wide variety of real world small instance size and large instance size applications in the context of binary classification, multi-class problem and regression are then reported to show that RKELM can perform at competitive level of generalized performance as the SVM/LS-SVM at only a fraction of the computational effort incurred.

Online sequential reduced kernel extreme learning machine

In this paper, we present an Online Sequential Reduced Kernel Extreme Learning Machine (OS-RKELM). In OS-RKELM, only a small part of the instances in the original training samples is employed for training the kernel neurons, while the output weights are attained analytically. Similar to the Online Sequential Extreme Learning Machine (OS-ELM), OS-RKELM learns data samples in a chunk-by-chunk or one-by-one mode and does not require an archival of the data sample once it has been learned. OS-RKELM also contains few control parameters, thus avoiding the need for cumbersome fine-tuning of the algorithm. OS-RKELM supports a widespread types of kernels as hidden neurons and is capable of addressing the singular problem that arises when the initial training samples are smaller than the neuron size. A comprehensive performance evaluation of the OS-RKELM against other state-of-the-art sequential learning algorithms, including OS-ELM, Large-scale Active Support Vector Machine (LASVM) and Budgeted Stochastic Gradient Descent Support Vector Machine (BSGD) using popular time series, regression and classification benchmarks have been conducted. Experimental results obtained indicate that the proposed OS-RKELM showcases improved prediction accuracy and efficiency over the OS-ELM, LASVM and BSGD in many cases.

MARK-ELM: Application of a novel Multiple Kernel Learning framework for improving the robustness of Network Intrusion Detection

Detection of cyber-based attacks on computer networks continues to be a relevant and challenging area of research. Daily reports of incidents appear in public media including major ex-filtrations of data for the purposes of stealing identities, credit card numbers, and intellectual property as well as to take control of network resources. Methods used by attackers constantly change in order to defeat techniques employed by information technology (IT) teams intended to discover or block intrusions. “Zero Day” attacks whose “signatures” are not yet in IT databases are continually being uncovered. Machine learning approaches have been widely used to increase the effectiveness of intrusion detection platforms. While some machine learning techniques are effective at detecting certain types of attacks, there are no known methods that can be applied universally and achieve consistent results for multiple attack types. The focus of our research is the development of a framework that combines the outputs of multiple learners in order to improve the efficacy of network intrusion on data that contains instances of multiple classes of attacks. We have chosen the Extreme Learning Machine (ELM) as the core learning algorithm due to recent research that suggests that ELMs are straightforward to implement, computationally efficient and have excellent learning performance characteristics on par with the Support Vector Machine (SVM), one of the most widely used and best performing machine learning platforms (Liu, Gao, & Li, 2012). We introduce the novel Multiple Adaptive Reduced Kernel Extreme Learning Machine (MARK-ELM) which combines Multiple Kernel Boosting (Xia & Hoi, 2013) with the Multiple Classification Reduced Kernel ELM (Deng, Zheng, & Zhang, 2013). We tested this approach on several machine learning datasets as well as the KDD Cup 99 (Hettich & Bay, 1999) intrusion detection dataset. Our results indicate that MARK-ELM works well for the majority of University of California, Irvine (UCI) Machine Learning Repository small datasets and is scalable for larger datasets. For UCI datasets we achieved performance similar to the MKBoost Support Vector Machine (SVM) approach. In our experiments we demonstrate that MARK-ELM achieves superior detection rates and much lower false alarm rates than other approaches on intrusion detection data.

Cross-person activity recognition using reduced kernel extreme learning machine

Activity recognition based on mobile embedded accelerometer is very important for developing human-centric pervasive applications such as healthcare, personalized recommendation and so on. However, the distribution of accelerometer data is heavily affected by varying users. The performance will degrade when the model trained on one person is used to others. To solve this problem, we propose a fast and accurate cross-person activity recognition model, known as TransRKELM (Transfer learning Reduced Kernel Extreme Learning Machine) which uses RKELM (Reduced Kernel Extreme Learning Machine) to realize initial activity recognition model. In the online phase OS-RKELM (Online Sequential Reduced Kernel Extreme Learning Machine) is applied to update the initial model and adapt the recognition model to new device users based on recognition results with high confidence level efficiently. Experimental results show that, the proposed model can adapt the classifier to new device users quickly and obtain good recognition performance.