Semi-supervised Extreme Learning Machine Research Articles

A novel adaptive safe semi-supervised learning framework for pattern extraction and classification

<p>Manifold regularization semi-supervised learning is a powerful graph-based semi-supervised learning method. However, the performance of semi-supervised learning methods based on manifold regularization depends to some extent on the quality of the manifold graph and unlabeled samples. Intuitively speaking, the quality of the graph directly affects the final classification performance of the model. In response to the above problems, this paper first proposed an adaptive safety semi-supervised learning framework. The framework implements the weight assignment of the self-similarity graph during the model learning process. In order to adapt to the learning needs, accelerate the learning speed, and avoid the impact of the curse of dimensionality, the framework also optimizes the features of each sample point through an automatic weighting mechanism to extract effective features and eliminate redundant information in the learning task. In addition, the framework defines an adaptive risk measurement mechanism for the uncertainty and potential risks of unlabeled samples to determine the degree of risk of unlabeled samples. Finally, a new adaptive safe semi-supervised extreme learning machine was proposed. Comprehensive experimental results across various class imbalance scenarios demonstrated that our proposed method outperforms other methods in terms of classification accuracy, and other critical performance metrics.</p>

Financial Fraud Detection using Improved Artificial Humming Bird Algorithm with Modified Extreme Learning Machine

More and more industries, including the financial sector, are moving their operations online as internet usage continues to rise at an exponential rate. As a result, financial fraud is on the rise in all its guises and in all parts of the world, causing enormous economic damage. The purpose of financial fraud detection systems is to identify potential dangers, such as unauthorised access or unusual attacks. In recent years, this problem has been attacked using a variety of machine learning and data mining methods. Aalgorithms, on the other hand, are better able to deal with only a small quantity of labelled data and a large amount of unlabeled data, making them useful in situations where it would be impractical to rely solely on supervised learning algorithms to train a good-performing classifier. In this research, we propose a Semi-supervised Extreme Learning Machine (SKELM) built on top of the weighted kernel, which we call SELMWK. For the purpose of detecting financial fraud, this research proposes an enhanced artificial hummingbird algorithm (IAHA). The algorithm combines two essential techniques to enhance its capacity for optimisation. To begin, the Chebyshev chaotic map is used to seed the first population of artificial hummingbirds, which boosts the population's overall ability to do global searches. Second, the guided foraging phase incorporates the Levy flight to enlarge the search field and forestall early convergence. The experimental results demonstration that the suggested technique recovers the Internet monetary fraud detections.

A Semi-Supervised Extreme Learning Machine Algorithm Based on the New Weighted Kernel for Machine Smell

At present, machine sense of smell has shown its important role and advantages in many scenarios. The development of machine sense of smell is inseparable from the support of corresponding data and algorithms. However, the process of olfactory data collection is relatively cumbersome, and it is more difficult to collect labeled data. However, in many scenarios, to use a small amount of labeled data to train a good-performing classifier, it is not feasible to rely only on supervised learning algorithms, but semi-supervised learning algorithms can better cope with only a small amount of labeled data and a large amount of unlabeled data. This study combines the new weighted kernel with SKELM and proposes a semi-supervised extreme learning machine algorithm based on the weighted kernel, SELMWK. The experimental results show that the proposed SELMWK algorithm has good classification performance and can solve the semi-supervised gas classification task of the same domain data well on the used dataset.

Manifold semi-supervised learning for aluminum electrolysis temperature identification based on regularized hierarchical extreme learning machine

The aim of this article is to develop a soft approach for a real-time cell temperature prediction in the aluminum electrolysis reduction. Under the limited labeled data constraint, Laplacian semi-supervised learning methods, which can fully utilize the underlying structure of the data distribution and further extract information contained in all available data, has recently received extensive attention in the field of soft sensor modeling. Since the Laplacian underlying manifold is a constant, it remains a challenging task to improve the extrapolating ability for the case that only a few labeled samples are available. This study presents a soft modeling method based on a semi-supervised deep learning structure, which was developed from the hierarchical autoencoders with extreme learning machine. Furthermore, a Laplacian–Hessian semi-supervised extreme learning machine is built to learn all the geometric distribution information. The Laplacian–Hessian semi-supervised extreme learning machine method is applied to estimate the cell temperature in an aluminum reduction process. The experimental results demonstrate the performance and robustness of the proposed algorithm are superior to those of the existing state-of-the-art methods.

Semi-supervised learning-assisted imaging method for electrical capacitance tomography

Despite the enormous potential of the electrical capacitance tomography technology for the process industry, one of the main technological gaps and obstacles that must be overcome is the low accuracy reconstruction. To address this technical challenge, the data-dependent prior is introduced in this study, which is combined with the electrical measurement mechanism and the domain knowledge to reshape the tomographic imaging problem into a new optimization problem. The introduction of the data-dependent prior not only bridges the physical model and the advanced semi-supervised learning technique, but also improves the adaptability of the reconstruction model. A new numerical scheme that integrates the merits of the half-quadratic splitting method and the forward-backward splitting technique with a speedup strategy is proposed to solve the challenging imaging model, leading to the reduced computational burden. A new robust sparse semi-supervised extreme learning machine method that leverages both labeled and unlabeled samples and effectively handles high-dimensional image data is developed to predict the data-dependent prior, and the training is recast into a new optimization problem that is solved by a new numerical method. The novel imaging method achieves the reconstruction paradigm shift and acts as a catalyst for improving imaging quality. The evaluation of a series of reconstruction cases validates that the novel method shows satisfactory performances, outperforming both popular and classical imaging methods in terms of robustness and reconstruction accuracy.

Unsupervised domain adaptation for vibration-based robotic ground classification in dynamic environments

Vibration-based Robotic Ground Classification (V-RGC) is an ability of a field robot identifying the ground types (e.g., grass and clay) according to the proprioceptive vibration sequence. It has proved that the accurate and read-time V-RGC contributes a lot to the avoidance of non-geometric hazards, route planning, and pose estimation. However, V-RGC in a dynamic environment (i.e., data distribution may drift) has not yet been taken into consideration, which motivates us to propose a novel classification method named Joint Domain Adaptation Semi-Supervised Extreme Learning Machine (JDA-S2ELM). First, the projected maximum mean discrepancy (MMD) criterion is introduced to expand the classification boundaries from the source domain to the target domain in a computationally-efficient way. Second, the joint-distribution domain adaptation (DA) is proposed to realize a cascaded marginal- and conditional-distribution DA training framework, which shows a higher and more stable accuracy. Third, target-domain manifold regularization is added to smooth the classification boundaries to cut through low-density regions, thus further increasing the target-domain classification accuracy. The real-world experiment demonstrates that the proposed JDA-S2ELM could increase the target-domain accuracy from about 30% to 90%, which means that V-RGC is adaptable to a dynamic environment.

A Novel Ellipsoidal Semisupervised Extreme Learning Machine Algorithm and Its Application in Wind Turbine Blade Icing Fault Detection

The conventional semi-supervised extreme learning machine (SS-ELM) algorithm can provide a solution to the lack of labeled samples in wind turbine blade icing fault detection, but its performance is limited by the irrationality of the spherical nearest neighbor graph (SNNG) calculation strategy. To solve this problem, a novel ellipsoidal semi-supervised extreme learning machine (ESS-ELM) algorithm is proposed in this paper and applied to wind turbine blade icing fault detection. In this study, we creatively propose a novel ellipsoidal nearest neighbor graph (ENNG) calculation strategy that considers the distribution information of the labeled samples to construct the ESS-ELM algorithm. Different from the conventional SNNG, the ENNG can adaptively assign corresponding calculation weights to each feature in the data space according to the Fisher criterion, which allows it to better match the smoothness assumption of the manifold regularization learning framework. In addition, the two adjustable parameters of the enhancement factor and the degradation factor are ingeniously introduced into the ENNG, which further ensure the applicability and security of the proposed ESS-ELM. The superiority of the ESS-ELM algorithm is verified by extensive benchmark industrial fault datasets and the real-world icing dataset of two wind turbines. It is demonstrated that the proposed ESS-ELM algorithm achieves better performance than the ELM, its existing variants, and some state-of-the-art wind turbine blade icing detection methods.

A Multi-Strategy Whale Optimization Algorithm and Its Application

Whale Optimization Algorithm (WOA) is a key tool for solving complex engineering optimization problems, aiming at adjusting important parameters to satisfy constraints and optimal objectives. WOA has a simple structure, few parameters, high search capability, and easy implementation. However, it suffers from the same problems as other metaheuristic algorithms of being prone to local optima and slow convergence, for which the Multi-Strategy Whale Optimization Algorithm (MSWOA) is proposed. Four strategies are introduced in MSWOA. Firstly, a highly randomized chaotic logistic map is used to generate a high-quality initial population. Secondly, exploitation and exploration are enhanced by setting adaptive weights and dynamic convergence factors. Further, a Lévy flight mechanism is introduced to maintain the population diversity in each iteration. Finally, the Evolutionary Population Dynamics (EPD) mechanism is introduced to improve the efficiency of search agents in finding the optimum. Another problem lies in the Semi-Supervised Extreme Learning Machine (SSELM) based on manifold regularization is an effective classification and regression model, but the random generation of input weights and hidden layer thresholds and the grid selection of hyperparameters lead to unsatisfactory classification performance. To this end, we developed the MSWOA-SSELM model, optimally selected the parameters of SSLEM using MSWOA, and applied it to logging layer recognition, which effectively improved the accuracy of logging interpretation. By comparing the experiments with 14 swarm intelligence algorithms on 18 benchmark test functions, the CEC2017 benchmark suite, and an engineering application problem, the experimental results show that MSWOA is significantly superior and effective in solving global optimization problems. Finally, the proposed MSWOA-SSELM is applied in three wells and outperforms other classification models in terms of Accuracy (ACC), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE). It obtained the best results with 96.2567% ACC, MAE of 0.0749, and RMSE of 0.3870.

An Auto-Contouring Method for Kidney Using a Novel Semi-Supervised Leaning Extreme Learning Machine Method

Background: Adaptive radiation therapy planning requires contour segmentation of dangerous organs in medical images. However, manual contour rendering is the most time-consuming and laborious work in radiotherap planning. In order to solve this problem, we propose a novel semi-supervised leaning extreme learning machine (SSL-ELM) method to realize abdominal Magnetic Resonance Imaging (MRI) guided Adaptive Radiation Therapy (MR-ART) automatic contour rendering. Method/Material: Our algorithm is based on the assumption that data within the same class are close to each other. We use this heuristic method to improve the ELM algorithm. The experimental results show that our proposed method outperforms existing classification algorithms. We used a data set of eight patients with unresectable abdominal malignant tumors recruited by professionals approved by the Cleveland Medical Center Institution Review Board. Each group included MRI and airborne T1MRI with randomly selected treatment phases. Each MRI consisted of 16 slices with a resolution of 370×370 pixels. Manual and automatic contours of the kidney were compared using Dice similarity index (DSI). Results: The proposed SSL-ELM algorithm has better performance than most classification algorithms, and the experimental results also show that the DSI values are above 0.87, with some samples reaching 0.99.

Incremental semi-supervised Extreme Learning Machine for Mixed data stream classification

With an explosive growth of data generated in the Internet and other fields, the data stream classification has sparked broad interest recently. Nowadays, some of the challenges in data streams, such as concept drift detection and supervised data stream classification, have been well-developed. However, when confronted with mixed data streams (containing categorical and numerical values) or limited available labeled samples, many data stream methods cannot achieve a satisfying performance or even cannot work. To tackle these two problems, we proposed an Incremental Semi-supervised Extreme Learning Machine for Mixed data stream classification (MIS-ELM). To be specific, for the issue of mixed data in data streams, we designed a novel soft one-hot encoding method by combining the coupling object similarity method and the one-hot encoding method, which can embed categorical data into high-quality numerical data and is used in the data preprocessing phase of MIS-ELM; for the issue of limited labeled samples, we introduced an incremental learning method based on unlabeled data, which is employed in the training classifier phase of MIS-ELM. When no concept drift occurs in the data stream, MIS-ELM uses only unlabeled data for incremental learning to fine-tune the classifier trained in the previous sliding window. Also, MIS-ELM instinctively inherits the fast computability of ELM, so it is very suitable for the real-time processing of data streams. Finally, we evaluated the representation performance of the soft one-hot encoding and the classification performance of MIS-ELM, within real data streams. The experimental results demonstrate the superiority of the proposed methods over the state-of-the-art techniques in their areas, respectively.

Kernel Risk-Sensitive Loss based Hyper-graph Regularized Robust Extreme Learning Machine and Its Semi-supervised Extension for Classification

Kernel Risk-Sensitive Loss (KRSL) is a nonlinear similarity measure defined in kernel space, which enable the gradient based method to achieve higher accuracy while effectively weakening the negative effects caused by noise and outliers. Defined as a kernel function expectation between two random variables, KRSL has been successfully applied in robust machine learning and signal processing. Extreme Learning Machine, as one of the most popular methods of machine learning, has attracted great attention in supervised learning and semi-supervised learning. However, when the data contains noise and outliers, the manifold structure of the data is not considered or the neural network structure is too complex, the performance of traditional ELM methods will decline. Therefore, based on KRSL, hyper-graph regularization and L2,1-norm, we first propose a more robust ELM method named Kernel Risk-Sensitive Loss Based Hyper-graph Regularized Robust Extreme Learning Machine (KRSL-HRELM). In KRSL-HRELM, KRSL is introduced into ELM to enhance its ability to handle noise and outliers. Moreover, the hyper-graph regularization is integrated into the method to learn the higher-order geometric structure information between the data. In addition, the L2,1-norm is introduced to constrain the output weight matrix to obtain a sparse network model. Inspired by other semi-supervised ELM methods, we extend KRSL-HRELM to semi-supervised learning and propose its semi-supervised version semi-supervised KRSL-HRELM (SS- KRSL-HRELM). Empirical studies on a large number of real-world datasets show that the proposed methods are competitive with other advanced supervised or semi-supervised learning methods in terms of robustness and efficiency.

Semi-Supervised Classification via Hypergraph Convolutional Extreme Learning Machine

Extreme Learning Machine (ELM) is characterized by simplicity, generalization ability, and computational efficiency. However, previous ELMs fail to consider the inherent high-order relationship among data points, resulting in being powerless on structured data and poor robustness on noise data. This paper presents a novel semi-supervised ELM, termed Hypergraph Convolutional ELM (HGCELM), based on using hypergraph convolution to extend ELM into the non-Euclidean domain. The method inherits all the advantages from ELM, and consists of a random hypergraph convolutional layer followed by a hypergraph convolutional regression layer, enabling it to model complex intraclass variations. We show that the traditional ELM is a special case of the HGCELM model in the regular Euclidean domain. Extensive experimental results show that HGCELM remarkably outperforms eight competitive methods on 26 classification benchmarks.

Semi-Supervised Extreme Learning Machine Channel Estimator and Equalizer for Vehicle to Vehicle Communications

Wireless vehicular communications are a promising technology. Most applications related to vehicular communications aim to improve road safety and have special requirements concerning latency and reliability. The traditional channel estimation techniques used in the IEEE 802.11 standard do not properly perform over vehicular channels. This is because vehicular communications are subject to non-stationary, time-varying, frequency-selective wireless channels. Therefore, the main goal of this work is the introduction of a new channel estimation and equalization technique based on a Semi-supervised Extreme Learning Machine (SS-ELM) in order to address the harsh characteristics of the vehicular channel and improve the performance of the communication link. The performance of the proposed technique is compared with traditional estimators, as well as state-of-the-art machine-learning-based algorithms over an urban scenario setup in terms of bit error rate. The proposed SS-ELM scheme outperformed the extreme learning machine and the fully complex extreme learning machine algorithms for the evaluated scenarios. Compared to traditional techniques, the proposed SS-ELM scheme has a very similar performance. It is also observed that, although the SS-ELM scheme requires the largest operation time among the evaluated techniques, its execution time is still far away from the latency requirements specified by the standard for safety applications.

A Multi-Strategy Marine Predator Algorithm and Its Application in Joint Regularization Semi-Supervised ELM

A novel semi-supervised learning method is proposed to better utilize labeled and unlabeled samples to improve classification performance. However, there is exists the limitation that Laplace regularization in a semi-supervised extreme learning machine (SSELM) tends to lead to poor generalization ability and it ignores the role of labeled information. To solve the above problems, a Joint Regularized Semi-Supervised Extreme Learning Machine (JRSSELM) is proposed, which uses Hessian regularization instead of Laplace regularization and adds supervised information regularization. In order to solve the problem of slow convergence speed and the easy to fall into local optimum of marine predator algorithm (MPA), a multi-strategy marine predator algorithm (MSMPA) is proposed, which first uses a chaotic opposition learning strategy to generate high-quality initial population, then uses adaptive inertia weights and adaptive step control factor to improve the exploration, utilization, and convergence speed, and then uses neighborhood dimensional learning strategy to maintain population diversity. The parameters in JRSSELM are then optimized using MSMPA. The MSMPA-JRSSELM is applied to logging oil formation identification. The experimental results show that MSMPA shows obvious superiority and strong competitiveness in terms of convergence accuracy and convergence speed. Also, the classification performance of MSMPA-JRSSELM is better than other classification methods, and the practical application is remarkable.

Classification of the Complex Agricultural Planting Structure with a Semi-Supervised Extreme Learning Machine Framework

Many approaches have been developed to analyze remote sensing images. However, for the classification of large-scale problems, most algorithms showed low computational efficiency and low accuracy. In this paper, the newly developed semi-supervised extreme learning machine (SS-ELM) framework with k-means clustering algorithm for image segmentation and co-training algorithm to enlarge the sample sets was used to classify the agricultural planting structure at large-scale areas. Data sets collected from a small-scale area within the Hetao Irrigation District (HID) at the upper reaches of the Yellow River basin were used to evaluate the SS-ELM framework. The results of the SS-ELM algorithm were compared with those of the random forest (RF), ELM, support vector machine (SVM) and semi-supervised support vector machine (S-SVM) algorithms. Then the SS-ELM algorithm was applied to analyze the complex planting structure of HID in 1986–2010 by comparing the remote sensing estimated results with the statistical data. In the small-scale case, the SS-ELM algorithm performed better than the RF, ELM, SVM, and S-SVM algorithms. For the SS-ELM algorithm, the average overall accuracy (OA) was in a range of 83.00–92.17%. On the contrary, for the other four algorithms, their average OA values ranged from 56.97% to 92.84%. Whereas, in the classification of planting structure in HID, the SS-ELM algorithm had an excellent performance in classification accuracy and computational efficiency for three major planting crops including maize, wheat, and sunflowers. The estimated areas by using the SS-ELM algorithm based on the remote sensing images were consistent with the statistical data, and their difference was within a range of 3–25%. This implied that the SS-ELM framework could be served as an effective method for the classification of complex planting structures with relatively fast training, good generalization, universal approximation capability, and reasonable learning accuracy.

CSELM‐QE: A Composite Semi‐supervised Extreme Learning Machine with Unlabeled RSS Quality Estimation for Radio Map Construction

Wireless local area network (WLAN) fingerprint-based localization has become the most attractive and popular approach for indoor localization. However, the primary concern for its practical implementation is the laborious manual effort of calibrating sufficient location-labeled fingerprints. The Semi-supervised extreme learning machine (SELM) performs well in reducing calibration effort. Traditional SELM methods only use Received signal strength (RSS) information to construct the neighbor graph and ignores location information, which helps recognizing prior information for manifold alignments. We propose Composite SELM (CSELM) method by using both RSS signals and location information to construct composite graph. Besides, the issue of unlabeled RSS data quality has not been solved. We propose a novel approach called Composite semisupervised extreme learning machine with unlabeled RSS Quality estimation (CSELM-QE) that takes into account the quality of unlabeled RSS data and combines the composite neighbor graph, which considers location information in the semi-supervised extreme learning machine. Experimental results show that the CSELM-QE could construct a precise localization model, reduce the calibration effort for radio map construction and improve localization accuracy. Our quality estimation method can be applied to other methods that need to retain high quality unlabeled Received signal strength data to improve model accuracy.

Balanced Graph-based regularized semi-supervised extreme learning machine for EEG classification

Machine learning algorithms play a critical role in electroencephalograpy (EEG)-based brain-computer interface (BCI) systems. However, collecting labeled samples for classifier training and calibration is still difficult and time-consuming, especially for patients. As a promising alternative way to address the problem, semi-supervised learning has attracted much attention by exploiting both labeled and unlabeled samples in the training process. Nowadays, semi-supervised extreme learning machine (SS-ELM) is widely used in EEG classification due to its fast training speed and good generalization performance. However, the classification performance of SS-ELM largely depends on the quality of sample graph. The graphs of most semi-supervised algorithms are constructed by the similarity between labeled and unlabeled data called manifold graph. The more similar the structural information between samples, the greater probability they belong to the same class. In this paper, the label-consistency graph (LCG) and sample-similarity graph (SSG) are combined to constrain the model output. When the SSG is not accurate enough, the weight of LCG needs to be increased, and vice versa. The weight ratio of two graphs is optimized to obtain an optimal adjacency graph, and finally the best output weight vector is achieved. To verify the effectiveness of the proposed algorithm, it was validated and compared with several existing methods on two real datasets: BCI Competition IV Dataset 2a and BCI Competition III Dataset 4a. Experimental results show that our algorithm has achieved the promising results, especially when the number of labeled samples is small.

A joint optimization framework to semi-supervised RVFL and ELM networks for efficient data classification

Due to the inefficiency of gradient-based iterative ways in network training, randomization-based neural networks usually offer non-iterative closed form solutions. The random vector functional link (RVFL) and extreme learning machine (ELM) are two popular randomized networks which provide us unified frameworks for both regression and multi-class classification. Currently, existing studies on RVFL and ELM focused mainly on supervised tasks even though we usually have only a small number of labeled samples but a large number of unlabeled samples. Therefore, it is necessary to make both models appropriately utilize both labeled and unlabeled samples; that is, we should develop their semi-supervised extensions. In this paper, we propose a joint optimization framework to semi-supervised RVFL and ELM networks. In the formulated JOSRVFL (jointly optimized semi-supervised RVFL) and JOSELM, the output weight matrix and the label indicator matrix of the unlabeled samples can be jointly optimized in an iterative manner. We provide a novel approach to optimize the JOSRVFL and JOSELM objective functions. Extensive experiments on benchmark data sets and Electroencephalography-based emotion recognition tasks showed the excellent performance of the proposed JOSRVFL and JOSELM models. Moreover, because the direct input–output connections help to regularize the randomization, JOSRVFL could obtain superior performance to JOSELM in most cases.

A Novel Superheat Identification of Aluminum Electrolysis with Kernel Semi-supervised Extreme Learning Machine

In the aluminium reduction production industry, the superheat temperature (ST) is a vital index, which gives the distribution of the current efficiency. Keeping ST in an approximate range improves the lifespan of the electrolysis bath. However, in the practical production process, ST identification result is commonly evaluated by the experimental workers, the real-time measurement of ST is still a challenge that has not been solved. A novel ST measurement method based on kernel extreme learning machine (K-ELM) is studied in this paper. First, a few of input variables are selected according to expert experience. Then, a new activation function (IG kernel function) and regularization term are constructed in ELM to construct IG-SSELM. Finally, the ST model is built with all the dataset samples and further applied to ST real-time detection. The proposed method is applied to ST identification for the first time in the aluminium industrial process which is superior to the existing ST methods in accuracy and robustness.

Fisher-regularized supervised and semi-supervised extreme learning machine

The structural information of data contains useful prior knowledge and thus is important for designing classifiers. Extreme learning machine (ELM) has been a potential technique in handling classification problems. However, it only simply considers the prior class-based structural information and ignores the prior knowledge from statistics and geometry of data. In this paper, to capture more structural information of the data, we first propose a Fisher-regularized extreme learning machine (called Fisher-ELM) by applying Fisher regularization into the ELM learning framework, the main goals of which is to build an optimal hyperplane such that the output weight and within-class scatter are minimized simultaneously. The proposed Fisher-ELM reflects both the global characteristics and local properties of samples. Intuitively, the Fisher-ELM can approximatively fulfill the Fisher criterion and can obtain good statistical separability. Then, we exploit graph structural formulation to obtain semi-supervised Fisher-ELM version (called Lap-FisherELM) by introducing manifold regularization that characterizes the geometric information of the marginal distribution embedded in unlabeled samples. An efficient successive overrelaxation algorithm is used to solve the proposed Fisher-ELM and Lap-FisherELM, which converges linearly to a solution, and can process very large datasets that need not reside in memory. The proposed Fisher-ELM and Lap-FisherELM do not need to deal with the extra matrix and burden the computations related to the variable switching, which makes them more suitable for relatively large-scale problems. Experiments on several datasets verify the effectiveness of the proposed methods.