Semi-supervised Support Vector Machine Research Articles

Semi-Supervised Support Vector Machine for Digital Twins Based Brain Image Fusion.

The purpose is to explore the feature recognition, diagnosis, and forecasting performances of Semi-Supervised Support Vector Machines (S3VMs) for brain image fusion Digital Twins (DTs). Both unlabeled and labeled data are used regarding many unlabeled data in brain images, and semi supervised support vector machine (SVM) is proposed. Meantime, the AlexNet model is improved, and the brain images in real space are mapped to virtual space by using digital twins. Moreover, a diagnosis and prediction model of brain image fusion digital twins based on semi supervised SVM and improved AlexNet is constructed. Magnetic Resonance Imaging (MRI) data from the Brain Tumor Department of a Hospital are collected to test the performance of the constructed model through simulation experiments. Some state-of-art models are included for performance comparison: Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), AlexNet, and Multi-Layer Perceptron (MLP). Results demonstrate that the proposed model can provide a feature recognition and extraction accuracy of 92.52%, at least an improvement of 2.76% compared to other models. Its training lasts for about 100 s, and the test takes about 0.68 s. The Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) of the proposed model are 4.91 and 5.59%, respectively. Regarding the assessment indicators of brain image segmentation and fusion, the proposed model can provide a 79.55% Jaccard coefficient, a 90.43% Positive Predictive Value (PPV), a 73.09% Sensitivity, and a 75.58% Dice Similarity Coefficient (DSC), remarkably better than other models. Acceleration efficiency analysis suggests that the improved AlexNet model is suitable for processing massive brain image data with a higher speedup indicator. To sum up, the constructed model can provide high accuracy, good acceleration efficiency, and excellent segmentation and recognition performances while ensuring low errors, which can provide an experimental basis for brain image feature recognition and digital diagnosis.

Semi-supervised multi-Layer convolution kernel learning in credit evaluation

In many practical credit evaluation problems, a lot of manpower as well as financial and material resources are required to label samples. Therefore, in the process of labeling, only a small number of samples with category labels can be obtained to train classification models and a large number of customer samples is abandoned without category labels. To solve this problem, we introduce a semi-supervised support vector machine (SVM) technology and combines it with a multi-layer convolution kernel to construct a semi-supervised multi-layer convolution kernel SVM (SSMCK) for category customer credit assessment data sets. We first use a basic solution of the generalized differential operator to generate a base convolution kernel function in the H1 space, and then use the multi-layer strategy of deep learning to construct the multi-layer convolution kernel in the H2 and H3 space (called the family of multi-layer convolution kernel) by using the kernel functions in the H1 space. We further propose a semi-supervised multi-layer convolution kernel SVM algorithm based on the category center estimation and develop two novel SSMCK methods to improve the classification ability: the SSMCK based on multi-kernel learning (SSMCK-MKL) and the SSMCK based on alternative optimization (SSMCK-AO). Finally, experimental verification and analysis is carried out on three customer credit evaluation data sets. The results show that our methods outperforms or are comparable to some the state-of-the-art credit evaluation models.

Semi-supervised support vector regression based on data similarity and its application to rock-mechanics parameters estimation

Rock-mechanics parameters such as Young’s modulus and Poisson’s ratio are critical to geomechanical analysis and resource exploration. Because these parameters come from laboratory measurement, they present some characteristics such as insufficient samples and contamination of outliers. In this paper, a novel semi-supervised support vector machine soft sensor is devised considering the characteristics of the parameters. First, it takes into account data similarity and selects labeled data set that are most similar to the continuous unlabeled data set at each iteration to improve estimation performance. Meanwhile, an outlier deletion algorithm is developed for a better similarity comparison. After that, a semi-supervised approach is presented for the estimation of rock-mechanics parameters, it can leverage continuous unlabeled data to train the model dynamically. Finally, the verification of our method is carried out on data set from UCI (University of California, Irvine) and several drilling sites. The results demonstrate that our method outperforms eight well-known methods in estimation accuracy.

Porphyry Cu-Au prospectivity modelling using semi-supervised learning algorithm in Dehsalm district, eastern Iran

Porphyry Cu-Au prospectivity modelling using semi-supervised learning algorithm in Dehsalm district, eastern Iran

Laplacian large margin distribution machine for semi-supervised classification

Semi-Supervised Learning (SSL) has attracted much attention in the field of machine learning and data mining. As an extension of Support Vector Machine (SVM), the Semi-Supervised Support Vector Machine (S3VM) was proposed for SSL. Recent studies have disclosed that optimising the margin distribution is more crucial than maximising the minimum margin in generating a better classification. However, the existing S3VM models still follow the idea of maximising the minimum margin. Therefore, this paper proposes a novel Laplacian Large margin Distribution Machine (LapLDM) for SSL to enhance the classification performance. This method can optimise the margin distribution by maximising the first-order (margin mean) and minimising the second-order (margin variance) statistics of margins, and exploit the geometry information of marginal distribution embedded in the unlabelled data through the Laplacian regularizer. Then this paper develops a Preconditioned Conjugate Gradient (PCG) algorithm to solve the nonlinear LapLDM model on those regular-scaled data sets and a Stochastic Gradient Descent with Variance Reduction (SVRG) algorithm to solve the linear LapLDM model on those large-scaled data sets. These algorithms can accelerate the implementing efficiencies of proposed models and make them available for those large-scaled problems. Finally, the numerical results on four artificial and fourteen public benchmark data sets demonstrate that the LapLDM is superior to some well-known S3VM models.

Soft fault diagnosis of analog circuits based on semi-supervised support vector machine

Soft fault diagnosis has been validated as a very challenging problem in analog circuits. In order to improve the generalization ability and close to the practical application of fault diagnosis models, a novel method for obtaining a large number of training samples in soft fault interval is proposed in the present work. Training samples are randomly generated in the interval of soft fault to adapt the continuously change of component parameters. Limits of experimental conditions, lead to the limited number of samples labeled by experts, so that training samples are classified using the semi-supervised support vector machine (S3VM) algorithm. Manifold learning algorithm has been used for the feature extraction and the dimension reduction of soft fault time domain response data in analog circuits. Then the semi-supervised Gaussian mixture model (SGMM) is applied to cluster decision trees. Finally, the S3VM classification is used for the soft fault diagnosis in analog circuits. Experimental results show that the proposed method can be utilized in the single and double soft fault diagnosis of analog circuits. It is observed that the S3VM method is extremely significant as a guide in actual engineering applications, although the diagnosis rate is slightly less than the fixed-parameter offset soft faults.

A Novel Approach to Oil Layer Recognition Model Using Whale Optimization Algorithm and Semi-Supervised SVM

The dataset distribution of actual logging is asymmetric, as most logging data are unlabeled. With the traditional classification model, it is hard to predict the oil and gas reservoir accurately. Therefore, a novel approach to the oil layer recognition model using the improved whale swarm algorithm (WOA) and semi-supervised support vector machine (S3VM) is proposed in this paper. At first, in order to overcome the shortcomings of the Whale Optimization Algorithm applied in the parameter-optimization of the S3VM model, such as falling into a local optimization and low convergence precision, an improved WOA was proposed according to the adaptive cloud strategy and the catfish effect. Then, the improved WOA was used to optimize the kernel parameters of S3VM for oil layer recognition. In this paper, the improved WOA is used to test 15 benchmark functions of CEC2005 compared with five other algorithms. The IWOA–S3VM model is used to classify the five kinds of UCI datasets compared with the other two algorithms. Finally, the IWOA–S3VM model is used for oil layer recognition. The result shows that (1) the improved WOA has better convergence speed and optimization ability than the other five algorithms, and (2) the IWOA–S3VM model has better recognition precision when the dataset contains a labeled and unlabeled dataset in oil layer recognition.

One novel class of Bézier smooth semi-supervised support vector machines for classification

The semi-supervised support vector machine (S3VM) for classification is introduced for dealing with quantities of unlabeled data in the real world. Labeled data are utilized to train the algorithm and then were adapted to classify the unlabeled data. However, this algorithm has several drawbacks, such as the non-smooth term of semi-supervised objective function negatively affects the classification precision. Moreover, it is required to endure heavy burden in solving two quadratic programming problems with inversion matrix operation. To cope with this problem, this article puts forward a novel class of Bezier smooth semi-supervised support vector machines (BS4VMs), based on the approximation property of Bezier function to the non-smooth term. Because of this approximation, a fast quasi-Newton method for solving BS4VMs can be used to decrease the calculating time scale. This new kind of algorithm enhances the generalization and robustness of S3VM for nonlinear case as well. Further, to show how the BS4VMs can be practically implemented, experiments on synthetic, UCI dataset, USPS dataset, and large-scale NDC database are offered. The theoretical analysis and experiments comparisons clearly confirm the superiority of BS4VMs in both classification accuracy and calculating time.

Semisupervised SVM by Hybrid Whale Optimization Algorithm and Its Application in Oil Layer Recognition

In many fields, such as oil logging, it is expensive to obtain labeled data, and a large amount of inexpensive unlabeled data are not used. Therefore, it is necessary to use semisupervised learning to obtain accurate classification with limited labeled data and many unlabeled data. The semisupervised support vector machine (S3VM) is the most useful method in semisupervised learning. Nevertheless, S3VM model performance will degrade when the sample number of categories is not even or have lots of unlabeled samples. Thus, a new semisupervised SVM by hybrid whale optimization algorithm (HWOA-S3VM) is proposed in this paper. Firstly, a tradeoff control parameter is added in S3VM to deal with an uneven sample of category which can cause S3VM to degrade. Then, a hybrid whale optimization algorithm (HWOA) is used to optimize the model parameters of S3VM to increase the classification accuracy. For HWOA improvement, an opposition-based cubic mapping is used to initialize the WOA population to improve the convergence speed, and the catfish effect is used to help WOA jump out of the local optimum and obtain the global optimization ability. In the experiments, firstly, the HWOA is tested by 12 classic benchmark functions of CEC2005 and four functions of CEC2014 compared with the other five algorithms. Then, six UCI datasets are used to test the performance of HWOA-S3VM and compared with the other four algorithms. Finally, we applied HWOA-S3VM to perform oil layer recognition of three oil well datasets. These experimental results show that (1) HWOA has a higher convergence speed and better global searchability than other algorithms. (2) HWOA-S3VM model has higher classification accuracy on UCI datasets than other algorithms when combined, labeled, and unlabeled data are used as the training dataset. (3) The recognition accuracy and speed of the HWOA-S3VM model are superior to the other four algorithms when applied in oil layer recognition.

An Online Semisupervised Learning Model for Pedestrians’ Crossing Intention Recognition of Connected Autonomous Vehicle Based on Mobile Edge Computing Applications

One of the major challenges that connected autonomous vehicles (CAVs) are facing today is driving in urban environments. To achieve this goal, CAVs need to have the ability to understand the crossing intention of pedestrians. However, for autonomous vehicles, it is quite challenging to understand pedestrians’ crossing intentions. Because the pedestrian is a very complex individual, their intention to cross the street is affected by the weather, the surrounding traffic environment, and even his own emotions. If the established street crossing intention recognition model cannot be updated in real time according to the diversity of samples, the efficiency of human-machine interaction and the interaction safety will be greatly affected. Based on the above problems, this paper established a pedestrian crossing intention model based on the online semisupervised support vector machine algorithm (OS3VM). In order to verify the effectiveness of the model, this paper collects a large amount of pedestrian crossing data and vehicle movement data based on laser scanner, and determines the main feature components of the model input through feature extraction and principal component analysis (PCA). The comparison results of recognition accuracy of SVM, S3VM, and OS3VM indicate that the proposed OS3VM model exhibits a better ability to recognize pedestrian crossing intentions than the SVM and S3VM models, and the accuracy achieves 94.83%. Therefore, the OS3VM model can reduce the number of labeled samples for training the classifier and improve the recognition accuracy.

Photovoltaic Fault Diagnosis Via Semisupervised Ladder Network With String Voltage and Current Measures

In recent years, many supervised learning algorithms have been successfully applied for photovoltaic (PV) fault diagnosis. In practice, it is not possible to effectively obtain labels of large samples, limiting the engineering application of these algorithms. As for the unsupervised learning algorithm, it is completely adaptive learning, requiring a large number of samples to better learn the potential features in the data. To address the above problems, an improved online fault diagnosis method is proposed, which uses a small number of labeled samples to train the semisupervised ladder network (SSLN) fault diagnosis model to realize the diagnosis of line-to-line faults, open-circuit faults, partial shadow faults, and hybrid faults. In the proposed method, only the real-time operating voltage and current of PV array are needed for fault diagnosis. The sequential voltage and current of the PV array are first normalized, and the sequential power waveforms are obtained through numerical calculation. Then, the SSLN is used to extract the fault features from the sequence power waveforms. Finally, the classification is realized using the SSLN's noiseless encoder. To eliminate overfitting and improve convergence, the activation function, optimizer, and loss function of the SSLN is studied and improved. Meanwhile, numerical simulations and measured data verify that the proposed method provides strong anti-interference, and the diagnostic accuracies of both exceed 98%. Comparative experiments show that the proposed method outperforms algorithms such as squared-loss mutual information regularization, semisupervised support vector machine, graph-based semisupervised learning, and semisupervised extreme learning machine.

Classification of the Complex Agricultural Planting Structure with a Semi-Supervised Extreme Learning Machine Framework

Many approaches have been developed to analyze remote sensing images. However, for the classification of large-scale problems, most algorithms showed low computational efficiency and low accuracy. In this paper, the newly developed semi-supervised extreme learning machine (SS-ELM) framework with k-means clustering algorithm for image segmentation and co-training algorithm to enlarge the sample sets was used to classify the agricultural planting structure at large-scale areas. Data sets collected from a small-scale area within the Hetao Irrigation District (HID) at the upper reaches of the Yellow River basin were used to evaluate the SS-ELM framework. The results of the SS-ELM algorithm were compared with those of the random forest (RF), ELM, support vector machine (SVM) and semi-supervised support vector machine (S-SVM) algorithms. Then the SS-ELM algorithm was applied to analyze the complex planting structure of HID in 1986–2010 by comparing the remote sensing estimated results with the statistical data. In the small-scale case, the SS-ELM algorithm performed better than the RF, ELM, SVM, and S-SVM algorithms. For the SS-ELM algorithm, the average overall accuracy (OA) was in a range of 83.00–92.17%. On the contrary, for the other four algorithms, their average OA values ranged from 56.97% to 92.84%. Whereas, in the classification of planting structure in HID, the SS-ELM algorithm had an excellent performance in classification accuracy and computational efficiency for three major planting crops including maize, wheat, and sunflowers. The estimated areas by using the SS-ELM algorithm based on the remote sensing images were consistent with the statistical data, and their difference was within a range of 3–25%. This implied that the SS-ELM framework could be served as an effective method for the classification of complex planting structures with relatively fast training, good generalization, universal approximation capability, and reasonable learning accuracy.

A New Collaborative Filtering Recommendation Method Based on Transductive SVM and Active Learning

In the collaborative filtering (CF) recommendation applications, the sparsity of user rating data, the effectiveness of cold start, the strategy of item information neglection, and user profiles construction are critical to both the efficiency and effectiveness of the recommendation algorithm. In order to solve the above problems, a personalized recommendation approach combining semisupervised support vector machine and active learning (AL) is proposed in this paper, which combines the benefits of both TSVM (Transductive Support Vector Machine) and AL. Firstly, a “maximum-minimum segmentation” of version space-based AL strategy is developed to choose the most informative unlabeled samples for human annotation; it aims to choose the least data which is enough to train a high-quality model. And then, an AL-based semisupervised TSVM algorithm is proposed to make full use of the distribution characteristics of unlabeled samples by adding a manifold regularization into objective function, which is helpful to make the proposed algorithm to overcome the traditional drawbacks of TSVM. Furthermore, during the procedure of recommendation model construction, not only user behavior information and item information, but also demographic information is utilized. Due to the benefits of the above design, the quality of unlabeled sample annotation can be improved; meanwhile, both the data sparsity and cold start problems are alleviated. Finally, the effectiveness of the proposed algorithm is verified based on UCI datasets, and then it is applied to personalized recommendation. The experimental results show the superiority of the proposed method in both effectiveness and efficiency.

Development of a document classification method by using geodesic distance to calculate similarity of documents

Currently, the Internet has given people the opportunity to access to human knowledge quickly and conveniently through various channels such as Web pages, social networks, digital libraries, portals... However, with the process of exchanging and updating information quickly, the volume of information stored (in the form of digital documents) is increasing rapidly. Therefore, we are facing challenges in representing, storing, sorting and classifying documents. In this paper, we present a new approach to text classification. This approach is based on semi-supervised machine learning and Support Vector Machine (SVM). The new point of the study is that instead of calculating the distance between the vectors by Euclidean distance, we use geodesic distance. To do this, the text must first be expressed as an n-dimensional vector. In the n-dimensional vector space, each vector is represented by one point; use geodesic distance to calculate the distance from a point to nearby points and connect into a graph. The classification is based on calculating the shortest path between vertices on the graph through a kernel function. We conducted experiments on articles taken from Reuters on 5 different topics. To evaluate the proposed method, we tested the SVM method with the traditional calculation based on Euclidean distance and the method we proposed based on geodesic distance. The experiment was performed on the same data set of 5 topics: Business, Markets, World, Politics, and Technology. The results showed that the correct classification rate is better than the traditional SVM method based on Euclidean distance (average of 3.2 %)

Quantum algorithm for Help-Training semi-supervised support vector machine

Semi-supervised support vector machine ( $$S^3VM$$ ) is a popular strategy for many machine learning tasks due to the expensiveness of getting enough labeled data. In this paper, we propose a quantum Help-Training $$S^3VM$$ and design a quantum Parzen window model to select $$n_1+n_2$$ unlabeled data from l labeled and n unlabeled data set in each iteration, the time complexity is $$O(\tau \sqrt{nn_1}+\tau \sqrt{nn_2}+\tau \sqrt{n})$$ for $$\tau $$ iterations, which exhibits a quadratic speed-up over classical algorithm, we adopt quantum linear system to build Lagrangian multipliers with accuracy $$\varepsilon $$ , the time complexity is $$O(\tau \kappa ^3$$ $$\varepsilon ^{-3} \hbox {polylog}(N(n+l)))$$ , where condition number is $$\kappa $$ and feature dimension is N, it is exponentially faster than classical $$S^3VM$$ algorithm. Our scheme has two significant merits, (i) we provide the first quantum method for semi-supervised learning, which uses multiple unlabeled data with quantum superposition to predict Lagrangian multipliers at the same time, (ii) quantum matrix decomposition method avoids building matrices of different dimensions in one iteration; specially, this work provides inspiration to explore the potential quantum machine learning applications.

Pixel-Level Classification of Pollution Severity on Insulators Using Photothermal Radiometry and Multiclass Semisupervised Support Vector Machine

Pollution flashover is a serious accident with a wide range of impacts in power grid. Accurate and timely classification of pollution severity is both a key to preventing pollution flashovers and a crucial challenge. This article proposes an innovative method based on photothermal radiometry (PTR) and multiclass semisupervised support vector machine for the classification of insulator pollution severity. The introduction of time-dimension information makes it possible to achieve pixel-level image identification instead of image- or region-level image identification. The PTR physical model for pollution severity measurement is established to determine the effect of pollution severity parameters such as the equivalent salt deposit density and the nonsoluble deposit density on the transient and frequency-domain thermal radiation characteristics of contamination layer. The relevant features are extracted by using principal component analysis. A semisupervised classifier is proposed to solve the problem of poor generalization due to insufficient labeled samples in industrial applications. Experimental results verify the satisfactory efficiency and accuracy of the proposed method and an estimation framework for fast, accurate, and nondestructive industrial application without the tedious work of labeling large amounts of data is concluded.

Online detection of bearing incipient fault with semi-supervised architecture and deep feature representation

Although researchers have made substantial progress in bearing fault detection and diagnosis recently, incipient fault detection, especially online detection, is still at an initial stage. Generally speaking, online detection of incipient faults is still subject to the following challenges: (1) improving discriminative ability of incipient fault features; (2) adaptive recognition of the distribution inconsistency that exists in online sequential data; (3) achieving automatic detections with avoiding manual adjustment of detection criterion; and (4) reducing false alarm rate. To address these challenges, this paper presents a new approach for bearing incipient fault online detection using semi-supervised architecture and deep feature representation. This approach is simple and effective. First, we extract deep features using stacked denoising auto-encoder from the target bearing's normal state data and an auxiliary bearing's fault state data. Second, we introduce safe semi-supervised support vector machine (S4VM), a kind of semi-supervised classifier, to identify the sequentially arrived data of the target bearing as normal or anomalous. To update the classifier effectively, we use the principal curve to generate synthetic fault data for keeping data classes balanced during online condition monitoring. Finally, we propose a new fault alarm criterion based on S4VM generalization error upper bound to adaptively recognize the occurrence of an incipient fault. The experimental results on three datasets (IEEE PHM Challenge 2012, IMS and XJTU-SY) demonstrate the effectiveness and high reliability of the proposed approach.

Deep Relation Network for Hyperspectral Image Few-Shot Classification

Deep learning has achieved great success in hyperspectral image classification. However, when processing new hyperspectral images, the existing deep learning models must be retrained from scratch with sufficient samples, which is inefficient and undesirable in practical tasks. This paper aims to explore how to accurately classify new hyperspectral images with only a few labeled samples, i.e., the hyperspectral images few-shot classification. Specifically, we design a new deep classification model based on relational network and train it with the idea of meta-learning. Firstly, the feature learning module and the relation learning module of the model can make full use of the spatial–spectral information in hyperspectral images and carry out relation learning by comparing the similarity between samples. Secondly, the task-based learning strategy can enable the model to continuously enhance its ability to learn how to learn with a large number of tasks randomly generated from different data sets. Benefitting from the above two points, the proposed method has excellent generalization ability and can obtain satisfactory classification results with only a few labeled samples. In order to verify the performance of the proposed method, experiments were carried out on three public data sets. The results indicate that the proposed method can achieve better classification results than the traditional semisupervised support vector machine and semisupervised deep learning models.

A novel semi-supervised support vector machine with asymmetric squared loss

Laplacian support vector machine (LapSVM), which is based on the semi-supervised manifold regularization learning framework, performs better than the standard SVM, especially for the case where the supervised information is insufficient. However, the use of hinge loss leads to the sensitivity of LapSVM to noise around the decision boundary. To enhance the performance of LapSVM, we present a novel semi-supervised SVM with the asymmetric squared loss (asy-LapSVM) which deals with the expectile distance and is less sensitive to noise-corrupted data. We further present a simple and efficient functional iterative method to solve the proposed asy-LapSVM, in addition, we prove the convergence of the functional iterative method from two aspects of theory and experiment. Numerical experiments performed on a number of commonly used datasets with noise of different variances demonstrate the validity of the proposed asy-LapSVM and the feasibility of the presented functional iterative method.

A Dynamic Active Safe Semi-Supervised Learning Framework for Fault Identification in Labeled Expensive Chemical Processes

A novel active semi-supervised learning framework using unlabeled data is proposed for fault identification in labeled expensive chemical processes. A principal component analysis (PCA) feature selection strategy is first given to calculate the weight of the variables. Secondly, the identification model is trained based on the obtained key process variables. Thirdly, the pseudo label confidence of identification model is dynamically optimized with an historical, current, and future pseudo label confidence mean. To increase the upper limit of the identification model that is self-learning with high entropy process data, active learning is used to identify process data and diagnosis fault causes by ontology. Finally, a PCA-dynamic active safe semi-supervised support vector machine (PCA-DAS4VM) for fault identification in labeled expensive chemical processes is built. The application in the Tennessee Eastman (TE) process shows that this hybrid technology is able to: (i) eliminate chemical process noise and redundant process variables simultaneously, (ii) combine historical pseudo label confidence with future pseudo label confidence to improve the identification accuracy of abnormal working conditions, (iii) efficiently select and diagnose high entropy unlabeled process data, and (iv) fully utilize unlabeled data to enhance the identification performance.