Large Margin Distribution Machine Research Articles

A [formula omitted]-means triangular synthesis large margin classifier with unified pinball loss for imbalanced data

Large Margin Distribution Machine (LDM) improves the Support Vector Machine (SVM) by integrating the marginal distribution of samples into the objective function, which exhibits excellent classification and generalization performance. However, most of the existing LDM-based models exhibit inherent flaws: (1) The assumption of a category uniform distribution, resulting in an inability to effectively address the issue of class imbalance. (2) The inheritance of the hinge loss in SVM, leading to inferior anti-noise performance. Given the shortcomings above, this paper constructs a novel K-means Triangular Synthesis (KTS) method for imbalanced data classification and introduces Unified Pinball (UP) loss to ensure robust performance, demonstrated as a novel KTS large margin classifier with UP Loss (KTS-UPLMC) model. Specifically, the KTS method utilizes the triangular synthesis and generates samples based on the cluster density, effectively mitigating the problems of introducing noise samples and strip-shaped sample distribution in the traditional synthetic minority oversampling technique, further alleviating the LDM’s classification line offset. In addition, the UP Loss considers quantile distance, improving the anti-noise performance of the model. Comparative experiments on artificial datasets and benchmark datasets validate the effectiveness and noise resistance of the proposed KTS-UPLMC in imbalanced data classification problems. Furthermore, the stability of the KTS-UPLMC is substantiated by the parameter sensitivity analysis experiment. In conclusion, the KTS method effectively addresses category imbalance, while the integration of the UP Loss enhances noise resistance.

A novel cost-sensitive three-way intuitionistic fuzzy large margin classifier

Three-way decision (3WD) has been widely applied in diverse fields in tackling uncertainty, particularly in classification domain. As a discriminative learning algorithm, Large Margin Distribution Machine (LDM) aims to maximize the inter-class margin by leveraging the marginal distribution of samples, which captures the underlying structure and achieves superior classification performance. However, existing LDMs still suffer from some inherent flaws, including: i) the neglect of fuzzy decision items, leading to the inadequate handling of uncertainty; ii) the utilization of a cost-insensitive mechanism, resulting in high misclassification costs; and iii) the disregard of sample credibility, contributing to the noise susceptibility. To address these limitations, we introduce the intuitionistic fuzzy (IF) and cost-sensitive 3WD (CS3WD), presenting an innovative model known as the CS3W-IFLMC model. The IF theory is employed to quantify sample confidence levels, augmenting noise suppression in our proposed model. Moreover, the integration of the CS3WD method effectively reduces the overall decision cost and further improves the handling of uncertainty in datasets. Consequently, the CS3W-IFLMC model demonstrates superior noise resilience and generalization performance. Our comparative experiments validate the efficacy of the proposed CS3W-IFLMC model in achieving robustness to noise while upholding a competitive classification performance.

Weighted Linear Loss Large Margin Distribution Machine for Pattern Classification

Weighted Linear Loss Large Margin Distribution Machine for Pattern Classification

A Novel Fuzzy Large Margin Distribution Machine With Unified Pinball Loss

A Novel Fuzzy Large Margin Distribution Machine With Unified Pinball Loss

Multiview Large Margin Distribution Machine.

Margin distribution has been proven to play a crucial role in improving generalization ability. In recent studies, many methods are designed using large margin distribution machine (LDM), which combines margin distribution with support vector machine (SVM), such that a better performance can be achieved. However, these methods are usually proposed based on single-view data and ignore the connection between different views. In this article, we propose a new multiview margin distribution model, called MVLDM, which constructs both multiview margin mean and variance. Besides, a framework is proposed to achieve multiview learning (MVL). MVLDM provides a new way to explore the utilization of complementary information in MVL from the perspective of margin distribution and satisfies both the consistency principle and the complementarity principle. In the theoretical analysis, we used Rademacher complexity theory to analyze the consistency error bound and generalization error bound of the MVLDM. In the experiments, we constructed a new performance metric, the view consistency rate (VCR), for the characteristics of multiview data. The effectiveness of MVLDM was evaluated using both VCR and other traditional performance metrics. The experimental results show that MVLDM is superior to other benchmark methods.

Simpler large margin distribution machine via weighted linear loss for large-scale classification

Simpler large margin distribution machine via weighted linear loss for large-scale classification

Fuzzy large margin distribution machine for classification

Fuzzy large margin distribution machine for classification

Vibration-based looseness identification of bolted structures via quasi-analytic wavelet packet and optimized large margin distribution machine

Bolted joints are the most widely utilized connection types in industries, and therein looseness identification of bolted structures is of great significance to guarantee structural reliability. In this article, a comprehensive study of bolt looseness identification under random excitation is presented. To fulfill this task, this research focuses on three prominent difficulties, including nonstationary signal processing, subtle feature extraction, and robust state classification. First, a novel filter bank structure of quasi-analytic dual-tree complex wavelet packet transform is constructed to analyze the measured vibration response signals, for purpose of capturing subtle feature information. Then, multiple features are extracted from subband signals to capture the variations of dynamic characteristics, and sensitive features are selected by Laplacian score to construct the low-dimensional feature set. Subsequently, a novel classifier with better generalization performance, named large margin distribution machine, is optimized with the wavelet kernel function and the whale optimization algorithm, in order to handle the intrinsic uncertainty related to the looseness states of bolted structures. After feeding the low-dimensional feature set, the proposed classifier is trained to identify looseness states of bolted structures. Finally, experiments of a two-bolt lapped beam under random excitation are conducted to verify the effectiveness of the proposed method, and two typical loading conditions (paired-bolt looseness and single-bolt looseness) are considered. Besides, the superiority of the proposed method is demonstrated by comparing with other analogical methods. This research can provide a promising implement in practical applications of bolt looseness identification under random excitation.

Hyperspectral Image Classification Based on the Gabor Feature with Correlation Information

Gabor filter is widely used to extract spatial texture features of hyperspectral images (HSI) for HSI classification; however, a single Gabor filter cannot obtain the complete image features. In the paper, we propose an HSI classification method that combines the Gabor filter (GF) and domain-transformation standard convolution (DTNC) filter. First, we use the Gabor filter to extract spatial texture features from the first two principal components of the dimensionality-reduction HSI with PCA. Second, we use the DTNC filter to extract spatial correlation features from HSI in all bands. Finally, the Large Margin Distribution Machine (LDM) uses the linear fusion of the two kinds of spatial features to classify HSI. The experimental results show that the classification accuracy of Indian Pines, Pavia University, and Kennedy Space Center data sets is 96.64, 98.23, and 98.95% with only 4, 3, and 6% training samples, respectively; and these accuracies are 2–20% higher than the other tested methods. Compared with the hyperspectral information based on SVM, EPF, IFRF, PCA-EPFs, LDM-FL, and GFDN method, the proposed method, GFDTNCLDM, significantly improves the accuracy of HSI classification.

Intelligent Fault Diagnosis for Bridge via Modal Analysis

Due to natural disasters and man-made reasons, bridges are prone to structural damage during long-term usage, which reduces the associated carrying capacity, increases natural aging, and reduces safety. It is urgent to monitor the health status of bridge structure via intelligent technology. This paper proposes a bridge fault recognition structure. First, the signals of bridge parameter are collected by using distributed sensors. Then, the collected signals are processed by signal processing to extract the features in time and frequency domain. Lastly, the extracted features are used to learn an intelligent classifier. The large margin distribution machine is adopted as a classification model. The experimental results have proven the feasibility of the proposed bridge fault recognition structure.

Minimum Distribution Support Vector Clustering.

Support vector clustering (SVC) is a boundary-based algorithm, which has several advantages over other clustering methods, including identifying clusters of arbitrary shapes and numbers. Leveraged by the high generalization ability of the large margin distribution machine (LDM) and the optimal margin distribution clustering (ODMC), we propose a new clustering method: minimum distribution for support vector clustering (MDSVC), for improving the robustness of boundary point recognition, which characterizes the optimal hypersphere by the first-order and second-order statistics and tries to minimize the mean and variance simultaneously. In addition, we further prove, theoretically, that our algorithm can obtain better generalization performance. Some instructive insights for adjusting the number of support vector points are gained. For the optimization problem of MDSVC, we propose a double coordinate descent algorithm for small and medium samples. The experimental results on both artificial and real datasets indicate that our MDSVC has a significant improvement in generalization performance compared to SVC.

Laplacian large margin distribution machine for semi-supervised classification

Semi-Supervised Learning (SSL) has attracted much attention in the field of machine learning and data mining. As an extension of Support Vector Machine (SVM), the Semi-Supervised Support Vector Machine (S3VM) was proposed for SSL. Recent studies have disclosed that optimising the margin distribution is more crucial than maximising the minimum margin in generating a better classification. However, the existing S3VM models still follow the idea of maximising the minimum margin. Therefore, this paper proposes a novel Laplacian Large margin Distribution Machine (LapLDM) for SSL to enhance the classification performance. This method can optimise the margin distribution by maximising the first-order (margin mean) and minimising the second-order (margin variance) statistics of margins, and exploit the geometry information of marginal distribution embedded in the unlabelled data through the Laplacian regularizer. Then this paper develops a Preconditioned Conjugate Gradient (PCG) algorithm to solve the nonlinear LapLDM model on those regular-scaled data sets and a Stochastic Gradient Descent with Variance Reduction (SVRG) algorithm to solve the linear LapLDM model on those large-scaled data sets. These algorithms can accelerate the implementing efficiencies of proposed models and make them available for those large-scaled problems. Finally, the numerical results on four artificial and fourteen public benchmark data sets demonstrate that the LapLDM is superior to some well-known S3VM models.

Least squares large margin distribution machine for regression

Better prediction ability is the main objective of any regression-based model. Large margin Distribution Machine for Regression (LDMR) is an efficient approach where it tries to reduce both loss functions, i.e. e-insensitive and quadratic loss to diminish the effects of outliers. However, still, it has a significant drawback, i.e. high computational complexity. To achieve the improved generalization of the regression model with less computational cost, we propose an enhanced form of LDMR named as Least Squares Large margin Distribution Machine-based Regression (LS-LDMR) by transforming the inequality conditions alleviate to equality conditions. The elucidation is attained by handling a system of linear equations where we need to measure the inverse of the matrix only. Hence, there is no need to solve the large size of the quadratic programming problem, unlike in the case of other regression-based algorithms as SVR, Twin SVR, and LDMR. The numerical experiment has been performed on the benchmark real-life datasets along with synthetically generated datasets by using the linear and Gaussian kernel. All the experiments of presented LS-LDMR are analyzed with standard SVR, Twin SVR, primal least squares Twin SVR (PLSTSVR), e-Huber SVR (e-HSVR), e-support vector quantile regression (e-SVQR), minimum deviation regression (MDR), and LDMR, which shows the effectiveness and usability of LS-LDMR. This approach is also statistically validated and verified in terms of various metrics.

Fault Detection of the Wind Turbine Variable Pitch System Based on Large Margin Distribution Machine Optimized by the State Transition Algorithm

Aiming at solving the problem that the parameters of a fault detection model are difficult to be optimized, the paper proposes the fault detection of the wind turbine variable pitch system based on large margin distribution machine (LDM) which is optimized by the state transition algorithm (STA). By setting the three parameters of the LDM model as a three-dimensional vector which was searched by STA, by using the accuracy of fault detection model as the fitness function of STA, and by adopting the four state transformation operators of STA to carry out global search in the form of point, line, surface, and sphere in the search space, the global optimal parameters of LDM fault detection model are obtained and used to train the model. Compared with the grid search (GS) method, particle swarm optimization (PSO) algorithm, and genetic algorithm (GA), the proposed model method has lower false positive rate (FPR) and false negative rate (FNR) in the fault detection of wind turbine variable pitch system in a real wind farm.

Software defect prediction model based on distance metric learning

Software defect prediction (SDP) is a very important way for analyzing software quality and reducing development costs. The data during software lifecycle can be used to predict software defect. Currently, many SDP models have been proposed; however, their performance was not always ideal. In many existing prediction models based on machine learning, the distance metric between samples has significant impact on the performance of the SDP model. In addition, most samples are usually class imbalanced. To solve these issues, in this paper, a novel distance metric learning based on cost-sensitive learning (CSL) is proposed for reducing the impact of class imbalance of samples, which is then applied to the large margin distribution machine (LDM) to substitute the traditional kernel function. Further, the improvement and optimization of LDM based on CSL are also studied, and the improved LDM is used as the SDP model, called as CS-ILDM. Subsequently, the proposed CS-ILDM is applied to five publicly available data sets from the NASA Metrics Data Program repository and its performance is compared to other existing SDP models. The experimental results confirm that the proposed CS-ILDM not only has good prediction performance, but also can reduce the misprediction cost and avoid the impact of class imbalance of samples.

Nonparallel support vector machine with large margin distribution for pattern classification

The large margin distribution machine (LDM) combines the working principle of support vector machine (SVM) and the margin distribution to directly improve the algorithm's generalization. The margin distribution can be expressed with the margin mean and margin variance. It has been proved to be an efficient algorithm for binary classification. Inspired by the LDM, a novel classifier termed as LMD-NPSVM is proposed to improve the generalization performance of the nonparallel support vector machine (NPSVM) in this paper. Firstly, to meet the structure of NPSVM, the large margin distribution is reconstructed. Then, the linear LMD-NPSVM is built by introducing the reconstructed margin distribution into NPSVM. In addition, the linear case is extended to the nonlinear case with a kernel trick. All experiments show that our LMD-NPSVM is superior to the state-of-the-art algorithms in generalization performance.

Twin support vector machine based on adjustable large margin distribution for pattern classification

This paper researches the value of the margin distribution in binary classifier. The central idea of large margin distribution machine (LDM) is to optimize the margin distribution, such as maximizing the margin mean and minimizing the margin variance. Compared to support vector machine (SVM), LDM demonstrates the good generalization performance. In order to improve the generalization performance of twin support vector machine (TSVM), a twin support vector machine based on adjustable large margin distribution (ALD-TSVM) is proposed in this paper. Firstly, the margin distribution is redefined to construct a pair of adjustable supporting hyperplanes. Then, the redefined margin distribution is introduced onto TSVM to obtain the models of ALD-TSVM, including linear case and nonlinear case. ALD-TSVM is a general learning method which can be used in any place where TSVM and LDM can be applied. Finally, the novel method is compared with other classification algorithms by doing experiments on toy dataset, UCI datasets and image datasets. The experimental results show that ALD-TSVM obtains better classification performance.

Multiple Spatial Features Extraction and Fusion for Hyperspectral Images Classification

In recent decades, spatial feature extraction has greatly improved the performance of hyperspectral image (HSI) classification. This paper presents an HSI classification method based on multiple spatial features extraction and fusion (MSFs-EF). The method consists of five sequential steps. 1- Principal component analysis is applied for HSI dimensionality reduction. 2- A mean curvature filter is used to extract the spatial texture features from the HSI. 3- The spatial correlation features are obtained using a domain transform normalized convolution filter. 4- Spatial texture features and spatial correlation features are combined. 5- The multiple spatial features are classified using the Large Margin Distribution Machine. Three hyperspectral data sets are used to verify the performance. This method improves the accuracy of HSI classification compared with SVM method, edge-preserving filter, recursive filter method, and deep learning method. In the case of ratios of training samples of 5%, 0.6%, and 5%, the overall accuracy of three data sets reaches 98.23%, 99.17%, and 98.21% respectively, and are about 1.3%∼19%, 0.2%∼13%, and 0.4%∼13% higher than other fourteen algorithms. In the case of ratios of training samples of 10%, 1%, and 10%, the overall accuracy of the three data sets reaches 98.63%, 99.53%, and 98.99%, respectively and still outperform other methods.

Adaptive Hyperspectral Image Classification Based on the Fusion of Manifolds Filter and Spatial Correlation Features

In recent decades, the studies that obtain abundant spatial texture features, using a wide variety of filters for improving the performance of hyperspectral image (HSI) classification, have become a hotspot. However, the classification methods based on various filters are easy to fall into local feature extraction and neglect informative spatial correlation features. This paper presents an adaptive HSI classification method based on the fusion of adaptive manifold filter and spatial correlation feature (AMSCF). In which we use an adaptive manifold filter to extract spatial texture features, and use the domain transform normalized convolution filter and interpolated convolution filter to obtain the spatial correlation features. Besides, the spatial texture features and two correlation features are separately fused and classified by Large Margin Distribution Machine (LDM) to obtain the best classification. The experimental results demonstrate that the proposed AMSCF method is better than the other classification methods.

Imbalanced Data Classification Based on Cost-Sensitive Large Margin Distribution Machine

Imbalanced Data Classification Based on Cost-Sensitive Large Margin Distribution Machine