Large Margin Distribution Machine Research Articles

Hyperspectral Image Classification Based on Fusion of Curvature Filter and Domain Transform Recursive Filter

In recent decades, in order to enhance the performance of hyperspectral image classification, the spatial information of hyperspectral image obtained by various methods has become a research hotspot. For this work, it proposes a new classification method based on the fusion of two spatial information, which will be classified by a large margin distribution machine (LDM). First, the spatial texture information is extracted from the top of the principal component analysis for hyperspectral images by a curvature filter (CF). Second, the spatial correlation information of a hyperspectral image is completed by using domain transform recursive filter (DTRF). Last, the spatial texture information and correlation information are fused to be classified with LDM. The experimental results of hyperspectral images classification demonstrate that the proposed curvature filter and domain transform recursive filter with LDM(CFDTRF-LDM) method is superior to other classification methods.

Hyperspectral image classification based on bilateral filter with linear spatial correlation information

ABSTRACT Support Vector Machine (SVM) with the margin theory is widely used for the hyperspectral classification. However, the margin model is a single interval and does not represent the complete distribution of hyperspectral image data sets. In addition, the spatial texture information obtained by filtering in recent years has become a hot research topic for improving classification of hyperspectral images, but the spatial correlation information is often lost in the spatial texture information extraction. To solve this problem, this paper proposed an algorithm with large margin distribution machine (LDM) that combined the spatial information obtained by the bilateral filter and linear spatial correlation information (BFLSCI-LDM). First, spatial features were extracted by bilateral filter from hyperspectral image whose dimensionality was reduced by principal component analysis. Next, the linear spatial correlation information was constructed for hyperspectral images. Finally, the spatial information and original spectral information were combined for LDM. The experimental results of actual hyperspectral images indicated that the proposed BFLSCI-LDM method was superior to other classification methods, including the original SVM with the raw spectral features, the dimensionality reduction features, and spatial-spectral information, the method of edge-preserving filter and recursive filter, and the LDM-based method.

Adaptive scalable kernel for hyperspectral image classification

We present a framework based on the development of adaptive scalable kernel (ASK) for hyperspectral image classification, which can achieve an excellent status in removing insignificant details and defending crucial features. The proposed method consists of three steps. First, the spectral feature extraction based on interval gradient and a fast morphological filter is used to reduce the high dimensionality. Second, a powerful spatial structure extraction method based on adaptive scale kernels is adopted to enhance the performance of structure-preserving filtering. Depending on patch-based statistics, this model identifies small-scale texture from large-scale structure and finds an optimal per-pixel smoothing scale. Third, the obtained spectral structure feature maps are classified with the large-margin distribution machine. The experimental results show that the proposed spatial structure extraction method based on ASK achieves the state-of-the-art performance in terms of classification accuracy and computational efficiency.

Pancreatic cancer biomarker detection by two support vector strategies for recursive feature elimination.

Pancreatic cancer is one of the worst malignant tumors in prognosis. Therefore, to reduce the mortality rate of pancreatic cancer, early diagnosis and prompt treatment are particularly important. We put forward a new feature-selection method that was used to find clinical markers for pancreatic cancer by combination of Support Vector Machine Recursive Feature Elimination (SVM-RFE) and Large Margin Distribution Machine Recursive Feature Elimination (LDM-RFE) algorithms. As a result, seven differentially expressed genes were predicted as specific biomarkers for pancreatic cancer because of their highest accuracy of classification on cancer and normal samples. Three (MMP7, FOS and A2M) out of the seven predicted gene markers were found to encode proteins secreted into urine, providing potential diagnostic evidences for pancreatic cancer.

Large-margin Distribution Machine-based regression

This paper presents an efficient and robust Large-margin Distribution Machine formulation for regression. The proposed model is termed as ‘Large-margin Distribution Machine-based Regression’ (LDMR) model, and it is in the spirit of Large-margin Distribution Machine (LDM) (Zhang and Zhou, in: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, 2014) classification model. The LDM model optimizes the margin distribution instead of minimizing a single-point margin as is done in the traditional SVM. The optimization problem of the LDMR model has been mathematically derived from the optimization problem of the LDM model using an interesting result of Bi and Bennett (Neurocomputing 55(1):79–108, 2003). The resulting LDMR formulation attempts to minimize the $$\epsilon$$ -insensitive loss function and the quadratic loss function simultaneously. Further, the successive over-relaxation technique (Mangasarian and Musicant, IEEE Trans Neural Netw 10(5):1032−1037, 1999) has also been applied to speed up the training procedure of the proposed LDMR model. The experimental results on artificial datasets, UCI datasets and time-series financial datasets show that the proposed LDMR model owns better generalization ability than other existing models and is less sensitive to the presence of outliers.

Neuroimaging-based diagnosis of Parkinson's disease with deep neural mapping large margin distribution machine

Abstract Neuroimaging has shown its effectiveness for diagnosis of Parkinson's disease (PD), and the neuroimaging-based computer-aided diagnosis (CAD) then attracts considerable attention. In a CAD system, the classifier module is one of the key components, which directly decides the classification performance. As a newly proposed classifier, the large margin distribution machine (LDM) has excellent generalization by maximizing the margin mean and minimizing the margin variance simultaneously. However, LDM still suffers from the problem of kernel selection. In this work, we propose a deep neural mapping large margin distribution machine (DNMLDM) algorithm by adopting the deep neural network (DNN) to perform a kernel mapping instead of the implicit kernel function in LDM. A two-stage joint training strategy is then developed, including the unsupervised layer-wise pre-training for DNN and then the supervised fine-tuning for all parameters in the whole networks. Two real-world PD datasets, namely the transcranial sonography (TCS) dataset and the magnetic resonance imaging (MRI) dataset, are used to evaluate the performance of DNMLDM algorithm. The experimental results show that the proposed DNMLDM outperforms all the compared algorithms on both datasets.

Hyperspectral Image Classification Based on Fusion of Guided Filter and Domain Transform Interpolated Convolution Filter

In recent years, the spatial texture features obtained by filtering have become a hot research topic to improve hyperspectral image classification, but spatial correlation information is often lost in spatial texture information extraction. To solve this problem, a spectral-spatial classification method based on guided filtering and by the algorithm Large Margin Distribution Machine (LDM) is proposed. More specifically, the spatial texture features can be extracted by a Guided filter (GDF) from hyperspectral images whose dimensionality is reduced by a Principal Component Analysis (PCA). Spatial correlation features of the hyperspectral image are then obtained using a Domain Transform Interpolated Convolution Filter. The last step is to fuse spatial texture features and correlation features for classification by LDM. The experimental results using the actual hyperspectral image indicate that the proposed GDFDTICF-LDM method is superior to other classification methods, such as the original Support Vector Machine (SVM) with raw spectral features, dimensionality reduction features and spatial-spectral information, methods of edge-preserving filter and recursive filter, and LDM-based methods.

Bimodal Vein Data Mining via Cross-Selected-Domain Knowledge Transfer

Recent success in large-scale image recognition challenge (i.e., ImageNet) fully demonstrates the capability of deep neural network (DNN) in learning complex and semantic representation, and this also motivates the generation of transfer learning model, which fine-tunes state-of-the-art DNN models with other small-scale databases for better performance. Driven by such an idea, a task-specific DNN model fine-tuned from VGG-face is constructed for both gender and identity recognition with hand vein information. Unlike the traditional transfer learning models, which fine-tune directly from source to target, we leverage the coarse-to-fine scheme to train the task-specific models in a step-aware way, such that the inherent correlation between the neighboring databases could serve as initialization base to relieve the problem of over-fitting, which is inevitable with the small-scaled hand vein database, and also speed up the convergence. Besides, the task-driven network training idea, which involves joint optimization of linear regression classifier and network parameters, is also adopted during training of each model to obtain more discriminative representation for specified tasks. Instead of adopting the trained linear regression classifier for gender and identity classification, the large margin distribution machine (LDM) is introduced to ensure the discriminative and generalization performance of the model simultaneously, and it should be noted that before feeding the gender feature vector into the LDM, a supervised feature selection step is incorporated to improve the classification performance by discarding the redundant feature and highlighting the important ones for gender classification. Rigorous experiments using the lab-made database are conducted to demonstrate the effectiveness and feasibility of the proposed model. What is more, additional experiment with a subset of the PolyU database illustrates its generalization ability and robustness.

Cost-sensitive large margin distribution machine for fault detection of wind turbines

Given the importance of the class-imbalanced data and misclassified unequal costs in large wind turbine datasets, this paper proposes a cost-sensitive large margin distribution machine (CLDM) for fault detection of wind turbines. The margin mean and margin variance are use to characterize the margin distribution. The objective function and constraints of the large margin distribution machine (LDM) are modified to be cost-sensitive. The class-imbalanced data and misclassified unequal costs are solved by selecting the appropriately cost-sensitive parameters. Then CLDM is designed to train and test data from wind turbines in a wind farm. In order to verify the effectiveness of CLDM, it is compared with support vector machine (SVM), cost-sensitive SVM, and LDM. Comprehensive experiments on 7 datasets from a benchmark model of wind turbines and 5 datasets from a real wind farm show that CLDM has better sensitivity, gMean and average misclassified cost than the other methods.

Generalization improvement for regularized least squares classification

In the past decades, regularized least squares classification (RLSC) is a commonly used supervised classification method in the machine learning filed because it can be easily resolved through the simple matrix analysis and achieve a close-form solution. Recently, some studies conjecture that the margin distribution is more crucial to the generalization performance. Moreover, from the view of margin distribution, RLSC only considers the first-order statistics (i.e., margin mean) and does not consider the actual higher-order statistics of margin distribution. In this paper, we propose a novel RLSC which takes into account the actual second-order (i.e., variance) information of margin distribution. It is intuitively expected that small margin variance will improve the generalization performance of RLSC from a geometric view. We incorporate the margin variance into the objective function of RLSC and achieve the optimal classifier by minimizing the margin variance. To evaluate the performance of our algorithm, we conduct a series of experiments on several benchmark datasets in comparison with RLSC, kernel minimum squared error, support vector machine and large margin distribution machine. And the empirical results verify the effectiveness of our algorithm and indicate that the margin distribution is helpful to improve the classification performance.

Sequential Monte Carlo-guided ensemble tracking.

A great deal of robustness is allowed when visual tracking is considered as a classification problem. This paper combines a finite number of weak classifiers in a SMC framework as a strong classifier. The time-varying ensemble parameters (confidence of weak classifiers) are regarded as sequential arriving states and their posterior distribution is estimated in a Bayesian manner. Therefore, both the adaptiveness and stability are kept for the ensemble classification in handling scene changes and target deformation. Moreover, to increase the tracking accuracy, weak classifiers including Support Vector Machine (SVM) and Large Margin Distribution Machine (LDM) are combined as a hybrid strong one, with adaptiveness to the sample scales. Comprehensive experiments are performed on benchmark videos with various tracking challenges, and the proposed method is demonstrated to be better than or comparable to the state-of-the-art trackers.

Large margin distribution machine for hyperspectral image classification

Support vector machine (SVM) classifiers are widely applied to hyperspectral image (HSI) classification and provide significant advantages in terms of accuracy, simplicity, and robustness. SVM is a well-known learning algorithm that maximizes the minimum margin. However, recent theoretical results pointed out that maximizing the minimum margin leads to a lower generalization performance than optimizing the margin distribution, and proved that the margin distribution is more important. In this paper, a large margin distribution machine (LDM) is applied to HSI classification, and optimizing the margin distribution achieves a better generalization performance than SVM. Since the raw HSI feature space is not the most effective space for representing HSI, we adopt factor analysis to learn an effective HSI feature and the learned features are further filtered by a structure-preserved filter to fully exploit the spatial structure information of HSI. The spatial structure information is integrated in the feature learning process to obtain a better HSI feature. Then we propose a multiclass LDM to classify the filtered HSI feature. Experimental results show that the proposed LDM with feature learning method achieves the classification performance of the state-of-the-art methods in terms of visual quality and three quantitative evaluations and indicates that LDM has a high generalization performance. (C) 2016 SPIE and IS&T

Large cost-sensitive margin distribution machine for imbalanced data classification

This paper develops cost-sensitive margin distribution learning and proposes Large Cost-Sensitive margin Distribution Machine (LCSDM) to get balanced detection rate on imbalanced training data. Recently, margin theory revealed that compared with a single margin, margin distribution is more critical to the generalization performance. Large margin Distribution Machine (LDM) is designed to get superior classification performance and strong generalization performance. However, LDM generally has imbalanced margin distribution between two classes on imbalanced training data. This generally leads to the lower detection rate of the minority class, which contradicts to the needs of high detection rate of the minority class in many real applications. Therefore, cost-sensitive margin distribution learning is brought forward to obtain balanced margin distribution and detection rate between two classes. What's more, this research deduces the relation between cost-sensitive parameter and in-class detection rate, and designs LCSDM to obtain balanced detection rate. Experimental results show that LCSDM can gradually increase the margin distribution of the minority class to obtain a more balanced detection rate. As a general learning method, LCSDM is especially applicable to imbalanced data classification.

Cost-Sensitive Large margin Distribution Machine for classification of imbalanced data

This paper proposes a new method to design a balanced classifier on imbalanced training data based on margin distribution theory. Recently, Large margin Distribution Machine (LDM) is put forward and it obtains superior classification performance compared with Support Vector Machine (SVM) and many state-of-the-art methods. However, one of the deficiencies of LDM is that it easily leads to the lower detection rate of the minority class than that of the majority class on imbalanced data which contradicts to the needs of high detection rate of the minority class in the real application. In this paper, Cost-Sensitive Large margin Distribution Machine (CS-LDM) is brought forward to improve the detection rate of the minority class by introducing cost-sensitive margin mean and cost-sensitive penalty. Theoretical and experimental results show that CS-LDM can gradually improve the detection rate of the minority class with the increasing of the cost parameter and obtain a balanced classifier when the cost parameter increases to a certain value. CS-LDM is superior to some popular cost-sensitive methods and can be used in many applications.

A Directed Acyclic Graph-Large Margin Distribution Machine Model for Music Symbol Classification.

Optical Music Recognition (OMR) has received increasing attention in recent years. In this paper, we propose a classifier based on a new method named Directed Acyclic Graph-Large margin Distribution Machine (DAG-LDM). The DAG-LDM is an improvement of the Large margin Distribution Machine (LDM), which is a binary classifier that optimizes the margin distribution by maximizing the margin mean and minimizing the margin variance simultaneously. We modify the LDM to the DAG-LDM to solve the multi-class music symbol classification problem. Tests are conducted on more than 10000 music symbol images, obtained from handwritten and printed images of music scores. The proposed method provides superior classification capability and achieves much higher classification accuracy than the state-of-the-art algorithms such as Support Vector Machines (SVMs) and Neural Networks (NNs).