Multi-view Support Vector Machines Research Articles

Multi-view support vector machine classifier via [formula omitted] soft-margin loss with structural information

Multi-view learning seeks to leverage the advantages of various views to complement each other and make full use of the latent information in the data. Nevertheless, effectively exploring and utilizing common and complementary information across diverse views remains challenging. In this paper, we propose two multi-view classifiers: multi-view support vector machine via L0/1 soft-margin loss (MvL0/1-SVM) and structural MvL0/1-SVM (MvSL0/1-SVM). The key difference between them is that MvSL0/1-SVM additionally fuses structural information, which simultaneously satisfies the consensus and complementarity principles. Despite the discrete nature inherent in the L0/1 soft-margin loss, we successfully establish the optimality theory for MvSL0/1-SVM. This includes demonstrating the existence of optimal solutions and elucidating their relationships with P-stationary points. Drawing inspiration from the P-stationary point optimality condition, we design and integrate a working set strategy into the proximal alternating direction method of multipliers. This integration significantly enhances the overall computational speed and diminishes the number of support vectors. Last but not least, numerical experiments show that our suggested models perform exceptionally well and have faster computational speed, affirming the rationality and effectiveness of our methods.

Multi-view universum support vector machines with insensitive pinball loss

Multi-view learning (MVL) is a promising field that seeks to make the most of information shared across different views, ultimately improving generalization performance. Numerous multi-view support vector machines (SVMs) have been proposed and shown great success. However, hinge loss is widely used by existing methods, which makes these SVMs sensitive to noise and unstable when re-sampling. Furthermore, many researchers have demonstrated the effectiveness of universum data for supervised learning. This paper proposes a novel MVL method called multi-view universum support vector machine with insensitive pinball loss (Pin-MvUSVM). Specifically, we first integrate the pinball loss into the multi-view learning framework, which is dependent on the quantization distance and is robust to noise. Then, we utilize universum data to explore prior knowledge for MVL, which improves the generalization performance. Moreover, we develop multi-view universum twin support vector machine with insensitive pinball loss (Pin-MvUTSVM) as a low-complexity alternative to improve computational efficiency. It solves a pair of smaller-scale quadratic programming problems (QPPs) and utilizes the dual form of the model to solve two convex optimization problems for obtaining the prediction classifier. We conduct classification experiments on four multi-view datasets, and our results show that the proposed convex models outperform several state-of-the-art models.

Safe screening rules for multi-view support vector machines

Multi-view learning aims to make use of the advantages of different views to complement each other and fully mines the potential information in the data. However, the complexity of multi-view learning algorithm is much higher than that of single view learning algorithm. Based on the optimality conditions of two classical multi-view models: SVM-2K and multi-view twin support vector machine (MvTwSVM), this paper analyzes the corresponding relationship between dual variables and samples, and derives their safe screening rules for the first time, termed as SSR-SVM-2K and SSR-MvTwSVM. It can assign or delete four groups of different dual variables in advance before solving the optimization problem, so as to greatly reduce the scale of the optimization problem and improve the solution speed. More importantly, the safe screening criterion is “safe”, that is, the solution of the reduced optimization problem is the same as that of the original problem before screening. In addition, we further give a sequence screening rule to speed up the parameter optimization process, and analyze its properties, including the similarities and differences of safe screening rules between multi-view SVMs and single-view SVMs, the computational complexity, and the relationship between the parameter interval and screening rate. Numerical experiments verify the effectiveness of the proposed methods.

Multi-view intuitionistic fuzzy support vector machines with insensitive pinball loss for classification of noisy data

Multi-view support vector machines (MvSVMs) have been widely used to solve multi-view classification problems. However, the conventional MvSVMs often overlook the presence of noise and outliers that commonly exist in the original data. In this paper, we propose two novel multi-view intuitionistic fuzzy support vector machines with insensitive pinball loss that can not only handle the general multi-view classification problems but also be robust to noisy data. In the proposed convex optimization models, the pinball loss is incorporated into the multi-view learning, which enables the maximization of the quantization distance. Moreover, to utilize multi-view information more effectively, the intuitionistic fuzzy score is introduced to assign a weight to each multi-view sample. The intuitionistic fuzzy score combines the membership and non-membership functions, which provides an efficient mechanism to assign weights to the multi-view samples. Further, we provide a discussion of the proposed models with the state-of-the-art technologies. Experiments are conducted on a series of datasets, and the results show that the proposed convex models outperform several state-of-the-art models. The code is available at 1.

Robust multi-view learning with the bounded LINEX loss

Multi-view learning, as a promising direction, emphasizes the consensus principle and the complementarity principle to boost the performance. By exploiting view-consistency or view-discrepancy among different views, numerous successful multi-view support vector machine models have been proposed. However, existing methods face two challenges. Firstly, most multi-view support vector machine models only consider the consensus principle, but ignore the complementarity principle. How to build a novel model with both principles has not been fully considered. Secondly, most multi-view support vector machine models neglect the robustness when the multi-view dataset is contaminated by the noisy samples, error-prone samples and view-inconsistent samples. Considering that the bounded linear-exponential (BLINEX) loss function possesses elegant merits, i.e., asymmetry and boundedness, developing a robust BLINEX-based model is worth exploring. Therefore, in this paper, we propose a BLINEX-based multi-view learning method called MVASY-BX, which explores the consensus and complementarity information with a between-view co-regularization term and importance weights of two views respectively. The mixed BLINEX loss is designed to make the model robust to noisy samples, view-inconsistent samples and error-prone samples. We solve linear and nonlinear MVASY-BX through the stochastic sub-gradient descent algorithm and the alternating direction method of multipliers respectively. Furthermore, we analyze the generalization error bound via Rademacher complexity. The comprehensive experiments confirm that our proposed model is more competitive than other benchmark methods.

Fusion Information Multi-View Classification Method for Remote Sensing Cloud Detection

In recent years, many studies have been carried out to detect clouds on remote sensing images. Due to the complex terrain, the variety of clouds, the density, and content of clouds are various, and the current model has difficulty accurately detecting the cloud in the image. In our strategy, a multi-view data training set based on super pixel is constructed. View A uses multi-level network to extract the boundary, texture, and deep abstract feature of super pixels. View B is the statistical feature of the three channels of the image. Privilege information View P contains the cloud content of super pixels and the tag status of adjacent super pixels. Finally, we propose a cloud detection method for remote sensing image classification based on multi-view support vector machine (SVM). The proposed method is tested on images of different terrain and cloud distribution in GF-1_WHU and Cloud-38 remote sensing datasets. Visual performance and quantitative analysis show that the method has excellent cloud detection performance.

Stability-based PAC-Bayes analysis for multi-view learning algorithms

Multi-view learning exploits structural constraints among multiple views to effectively learn from data. Although it has made great methodological achievements in recent years, the current generalization theory is still insufficient to prove the merit of multi-view learning. This paper blends stability into multi-view PAC-Bayes analysis to explore the generalization performance and effectiveness of multi-view learning algorithms. We propose a novel view-consistency regularization to produce an informative prior that helps to obtain a stability-based multi-view bound. Furthermore, we derive an upper bound on the stability coefficient that is involved in the PAC-Bayes bound of multi-view regularization algorithms for the purpose of computation, taking the multi-view support vector machine as an example. Experiments provide strong evidence on the advantageous generalization bounds of multi-view learning over single-view learning. We also explore strengths and weaknesses of the proposed stability-based bound compared with previous non-stability multi-view bounds experimentally.

Multi-View Modeling Method for Functional MRI Images

This paper proposes a new unsupervised fuzzy feature mapping method based on fMRI data and combines it with multi-view support vector machine to construct a classification model for computer-aided diagnosis of autism. Firstly, a multi-output TSK fuzzy system is adopted to map the original feature data to the linear separable high-dimensional space. Then a manifold regularization learning framework is introduced, and a new method of unsupervised fuzzy feature learning is proposed. Finally, a multi-view SVM algorithm is used for classification tasks. The experimental results show that the method in this paper can effectively extract important features from the fMRI data in the resting state and improve the model's interpretability on the premise of ensuring the superior and stable classification performance of the model.

Subgraph feature extraction based on multi-view dictionary learning for graph classification

Subgraph feature extraction of graph data has an efficiency problem that has become increasingly significant. A new architecture of subgraph feature extraction named GMADL is proposed in this paper. Dictionary learning approaches are put forward to extract the features of graph data to enhance the discrimination of model. To improve the efficiency of extraction, the analysis dictionary is designed as a bridge to generate the sparse code directly. Each sparse code represents the feature matrix of a graph. Through constructing the multi-view support vector machine (SVM) classifiers, the problem can be transferred into the multi-view problem so that the information of the whole view is utilized to predict the classification model. The comparison of the proposed architecture with the state-of-the-art approaches manifests the feasibility and the competitive performance in graph classification.

Multi-View Support Vector Machines with the Consensus and Complementarity Information

Multi-view learning (MVL) is an active direction in machine learning that aims at exploiting the consensus and complementarity information among multiple distinct feature sets to boost the generalization performance of the counterpart algorithm. So far, two classical SVM-based MVL methods are SVM-2K and multi-view twin support vector machine (MvTSVM). They are designed only for two-view classification and cannot tackle the general multi-view classification problem. They also cannot effectively leverage the complementarity information among different feature views. In this paper, we propose two novel multi-view support vector machines with the consensus and complementarity information for MVL that not only can deal with the two-view classification problem but also the general multi-view classification problem by jointly learning multiple different views in a non-pairwise way. The disagreement among different views is regarded as a constraint or a regularization term in the objective function which plays an important role in exploring the consensus information. Combination weights for the reconstruction of each view in regularization terms are learned to explore complementarity information among different views. Finally, an efficient iteration algorithm with the classical convex quadratic programming is developed for optimization. Experimental results validate the effectiveness of our proposed methods.

Multi-view SVM Classification with Feature Selection

With the rapid development of data mining technology, multi-view learning (MVL) has become a new research field, which has attracted wide attention of scholars at home and abroad. Multi-view learning is to combine multiple view data of the same entity for data classification, thereby improving learning performance. Previous multi-view research methods mainly concentrate on the relationship between different data views for classification problems. However, when the data is in high dimensions, it is necessary to perform feature selection in the multi-view data classification process. In this paper, we proposed a Multi-view Support Vector Machine Classification with Feature Selection (MSVMCFS) algorithm, which can not only classify multi-view data, but also select features for each view data in the process of classification. In the model, feature selection is performed by the l1 norm sparsity regularization, and consistency and complementarity between the two views are maintained. To achieve the optimization goal, we adopt linear programming to solve the model. The experimental results on 30 binary datasets demonstrate the validity of the model.

Multi-Dimension Support Vector Machine Based Crowd Detection and Localisation Framework for Varying Video Sequences

In this paper, we propose a novel method for anomalous crowd behaviour detection and localization with divergent centers in intelligent video sequence through multiple SVM (support vector machines) based appearance model. In multi-dimension SVM crowd detection, many features are available to track the object robustly with three main features which include 1) identification of an object by gray scale value, 2) histogram of oriented gradients (HOG) and 3) local binary pattern (LBP). We propose two more powerful features namely gray level co-occurrence matrix (GLCM) and Gaber feature for more accurate and authenticate tracking result. To combine and process the corresponding SVMs obtained from each features, a new collaborative strategy is developed on the basis of the confidence distribution of the video samples which are weighted by entropy method. We have adopted subspace evolution strategy for reconstructing the image of the object by constructing an update model. Also, we determine reconstruction error from the samples and again automatically build an update model for the target which is tracked in the video sequences. Considering the movement of the targeted object, occlusion problem is considered and overcome by constructing a collaborative model from that of appearance model and update model. Also if update model is of discriminative model type, binary classification problem is taken into account and overcome by collaborative model. We run the multi-view SVM tracking method in real time with subspace evolution strategy to track and detect the moving objects in the crowded scene accurately. As shown in the result part, our method also overcomes the occlusion problem that occurs frequently while objects under rotation and illumination change due to different environmental conditions.

Object Tracking with Multi-View Support Vector Machines

How to build an accurate and reliable appearance model to improve the performance is a crucial problem in object tracking. Since the multi-view learning can lead to more accurate and robust representation of the object, in this paper, we propose a novel tracking method via multi-view learning framework by using multiple support vector machines (SVM). The multi-view SVMs tracking method is constructed based on multiple views of features and a novel combination strategy. To realize a comprehensive representation, we select three different types of features, i.e., gray scale value, histogram of oriented gradients (HOG), and local binary pattern (LBP), to train the corresponding SVMs. These features represent the object from the perspectives of description, detection, and recognition, respectively . In order to realize the combination of the SVMs under the multi-view learning framework, we present a novel collaborative strategy with entropy criterion, which is acquired by the confidence distribution of the candidate samples. In addition, to learn the changes of the object and the scenario, we propose a novel update scheme based on subspace evolution strategy. The new scheme can control the model update adaptively and help to address the occlusion problems . We conduct our approach on several public video sequences and the experimental results demonstrate that our method is robust and accurate, and can achieve the state-of-the-art tracking performance.