Twin Support Vector Clustering Research Articles

Energy-based structural least squares twin support vector clustering

Clustering is an unsupervised learning algorithm and it is widely used in machine learning. Twin support vector clustering (TWSVC) is a new plane-based clustering algorithm, which exploits information from both within and between clusters to generate plane for each cluster. However, TWSVC suffered from relatively poor performance because it ignores the intrinsic structural information of data and solves a series of quadratic programming problems (QPPs). In order to address these problems, in this paper, we propose a novel energy-based structural least squares twin support vector clustering, termed as ESLSTWSVC. Firstly, based on least squares twin support vector clustering (LSTWSVC), we introduce within-class covariance matrix into the objective function of LSTWSVC to obtain more intrinsic structural information. ESLSTWSVC solves a series of system of linear equations rather than to solve QPPs in TWSVC, it leads to simple algorithm and less computation time. In addition, ESLSTWSVC converts the constraints of LSTWSVC into energy-based model by introducing an energy parameter for each cluster that makes ESLSTWSVC more robust. Experiments are performed on artificial datasets as well as UCI datasets, and the experimental results illustrate the effectiveness of the proposed algorithm.

Plane-based clustering with asymmetric distribution loss

Ramp-based twin support vector clustering (RampTWSVC) is a powerful clustering method, which measures the within-cluster and between-cluster scatter by the bounded ramp functions. However, the ramp loss is a symmetric function and it does not consider the data distribution, which makes the RampTWSVC not robust enough to deal with noisy datasets. In this paper, we construct an asymmetric distribution loss function and propose a novel plane-based clustering with asymmetric distribution loss (ADPC). The asymmetric distribution loss function combined the ramp loss with first-order and second-order statistics, it inherits the advantages of ramp loss function and captures the data distribution precisely, which makes ADPC more robust to the samples far from the cluster center plane. Moreover, two adjusting parameters t1 and t2 are introduced to flexibly control the form of the asymmetric distribution loss function and improve the performance of the algorithm. In addition, its nonlinear clustering formation is also provided by using the kernel trick. Some experiments on noise-corrupted benchmark UCI and artificial datasets have been provided, and the detailed comparison results show that the proposed ADPC has better performance on most datasets. Further experiments and analysis on the face clustering have been done to demonstrate the effectiveness and feasibility of the proposed ADPC.

Pinball Loss Twin Support Vector Clustering

Twin Support Vector Clustering (TWSVC) is a clustering algorithm inspired by the principles of Twin Support Vector Machine (TWSVM). TWSVC has already outperformed other traditional plane based clustering algorithms. However, TWSVC uses hinge loss, which maximizes shortest distance between clusters and hence suffers from noise-sensitivity and low re-sampling stability. In this article, we propose Pinball loss Twin Support Vector Clustering (pinTSVC) as a clustering algorithm. The proposed pinTSVC model incorporates the pinball loss function in the plane clustering formulation. Pinball loss function introduces favorable properties such as noise-insensitivity and re-sampling stability. The time complexity of the proposed pinTSVC remains equivalent to that of TWSVC. Extensive numerical experiments on noise-corrupted benchmark UCI and artificial datasets have been provided. Results of the proposed pinTSVC model are compared with TWSVC, Twin Bounded Support Vector Clustering (TBSVC) and Fuzzy c-means clustering (FCM). Detailed and exhaustive comparisons demonstrate the better performance and generalization of the proposed pinTSVC for noise-corrupted datasets. Further experiments and analysis on the performance of the above-mentioned clustering algorithms on structural MRI (sMRI) images taken from the ADNI database, face clustering, and facial expression clustering have been done to demonstrate the effectiveness and feasibility of the proposed pinTSVC model.

Sparse Twin Support Vector Clustering Using Pinball Loss.

Clustering is a widely used machine learning technique for unlabelled data. One of the recently proposed techniques is the twin support vector clustering (TWSVC) algorithm. The idea of TWSVC is to generate hyperplanes for each cluster. TWSVC utilizes the hinge loss function to penalize the misclassification. However, the hinge loss relies on shortest distance between different clusters, and is unstable for noise-corrupted datasets, and for re-sampling. In this paper, we propose a novel Sparse Pinball loss Twin Support Vector Clustering (SPTSVC). The proposed SPTSVC involves the ϵ-insensitive pinball loss function to formulate a sparse solution. Pinball loss function provides noise-insensitivity and re-sampling stability. The ϵ-insensitive zone provides sparsity to the model and improves testing time. Numerical experiments on synthetic as well as real world benchmark datasets are performed to show the efficacy of the proposed model. An analysis on the sparsity of various clustering algorithms is presented in this work. In order to show the feasibility and applicability of the proposed SPTSVC on biomedical data, experiments have been performed on epilepsy and breast cancer datasets.

General Plane-Based Clustering With Distribution Loss.

In this article, we propose a general model for plane-based clustering. The general model reveals the relationship between cluster assignment and cluster updating during clustering implementation, and it contains many existing plane-based clustering methods, e.g., k-plane clustering, proximal plane clustering, twin support vector clustering, and their extensions. Under this general model, one may obtain an appropriate clustering method for a specific purpose. The general model is a procedure corresponding to an optimization problem, which minimizes the total loss of the samples. Thereinto, the loss of a sample derives from both within-cluster and between-cluster information. We discuss the theoretical termination conditions and prove that the general model terminates in a finite number of steps at a local or weak local solution. Furthermore, we propose a distribution loss function that fluctuates with the input data and introduce it into the general model to obtain a plane-based clustering method (DPC). DPC can capture the data distribution precisely because of its statistical characteristics, and its termination that finitely terminates at a weak local solution is given immediately based on the general model. The experimental results show that our DPC outperforms the state-of-the-art plane-based clustering methods on many synthetic and benchmark data sets.

Least squares projection twin support vector clustering (LSPTSVC)

Clustering is a prominent unsupervised learning technique. In the literature, many plane based clustering algorithms are proposed, such as the twin support vector clustering (TWSVC) algorithm. In this work, we propose an alternative algorithm based on projection axes termed as least squares projection twin support vector clustering (LSPTSVC). The proposed LSPTSVC finds projection axis for every cluster in a manner that minimizes the within class scatter, and keeps the clusters of other classes far away. To solve the optimization problem, the concave-convex procedure (CCCP) is utilized in the proposed method. Moreover, the solution of proposed LSPTSVC involves a set of linear equations leading to very less training time. To verify the performance of the proposed algorithm, several experiments are performed on synthetic and real world benchmark datasets. Experimental results and statistical analysis show that the proposed LSPTSVC performs better than existing algorithms w.r.t. clustering accuracy as well as training time. Moreover, a comparison of the proposed method with existing algorithms is presented on biometric and biomedical applications. Better generalization performance is achieved by proposed LSPTSVC on clustering of facial images, and Alzheimer’s disease data.

Ramp-based twin support vector clustering

Traditional plane-based clustering methods measure the cost of within-cluster and between-cluster by quadratic, linear or some other unbounded functions, which may amplify the impact of cost. This letter introduces a ramp cost function into the plane-based clustering to propose a new clustering method, called ramp-based twin support vector clustering (RampTWSVC). RampTWSVC is more robust because of its boundness, and thus it is more easier to find the intrinsic clusters than other plane-based clustering methods. The non-convex programming problem in RampTWSVC is solved efficiently through an alternating iteration algorithm, and its local solution can be obtained in a finite number of iterations theoretically. In addition, the nonlinear manifold-based formation of RampTWSVC is also proposed by kernel trick. Experimental results on several benchmark datasets show the better performance of our RampTWSVC compared with other plane-based clustering methods.

Fuzzy semi-supervised weighted linear loss twin support vector clustering

In this paper, we propose two variants of Weighted Linear Loss Twin Support Vector Clustering (WLL-TWSVC) algorithm for identifying cluster planes. Unlike, Twin Support Vector Clustering (TWSVC) where the solution is obtained by solving a Quadratic Programming Problem (QPP) and a system of linear equations, WLL-TWSVC needs to solve the system of linear equations only. In order to improve the clustering accuracy with the help of limited amount of labeled data, we have extended TWSVC and WLL-TWSVC in the semi-supervised framework which are termed as Laplacian TWSVC (Lap-TWSVC) and Laplacian WLL-TWSVC (Lap-WLL-TWSVC) respectively. Further, to build a robust clustering algorithm which is not sensitive to noise and outliers, we introduce a fuzzy membership matrix and thus extends Lap-WLL-TWSVC to Fuzzy Laplacian WLL-TWSVC. The experimental results on several benchmark UCI datasets indicate that our proposed formulations achieve better clustering accuracy over other state-of-the-art plane-based clustering algorithms with comparatively lesser computational time. As an application to our proposed algorithms, we also perform image segmentation over Berkeley Segmentation dataset.

Clustering by twin support vector machine and least square twin support vector classifier with uniform output coding

The recently proposed twin support vector clustering (TWSVC) is a powerful clustering method. However, TWSVC may encounter the singularity problem and it is time consuming in the learning stage. In this paper, we introduce some efficient techniques into TWSVC, and propose two clustering models, called twin bounded support vector clustering (TBSVC) and least square twin bounded support vector clustering (LSTBSVC), respectively. TBSVC introduces a maximum margin regularization term into TWSVC, which not only avoids its singularity but also significantly improves the performance. LSTBSVC introduces the least square formation into TBSVC to greatly accelerate its learning speed. Moreover, a uniform output coding for LSTBSVC is introduced to cope with the non-uniformed problem in the learning procedures. In addition, nonlinear clustering is also extended to the above clustering methods by using the kernel trick. Experimental results show the effectiveness and efficiency of our methods.

L1-Norm Distance Minimization-Based Fast Robust Twin Support Vector $k$ -Plane Clustering.

Twin support vector clustering (TWSVC) is a recently proposed powerful k-plane clustering method. It, however, is prone to outliers due to the utilization of squared L2-norm distance. Besides, TWSVC is computationally expensive, attributing to the need of solving a series of constrained quadratic programming problems (CQPPs) in learning each clustering plane. To address these problems, this brief first develops a new k-plane clustering method called L1-norm distance minimization-based robust TWSVC by using robust L1-norm distance. To achieve this objective, we propose a novel iterative algorithm. In each iteration of the algorithm, one CQPP is solved. To speed up the computation of TWSVC and simultaneously inherit the merit of robustness, we further propose Fast RTWSVC and design an effective iterative algorithm to optimize it. Only a system of linear equations needs to be computed in each iteration. These characteristics make our methods more powerful and efficient than TWSVC. We also conduct some insightful analysis on the existence of local minimum and the convergence of the proposed algorithms. Theoretical insights and effectiveness of our methods are further supported by promising experimental results.

Tree-based localized fuzzy twin support vector clustering with square loss function

Twin support vector machine (TWSVM) is an efficient supervised learning algorithm, proposed for the classification problems. Motivated by its success, we propose Tree-based localized fuzzy twin support vector clustering (Tree-TWSVC). Tree-TWSVC is a novel clustering algorithm that builds the cluster model as a binary tree, where each node comprises of proposed TWSVM-based classifier, termed as localized fuzzy TWSVM (LF-TWSVM). The proposed clustering algorithm Tree-TWSVC has efficient learning time, achieved due to the tree structure and the formulation that leads to solving a series of systems of linear equations. Tree-TWSVC delivers good clustering accuracy because of the square loss function and it uses nearest neighbour graph based initialization method. The proposed algorithm restricts the cluster hyperplane from extending indefinitely by using cluster prototype, which further improves its accuracy. It can efficiently handle large datasets and outperforms other TWSVM-based clustering methods. In this work, we propose two implementations of Tree-TWSVC: Binary Tree-TWSVC and One-against-all Tree-TWSVC. To prove the efficacy of the proposed method, experiments are performed on a number of benchmark UCI datasets. We have also given the application of Tree-TWSVC as an image segmentation tool.

Fuzzy least squares twin support vector clustering

In this paper, we have formulated a fuzzy least squares version of recently proposed clustering method, namely twin support vector clustering (TWSVC). Here, a fuzzy membership value of each data pattern to different cluster is optimized and is further used for assigning each data pattern to one or other cluster. The formulation leads to finding k cluster center planes by solving modified primal problem of TWSVC, instead of the dual problem usually solved. We show that the solution of the proposed algorithm reduces to solving a series of system of linear equations as opposed to solving series of quadratic programming problems along with system of linear equations as in TWSVC. The experimental results on several publicly available datasets show that the proposed fuzzy least squares twin support vector clustering (F-LS-TWSVC) achieves comparable clustering accuracy to that of TWSVC with comparatively lesser computational time. Further, we have given an application of F-LS-TWSVC for segmentation of color images.

Twin support vector machine for clustering.

The twin support vector machine (TWSVM) is one of the powerful classification methods. In this brief, a TWSVM-type clustering method, called twin support vector clustering (TWSVC), is proposed. Our TWSVC includes both linear and nonlinear versions. It determines k cluster center planes by solving a series of quadratic programming problems. To make TWSVC more efficient and stable, an initialization algorithm based on the nearest neighbor graph is also suggested. The experimental results on several benchmark data sets have shown a comparable performance of our TWSVC.