Cannot-link Constraints Research Articles

Pairwise constraint-guided sparsity preserving projections

Concerning the deficiency of supervision information in the process of sparse reconstruction in Sparsity Preserving Projection(SPP),Pairwise Constraint-guided Sparsity Preserving Projection(PCSPP) was proposed,which introduced supervision information of must-link constraints and cannot-link constraints to guide sparse reconstruction in the process of sparsity reconstruction of training samples,making SPP fuse constraint supervise information efficiently.The experimental results in UMIST,YALE and AR face datasets show,in contrast to unsupervised sparsity preserving projections,our algorithm achieves approximately 5%~15% increase in recognition accuracy based on the nearest neighbor classifier and promotes efficiently the performance of dimensionality reduction classification.

Learning mid-perpendicular hyperplane similarity from cannot-link constraints

Pairwise constraints known as must-link and cannot-link constraints have been frequently used in semi-supervised clustering. In this paper, we propose a novel usage of cannot-link constraints and develop a method called Mid-Perpendicular Hyperplane Similarity (MPHS) for semi-supervised clustering. Since a cannot-link constraint means that the two objects linked by it are not in the same class, there is a mid-perpendicular hyperplane to distinguish them. For each cannot-link constraint, we first compute the corresponding mid-perpendicular hyperplane and then use distances of objects to this hyperplane to learn a new data representation and similarity matrix. Finally, we combine all the similarity matrices from all cannot-link constraints into single similarity matrix and perform kernel k-means on it to obtain the partition. We implement MPHS for two cases, i.e., a simple one performed in original input space when the data set is nearly linear-separable, and an advanced one in kernel-induced feature space when the data set is complex and nonlinear-separable. Experimental results on several UCI data sets and some image data sets show the effectiveness of our method.

Community structure detection in complex networks with partial background information

Constrained clustering has been well-studied in the unsupervised learning society. However, how to encode constraints into community structure detection, within complex networks, remains a challenging problem. In this paper, we propose a semi-supervised learning framework for community structure detection. This framework implicitly encodes the must-link and cannot-link constraints by modifying the adjacency matrix of network, which can also be regarded as de-noising the consensus matrix of community structures. Our proposed method gives consideration to both the topology and the functions (background information) of complex network, which enhances the interpretability of the results. The comparisons performed on both the synthetic benchmarks and the real-world networks show that the proposed framework can significantly improve the community detection performance with few constraints, which makes it an attractive methodology in the analysis of complex networks.

An Improved Cop-Kmeans Clustering for Solving Constraint Violation Based on MapReduce Framework

Clustering with pairwise constraints has received much attention in the clustering community recently. Particularly, must-link and cannot-link constraints between a given pair of instances in the data set are common prior knowledge incorporated in many clustering algorithms today. This approach has been shown to be successful in guiding a number of famous clustering algorithms towards more accurate results. However, recent work has also shown that the incorporation of must-link and cannot-link constraints makes clustering algorithms too much sensitive to “the assignment order of instances” and therefore results in consequent constraint violation. The major contributions of this paper are two folds. One is to address the issue of constraint violation in Cop-Kmeans by emphasizing a sequenced assignment of cannot-link instances after conducting a Breadth-First Search of the cannot-link set. The other is to reduce the computational complexity of Cop-Kmeans for massive data sets by adopting a MapReduce Framework. Experimental results show that our approach performs well on massive data sets while may overcome the problem of constraint violation.

Cross-media retrieval by intra-media and inter-media correlation mining

With the rapid development of multimedia content on the Internet, cross-media retrieval has become a key problem in both research and application. Cross-media retrieval is able to retrieve the results of the same semantics with the query, but with different media types. For instance, given a query image of Moraine Lake, besides retrieving the images about Moraine Lake, cross-media retrieval system can also retrieve the related media contents of different media types such as text description. As a result, measuring content similarity between different media is a challenging problem. In this paper, we propose a novel cross-media similarity measure. It considers both intra-media and inter-media correlation, which are ignored by existing works. Intra-media correlation focuses on semantic category information within each media, while inter-media correlation focuses on positive and negative correlations between different media types. Both of them are very important and their adaptive fusion can complement each other. To mine the intra-media correlation, we propose a heterogeneous similarity measure with nearest neighbors (HSNN). The heterogeneous similarity is obtained by computing the probability for two media objects belonging to the same semantic category. To mine the inter-media correlation, we propose a cross-media correlation propagation (CMCP) approach to simultaneously deal with positive and negative correlation between media objects of different media types, while existing works focus solely on the positive correlation. Negative correlation is very important because it provides effective exclusive information. The correlations are modeled as must-link constraints and cannot-link constraints, respectively. Furthermore, our approach is able to propagate the correlation between heterogeneous modalities. Finally, both HSNN and CMCP are flexible, so that any traditional similarity measure could be incorporated. An effective ranking model is learned by further fusion of multiple similarity measures through AdaRank for cross-media retrieval. The experimental results on two datasets show the effectiveness of our proposed approach, compared with state-of-the-art methods.

Comparison of Semi-Supervised Hierarchical Clustering Using Clusterwise Tolerance

This paper presents a new semi-supervised agglomerative hierarchical clustering algorithm with the ward method using clusterwise tolerance. Semi-supervised clustering has recently been noted and studied in many research fields. Must-link and cannot-link, called pairwise constraints, are frequently used in order to improve clustering properties in semi-supervised clustering. First, clusterwise tolerance based pairwise constraints are introduced in order to handle mustlink and cannot-link constraints. Next, a new semisupervised hierarchical clustering algorithm with the ward method is constructed based on the above discussions. The effectiveness of the proposed algorithms is, moreover, verified through numerical examples.

Improving constrained clustering via swarm intelligence

By simulating the clustering behavior of the real-world ant colonies, we propose in this paper a constrained ant clustering algorithm. This algorithm is embedded with the heuristic walk mechanism based on random walk to deal with the constrained clustering problems given pairwise must-link and cannot-link constraints. Experimental results show that our approach is more effective on both the synthetic datasets and the real datasets compared with the Cop-Kmeans and ant-based clustering algorithm.

On constrained spectral clustering and its applications

Constrained clustering has been well-studied for algorithms such as $K$-means and hierarchical clustering. However, how to satisfy many constraints in these algorithmic settings has been shown to be intractable. One alternative to encode many constraints is to use spectral clustering, which remains a developing area. In this paper, we propose a flexible framework for constrained spectral clustering. In contrast to some previous efforts that implicitly encode Must-Link and Cannot-Link constraints by modifying the graph Laplacian or constraining the underlying eigenspace, we present a more natural and principled formulation, which explicitly encodes the constraints as part of a constrained optimization problem. Our method offers several practical advantages: it can encode the degree of belief in Must-Link and Cannot-Link constraints; it guarantees to lower-bound how well the given constraints are satisfied using a user-specified threshold; it can be solved deterministically in polynomial time through generalized eigendecomposition. Furthermore, by inheriting the objective function from spectral clustering and encoding the constraints explicitly, much of the existing analysis of unconstrained spectral clustering techniques remains valid for our formulation. We validate the effectiveness of our approach by empirical results on both artificial and real datasets. We also demonstrate an innovative use of encoding large number of constraints: transfer learning via constraints.

Extracting elite pairwise constraints for clustering

Semi-supervised clustering under pairwise constraints (i.e. must-links and cannot-links) has been a hot topic in the data mining community in recent years. Since pairwise constraints provided by distinct domain experts may conflict with each other, a lot of research work has been conducted to evaluate the effects of noise imposing on semi-supervised clustering. In this paper, we introduce elite pairwise constraints, including elite must-link (EML) and elite cannot-link (ECL) constraints. In contrast to traditional constraints, both EML and ECL constraints are required to be satisfied in every optimal partition (i.e. a partition with the minimum criterion function). Therefore, no conflict will be caused by those new constraints. First, we prove that it is NP-hard to obtain EML or ECL constraints. Then, a heuristic method named Limit Crossing is proposed to achieve a fraction of those new constraints. In practice, this new method can always retrieve a lot of EML or ECL constraints. To evaluate the effectiveness of Limit Crossing, multi-partition based and distance based methods are also proposed in this paper to generate faux elite pairwise constraints. Extensive experiments have been conducted on both UCI and synthetic data sets using a semi-supervised clustering algorithm named COP-KMedoids. Experimental results demonstrate that COP-KMedoids under EML and ECL constraints generated by Limit Crossing can outperform those under either faux constraints or no constraints.

On Agglomerative Hierarchical Clustering Using Clusterwise Tolerance Based Pairwise Constraints

This paper presents semi-supervised agglomerative hierarchical clustering algorithm using clusterwise tolerance based pairwise constraints. In semi-supervised clustering, pairwise constraints, that is, must-link and cannot-link, are frequently used in order to improve clustering properties. From that sense, we will propose another way named clusterwise tolerance based pairwise constraints to handle must-link and cannot-link constraints inL2-space. In addition, we will propose semi-supervised agglomerative hierarchical clustering algorithm based on it. We will, moreover, show the effectiveness of the proposed method through numerical examples.

Constrained co-clustering with non-negative matrix factorisation

Co-clustering refers to the problem of deriving sub-matrices of the data matrix by simultaneously clustering the rows (data instances) and columns (features) of the matrix. While very effective in discovering useful knowledge, many of the co-clustering algorithms adopt a completely unsupervised approach. Integration of domain knowledge can guide the co-clustering process and greatly enhance the overall performance. We propose a semi-supervised Non-negative Matrix-factorisation (SS-NMF) based framework to integrate domain knowledge in the form of must-link and cannot-link constraints. Specifically, we augment the data matrix by integrating the constraints using metric learning and then perform NMF to obtain co-clustering. Under the proposed framework, we present two approaches to integrate domain knowledge, viz. a distance metric learning approach and an information theoretic metric learning approach. Through experiments performed on real-world web service data and publicly available text datasets, we demonstrate the performance of the proposed SS-NMF based approach for data co-clustering.

Linear Semi-Supervised Dimensionality Reduction with Pairwise Constraint for Multiple Subclasses

We propose a new method, called Subclass-oriented Dimensionality Reduction with Pairwise Constraints (SODRPaC), for dimensionality reduction. In a high dimensional space, it is common that a group of data points with one class may scatter in several different groups. Current linear semi-supervised dimensionality reduction methods would fail to achieve fair performances, as they assume two data points linked by a must-link constraint are close each other, while they are likely to be located in different groups. Inspired by the above observation, we classify the must-link constraint into two categories, which are the inter-subclass must-link constraint and the intra-subclass must-link constraint, respectively. We carefully generate cannot-link constraints by using must-link constraints, and then propose a new discriminant criterion by employing the cannot-link constraints and the compactness of shared nearest neighbors. The manifold regularization is also incorporated in our dimensionality reduction framework. Extensive experiments on both synthetic and practical data sets illustrate the effectiveness of our method.

Symmetric Graph Regularized Constraint Propagation

This paper presents a novel symmetric graph regularization framework for pairwise constraint propagation. We first decompose the challenging problem of pairwise constraint propagation into a series of two-class label propagation subproblems and then deal with these subproblems by quadratic optimization with symmetric graph regularization. More importantly, we clearly show that pairwise constraint propagation is actually equivalent to solving a Lyapunov matrix equation, which is widely used in Control Theory as a standard continuous-time equation. Different from most previous constraint propagation methods that suffer from severe limitations, our method can directly be applied to multi-class problem and also can effectively exploit both must-link and cannot-link constraints. The propagated constraints are further used to adjust the similarity between data points so that they can be incorporated into subsequent clustering. The proposed method has been tested in clustering tasks on six real-life data sets and then shown to achieve significant improvements with respect to the state of the arts.

Learning low-rank kernel matrices for constrained clustering

Constrained clustering methods (that usually use must-link and/or cannot-link constraints) have been received much attention in the last decade. Recently, kernel adaptation or kernel learning has been considered as a powerful approach for constrained clustering. However, these methods usually either allow only special forms of kernels or learn non-parametric kernel matrices and scale very poorly. Therefore, they either learn a metric that has low flexibility or are applicable only on small data sets due to their high computational complexity. In this paper, we propose a more efficient non-linear metric learning method that learns a low-rank kernel matrix from must-link and cannot-link constraints and the topological structure of data. We formulate the proposed method as a trace ratio optimization problem and learn appropriate distance metrics through finding optimal low-rank kernel matrices. We solve the proposed optimization problem much more efficiently than SDP solvers. Additionally, we show that the spectral clustering methods can be considered as a special form of low-rank kernel learning methods. Extensive experiments have demonstrated the superiority of the proposed method compared to recently introduced kernel learning methods.

Semisupervised Metric Learning by Maximizing Constraint Margin

Distance-metric learning is an old problem that has been researched in the supervised-learning field for a very long time. In this paper, we consider the problem of learning a proper distance metric under the guidance of some weak supervisory information. Specifically, this information is in the form of pairwise constraints which specify whether a pair of data points is in the same class ( must-link constraints) or in different classes ( cannot-link constraints). Given those constraints, our algorithm aims to learn a distance metric under which the points with must-link constraints are pushed as close as possible, while simultaneously, the points with cannot-link constraints are pulled away as far as possible. The kernelized version of our algorithm is also derived to tackle the nonlinear problem. Moreover, since in many cases, the data objects, such as images and videos, are more naturally represented as higher order tensors than vectors, we also extend our algorithm to learn the metrics directly from the tensors. Finally, experimental results are presented to show the effectiveness of our method.

A novel hypothesis-margin based approach for feature selection with side pairwise constraints

Feature selection is an important problem for pattern classification systems. As compared to unsupervised feature selection methods, the supervised ones have better performance. However, almost all existing supervised ones use class labels as supervised information, very less work has been done for other forms of supervision information such as pairwise constraints, which specifies whether a pair of data samples belongs to the same class ( must-link constraints) or different classes ( cannot-link constraints). In reality, pairwise constraints can be easily obtained by specifying whether some pairs of examples belong to the same class or not. Therefore, a new filter method for feature selection with pairwise constraints, called Constraint Score, was proposed. Unfortunately, Constraint Score does not consider the case where only cannot-link constraints are given. Also, the conclusion ‘ must-link constraints are more important than cannot-link constraints’ given by Constraint Score algorithm needs to be further verified, since ‘ cannot-link constraints’ seems more important than ‘ must-link constraints’ from the viewpoint of hypothesis-margin or margin. In addition, like the existing supervised feature selection methods, the currently proposed hypothesis-margin based approach for feature selection, called Simba, also utilizes class labels as supervision information. In this paper, to further study the feature selection problem aiming at pairwise constraints, we introduce a novel hypothesis-margin based approach for feature selection with side pairwise constraints, called Simba - sc , which only uses cannot-link constraints as supervision information. We compare our algorithm with the well-known Constraint Score, Fisher Score and Laplacian Score algorithms. Experiments are carried out on 6 UCI data sets using three different classifiers. Experimental results show that, with a few cannot-link constraints, Simba - sc achieves similar or even higher performance than Fisher Score with full class labels on all training data, and has better or comparable performance than Constraint Score.

Composite kernels for semi-supervised clustering

A critical problem related to kernel-based methods is how to select optimal kernels. A kernel function must conform to the learning target in order to obtain meaningful results. While solutions to the problem of estimating optimal kernel functions and corresponding parameters have been proposed in a supervised setting, it remains a challenge when no labeled data are available, and all we have is a set of pairwise must-link and cannot-link constraints. In this paper, we address the problem of optimizing the kernel function using pairwise constraints for semi-supervised clustering. We propose a new optimization criterion for automatically estimating the optimal parameters of composite Gaussian kernels, directly from the data and given constraints. We combine our proposal with a semi-supervised kernel-based algorithm to demonstrate experimentally the effectiveness of our approach. The results show that our method is very effective for kernel-based semi-supervised clustering.

Semi-Supervised Canonical Correlation Analysis Algorithm

In this paper,a semi-supervised canonical correlation analysis algorithm called Semi-CCA is developed, which uses supervision information in the form of pair-wise constraints in canonical correlation analysis (CCA).In this setting,besides abundant unlabeled data examples,the domain knowledge in the form of pair-wise constraints which specify whether a pair of data examples belongs to the same class (must-link constraints) or not (cannot-link constraints) is also available.Meanwhile,the relative importance of must-link constraints and cannot-link constraints is validated.Experimental results on the artificial dataset,multiple feature database and facial database including Yale and AR show that the proposed Semi-CCA can effectively enhance the classifier performance by using only a small amount of supervision information.

Neighbourhood preserving based semi-supervised dimensionality reduction

A semi-supervised linear dimensionality reduction method based on side information and neighbourhood preserving is proposed. In this problem, only must-link constraints (pairs of instances belong to the same class) and cannot-link constraints (pairs of instances belong to different classes) are given. The proposed neighbourhood preserving based semi-supervised dimensionality reduction algorithm can not only preserve the must-link and cannot-link constraints but can preserve the local structure of the input data in the low dimensional embedding subspace. Experimental results on several datasets demonstrate the effectiveness of the method.

Constraint Score: A new filter method for feature selection with pairwise constraints

Feature selection is an important preprocessing step in mining high-dimensional data. Generally, supervised feature selection methods with supervision information are superior to unsupervised ones without supervision information. In the literature, nearly all existing supervised feature selection methods use class labels as supervision information. In this paper, we propose to use another form of supervision information for feature selection, i.e. pairwise constraints, which specifies whether a pair of data samples belong to the same class ( must-link constraints) or different classes ( cannot-link constraints). Pairwise constraints arise naturally in many tasks and are more practical and inexpensive than class labels. This topic has not yet been addressed in feature selection research. We call our pairwise constraints guided feature selection algorithm as Constraint Score and compare it with the well-known Fisher Score and Laplacian Score algorithms. Experiments are carried out on several high-dimensional UCI and face data sets. Experimental results show that, with very few pairwise constraints, Constraint Score achieves similar or even higher performance than Fisher Score with full class labels on the whole training data, and significantly outperforms Laplacian Score.