Constrained Clustering Problem Research Articles

On coresets for fair clustering in metric and Euclidean spaces and their applications

Fair clustering is a constrained clustering problem where we need to partition a set of colored points. The fraction of points of each color in every cluster should be more or less equal to the fraction of points of this color in the dataset. The problem was recently introduced by Chierichetti et al. (2017) [1]. We propose a new construction of coresets for fair clustering for Euclidean and general metrics based on random sampling. For the Euclidean space Rd, we provide the first coreset whose size does not depend exponentially on the dimension d. The question of whether such constructions exist was asked by Schmidt et al. (2019) [2] and Huang et al. (2019) [5]. For general metrics, our construction provides the first coreset for fair clustering. New coresets appear to be a handy tool for designing better approximation and streaming algorithms for fair and other constrained clustering variants.

Shelved–Retrieved Method for Weakly Balanced Constrained Clustering Problems

Clustering problems are prevalent in areas such as transport and partitioning. Owing to the demand for centralized storage and limited resources, a complex variant of this problem has emerged, also referred to as the weakly balanced constrained clustering (WBCC) problem. Clusters must satisfy constraints regarding cluster weights and connectivity. However, existing methods fail to guarantee cluster connectivity in diverse scenarios, thereby resulting in additional transportation costs. In response to the aforementioned limitations, this study introduces a shelved–retrieved method. This method embeds adjacent relationships during power diagram construction to ensure cluster connectivity. Using the shelved–retrieved method, connected clusters are generated and iteratively adjusted to determine the optimal solutions. Further, experiments are conducted on three synthetic datasets, each with three objective functions, and the results are compared to those obtained using other techniques. Our method successfully generates clusters that satisfy the constraints imposed by the WBCC problem and consistently outperforms other techniques in terms of the evaluation measures.

Towards more efficient local search algorithms for constrained clustering

Constrained clustering extends clustering by integrating user constraints, and aims to determine an optimal assignment under the constraints. In this paper, we propose a local search algorithm called FastCCP to solve the constrained clustering problem. In the algorithm, instances connected by must-link constraints are first merged into nodes, and then, a local search method is performed to handle the cannot-link constraints while minimizing the Within-Cluster Sum of Squares (WCSS). Several strategies are proposed to enhance the solution diversity and achieve a trade-off between constraint satisfaction and WCSS minimization during the search. Furthermore, a node-filtering strategy is proposed to improve the efficiency of the algorithm. Experiments are performed on benchmark datasets to evaluate our algorithm. The comparative results indicate that our algorithm outperforms state-of-the-art algorithms in terms of both the solution quality and CPU runtime.

Tight FPT approximation for constrained k-center and k-supplier

In this work, we study a range of constrained versions of the k-supplier and k-center problems. In the classical (unconstrained) k-supplier problem, we are given a set of clients C in a metric space X, with distance function d(.,.). We are also given a set of feasible facility locations L⊆X. The goal is to open a set F of k facilities in L to minimize the maximum distance of any client to the closest open facility, i.e., minimize, cost(F,C)≡maxj∈C⁡{d(F,j)}, where d(F,j) is the distance of client j to the closest facility in F. The k-center problem is a special case of the k-supplier problem where L=C. We study various constrained versions of the k-supplier problem such as: capacitated, fault-tolerant, ℓ-diversity, etc. These problems fall under a broad framework of constrained clustering. A unified framework for constrained clustering was proposed by Ding and Xu [Algorithmica 2020] in the context of the k-median and k-means objectives. We extend this framework to the k-supplier and k-center objectives in this work. This unified framework allows us to obtain results simultaneously for the following constrained versions of the k-supplier problem: r-gather, r-capacity, balanced, chromatic, fault-tolerant, strongly private, ℓ-diversity, and fair k-supplier problems, with and without outliers. We design Fixed-Parameter Tractable (FPT) algorithms for these problems. FPT algorithms have polynomial running time if the parameter under consideration is a constant. This may be relevant even to a practitioner since the parameter k is a small number in many real clustering scenarios. We obtain the following results:•We give 3 and 2 approximation algorithms for the constrained k-supplier and k-center problems, respectively, with FPT running time kO(k)⋅nO(1), where n=|C∪L|. Moreover, these approximation guarantees are tight; that is, for any constant ε>0, no algorithm can achieve (3−ε) and (2−ε) approximation guarantees for the constrained k-supplier and k-center problems in FPT time, assuming FPT≠W[2].•We study the constrained clustering problem with outliers. Our algorithm gives 3 and 2 approximation guarantees for the constrained outlier k-supplier and k-center problems, respectively, with FPT running time (k+m)O(k)⋅nO(1), where n=|C∪L| and m is the number of outliers.•Our techniques generalise for distance function d(.,.)z. That is, for any positive real number z, if the cost of a client is defined by d(.,.)z instead of d(.,.), then our algorithm gives 3z and 2z approximation guarantees for the constrained k-supplier and k-center problems, respectively.

Theoretical analysis of classic and capacity constrained fuzzy clustering

In this paper we present a theoretical analysis on fuzzy centroid-based clustering methods. In addition to the formulation on the classical approaches, we consider constraints that may be useful in some practical applications, such as restrictions on the number of points in each group, and methods that deal with these constraints. We propose a more general formulation to the constrained clustering problem, where each point has an associated weight, and the sum of the weights of the points that compose each group is established a priori. For both classical and proposed approaches we discuss existence and uniqueness of solutions of the involved problems, providing mathematical foundations for the established formulas. Preliminary numerical experiments, performed by means of two-dimensional examples, are also presented.

Construction of Data Resource Sharing Platform for College Students’ Ideological and Political Education Based on Constraint Clustering

Online learning communities have changed the “silo” learning structure of traditional online learning and provided an effective support environment for wisdom sharing and collaborative knowledge building. The construction of a good online learning community has become a core issue in the field of online learning research. The study abstracts the construction of online learning community as a constrained clustering problem and proposes an intelligent construction method for online learning community based on constrained clustering. In particular, the three principles include the combination of limited freedom of choice and continuous iterative improvement, the combination of openness and standardization of the sharing platform, and the combination of public welfare and paid attributes of data resources. This paper discusses how to share valuable basic statistical data and social survey data in the field of college students’ ideological and political education (IPE) and make them public to the whole society by building a data resource sharing platform, so as to improve the utilization of data on the one hand and support comparative research topics in related fields through statistical analysis among survey data on the other hand. The experimental results show that the proportion of low-level posts (KC1 and KC2) of the two groups of students is relatively high (79.4% and 76.5%, respectively), indicating that the students in the learning community have carried out a lot of knowledge sharing and discussed and compared them to a certain extent. The proportion of high-level posts by students in both groups is very small (20.8% and 23.5%, respectively), but the number of high-level (KC3, KC4) posts by the students in the experimental group is significantly higher than that of the students in the control group, indicating that the students in the experimental group have good exchanges and discussions have resulted in meaningful negotiation and testing on some issues.

Identification of piecewise affine model for batch processes based on constrained clustering technique

In this paper, a novel identification method of piecewise affine (PWA) model for batch processes based on constrained clustering technique is proposed. In traditional clustering-based identification approaches, data classification and region partition are performed individually so that inseparable problem usually occurs in the partition phase. The proposed method uses a constrained K-means clustering algorithm to simultaneously perform both data classification and region partition, which is accomplished by imposing the complete and non-overlapping partition constraints into the clustering optimization problem. We employ a greedy iterative approach combined with the golden section search to efficiently solve the constrained clustering problem. This method can greatly improve the accuracy of the identified PWA model. Finally, we demonstrate the effectiveness of the proposed identification method.

MEACCP: A membrane evolutionary algorithm for capacitated clustering problem

The Capacitated Clustering Problem (CCP) is a constrained clustering problem with a wide range of applications in VLSI design, vehicle routing, and mail delivery. In order to improve the performance of solving CCP, this paper proposes a membrane evolutionary algorithm MEACCP based on the Membrane Evolutionary Algorithm Framework (MEAF). First, the membrane structure of MEACCP and its object representation are designed. Second, six evolution operators are designed to evolve objects within the membrane and membrane structures. Third, the MEACCP and the method of membrane population initialization are discussed. Finally, MEACCP experiments were conducted on 100 instances of four different types of benchmark datasets, and the experimental results were compared with the results of the current state-of-the-art CCP algorithms. The experimental results show that the number of instances where MEACCP can find the optimal solution is 4.3 ∼ 9.1 times that of the algorithms in comparison, and the stability is better than all comparison algorithms.

Подход к поиску глобального оптимума в задачах Constrained Clustering с привлечением оценок нескольких экспертов

An approach to solving the constrained clustering problem has been developed, based on the aggregation of data obtained as a result of evaluating the characteristics of clustered objects by several independent experts, and the analysis of alternative variants of clustering by constraint programming methods using original heuristics. Objects clusterized are represented as multisets, which makes it possible to use appropriate methods of aggregation of expert opinions. It is proposed to solve the constrained clustering problem as a constraint satisfaction problem. The main attention is paid to the issue of reducing the number and simplifying the constraints of the constraint satisfaction problem at the stage of its formalization. Within the framework of the approach, we have created: a) a method for estimating the optimal value of the objective function by hierarchical clustering of multisets, taking into account a priori constraints of the subject domain, and b) a method for generating additional constraints on the desired solution in the form of “smart tables”, based on the obtained estimate. The approach allows us to find the best partition in the problems of the class under consideration, which are characterized by a high dimension.

3SHACC: Three stages hybrid agglomerative constrained clustering

Traditionally within the unsupervised learning paradigm, hierarchical and partitional clustering techniques have been shown to produce better results when provided with partial information, leading to a renewed attention towards this topic. Constrained clustering is a semi-supervised learning problem that combines classic clustering techniques with background knowledge given in the form of a set of constraints. In this paper, we propose to incorporate constraints into the clustering process in three phases: the first phase is devoted to quantify constraint relevance and to learn a metric matrix according to such relevance, a second phase computing similarities between instances by means of the reconstruction coefficient and pairwise distances, and a third stage performing agglomerative hierarchical clustering with a reward-style stepped affinity function favoring merges satisfying the higher possible number of constraints. Experimental results, supported by Bayesian statistical testing, show a consistent improvement in favor of our proposal over previous approaches to the constrained clustering problem.

ME-MEOA/D[formula omitted]: Multiobjective constrained clustering through decomposition-based memetic elitism

Traditionally unsupervised, clustering techniques have received renewed attention recently, as they have been shown to produce better results when provided with incomplete information about the dataset in the form of constraints. Combining classic clustering and constraints leads to constrained clustering, a semi-supervised learning problem still unexplored in many aspects. Based on the exploration-exploitation requirements of constrained clustering, a memetic elitist multiobjective evolutionary algorithm based on decomposition is proposed, which combines classic multiobjective optimization strategies with single-objective optimization procedures. The application scheme of our proposal for the constrained clustering problem is scrutinized and compared to several state-of-the-art methods for 20 datasets with incremental levels of constraint-based information. Experimental results, supported by Bayesian statistical testing, show a consistent improvement in clustering and multiobjective optimization related measures in favor of our proposal over the state-of-the-art.

Enhancing instance-level constrained clustering through differential evolution

Clustering has always been a powerful tool in knowledge discovery. Traditionally unsupervised, it received renewed attention when it was shown to produce better results when provided with new types of information, thus leading to a new kind of semi-supervised learning: constrained clustering. This technique is a generalization of traditional clustering that considers additional information encoded by constraints. Constraints can be given in the form of instance-level must-link and cannot-link constraints, which this paper focuses on. We propose the first application of Differential Evolution to the constrained clustering problem, which has proven to produce a better exploration–exploitation trade-off when comparing with previous approaches. We will compare the results obtained by this proposal to those obtained by previous nature-inspired techniques and by some of the state-of-the-art algorithms on 25 datasets with incremental levels of constraint-based information, supporting our conclusions with the aid of Bayesian statistical tests.

Local search for constrained graph clustering in biological networks

Semi-supervised or constrained graph clustering incorporates prior information in order to improve clustering results. Pairwise constraints are often utilized to guide the clustering process. This work addresses a constrained graph clustering problem in biological networks where (1) subgraph connectivity constraints are strictly required to be satisfied and (2) clustering quality is assessed with respect to pairwise constraint violations. Existing constrained graph clustering methods often fail to fully satisfy connectivity constraints. This paper presents an iterated local search algorithm which aims to find the clustering with the highest quality in a short computing time. Experiments demonstrate how the proposed solutions are of good quality, often being optimal. Additionally, the proposed method significantly outperforms an existing branch-and-cut algorithm in terms of computational runtime and produces competitive results with regard to other local search techniques and graph clustering algorithms. Furthermore, a multilevel algorithm for clustering is designed to handle large-scale graphs. The performance of the overall scheme for a variety of coarsening methods from the literature is studied on a large number of biological networks exceeding 10,000 genes.

An approach to solution of constrained clustering problems using the constraint programming paradigm and the multiset theory

An integrated approach is proposed to solve the Constrained Clustering problems which are oriented on taking the opinion of a group of experts into account. A cluster and the objects to be clustered are proposed to be represented as multisets, and the distance between these to be determined by metrics in the Petrovsky multisets space. The approach is implemented within the Constraint Programming paradigm. In so doing, the significant complexity is in organizing effective processing the qualitative constraints, namely, the rules for assigning the objects to one or different clusters. The qualitative constraints are proposed to be represented and processed as table constraints of a new type, i.e., as the smart-tables of D-type. The main attention is confined to the problem of how to reduce the amount of constraints and how to simplify the Constrained Clustering problem. It is proposed to generate constraints for some pairs of objects rather than for all of them, with the generation based on a priori interval estimation of the optimal value of the clustering criterion. To do that, a modified method of multisets hierarchical clustering has been proposed. The approach proposed allows a global optimum to be found for the Constrained Clustering problems considered.

DILS: Constrained clustering through dual iterative local search

Clustering has always been a powerful tool in knowledge discovery. Traditionally unsupervised, it has received renewed attention recently as it has shown to produce better results when provided with new types of information, thus leading to a new kind of semi-supervised learning: constrained clustering. This technique is a generalization of traditional clustering that considers additional information encoded by constraints. Constraints can be given in the form of instance-level must-link and cannot-link constraints, which is the focus of this paper. We propose a new metaheuristic algorithm, the Dual Iterative Local Search, and prove its ability to produce quality results for the constrained clustering problem. We compare the results obtained by this proposal to those obtained by the state-of-the-art algorithms on 25 datasets with incremental levels of constraint-based information, supporting our conclusions with the aid of Bayesian statistical tests.

Approximation Schemes for Low-rank Binary Matrix Approximation Problems

We provide a randomized linear time approximation scheme for a generic problem about clustering of binary vectors subject to additional constraints. The new constrained clustering problem generalizes a number of problems and by solving it, we obtain the first linear time-approximation schemes for a number of well-studied fundamental problems concerning clustering of binary vectors and low-rank approximation of binary matrices. Among the problems solvable by our approach are L ow GF(2)-R ank A pproximation , L ow B oolean -R ank A pproximation , and various versions of B inary C lustering . For example, for L ow GF(2)-R ank A pproximation problem, where for an m × n binary matrix A and integer r > 0, we seek for a binary matrix B of GF(2) rank at most r such that the ℓ 0 -norm of matrix A−B is minimum, our algorithm, for any ϵ > 0 in time f ( r ,ϵ)⋅ n ⋅ m , where f is some computable function, outputs a (1+ϵ)-approximate solution with probability at least (1−1\ e ). This is the first linear time approximation scheme for these problems. We also give (deterministic) PTASes for these problems running in time n f ( r )1\ϵ 2 log 1\ϵ , where f is some function depending on the problem. Our algorithm for the constrained clustering problem is based on a novel sampling lemma, which is interesting on its own.

A Unified Framework for Clustering Constrained Data Without Locality Property

In this paper, we consider a class of constrained clustering problems of points in $$\mathbb {R}^{d}$$, where d could be rather high. A common feature of these problems is that their optimal clusterings no longer have the locality property (due to the additional constraints), which is a key property required by many algorithms for their unconstrained counterparts. To overcome the difficulty caused by the loss of locality, we present in this paper a unified framework, called Peeling-and-Enclosing, to iteratively solve two variants of the constrained clustering problems, constrained k-means clustering (k-CMeans) and constrained k-median clustering (k-CMedian). Our framework generalizes Kumar et al.’s (J ACM 57(2):5, 2010) elegant k-means clustering approach from unconstrained data to constrained data, and is based on two standalone geometric techniques, called Simplex Lemma and Weaker Simplex Lemma, for k-CMeans and k-CMedian, respectively. The simplex lemma (or weaker simplex lemma) enables us to efficiently approximate the mean (or median) point of an unknown set of points by searching a small-size grid, independent of the dimensionality of the space, in a simplex (or the surrounding region of a simplex), and thus can be used to handle high dimensional data. If k and $$\frac{1}{\epsilon }$$ are fixed numbers, our framework generates, in nearly linear time (i.e., $$O(n(\log n)^{k+1}d)$$), $$O((\log n)^{k})$$k-tuple candidates for the k mean or median points, and one of them induces a $$(1+\epsilon )$$-approximation for k-CMeans or k-CMedian, where n is the number of points. Combining this unified framework with a problem-specific selection algorithm (which determines the best k-tuple candidate), we obtain a $$(1+\epsilon )$$-approximation for each of the constrained clustering problems. Our framework improves considerably the best known results for these problems. We expect that our technique will be applicable to other variants of k-means and k-median clustering problems without locality.

A comparison of two hybrid methods for constrained clustering problems

This paper proposes two hybrid heuristics to solve the constrained clustering problem. This problem consists of partitioning a set of objects into clusters with similar members that satisfy must-link and cannot-link constraints. A must-link constraint indicates that two selected objects must be in the same cluster, and cannot-link constraint means that two selected objects must be in distinct clusters. The two proposed hybrid methods are biased random key genetic algorithm (BRKGA) with local search (LS) heuristic and column generation (CG) with path-relinking (PR) and local search (LS) heuristic. Computational experiments considering instances available in the literature are presented to demonstrate the efficacy of the proposed methods to solve the constrained clustering problem. Moreover, the results of the CG and BRKGA are compared with the CCCG, CP and CPRBBA method.

Multi-pitch Streaming of Harmonic Sound Mixtures

Multi-pitch analysis of concurrent sound sources is an important but challenging problem. It requires estimating pitch values of all harmonic sources in individual frames and streaming the pitch estimates into trajectories, each of which corresponds to a source. We address the streaming problem for monophonic sound sources. We take the original audio, plus frame-level pitch estimates from any multi-pitch estimation algorithm as inputs, and output a pitch trajectory for each source. Our approach does not require pre-training of source models from isolated recordings. Instead, it casts the problem as a constrained clustering problem, where each cluster corresponds to a source. The clustering objective is to minimize the timbre inconsistency within each cluster. We explore different timbre features for music and speech. For music, harmonic structure and a newly proposed feature called uniform discrete cepstrum (UDC) are found effective; while for speech, MFCC and UDC works well. We also show that timbre-consistency is insufficient for effective streaming. Constraints are imposed on pairs of pitch estimates according to their time-frequency relationships. We propose a new constrained clustering algorithm that satisfies as many constraints as possible while optimizing the clustering objective. We compare the proposed approach with other state-of-the-art supervised and unsupervised multi-pitch streaming approaches that are specifically designed for music or speech. Better or comparable results are shown.

Improving constrained clustering via swarm intelligence

By simulating the clustering behavior of the real-world ant colonies, we propose in this paper a constrained ant clustering algorithm. This algorithm is embedded with the heuristic walk mechanism based on random walk to deal with the constrained clustering problems given pairwise must-link and cannot-link constraints. Experimental results show that our approach is more effective on both the synthetic datasets and the real datasets compared with the Cop-Kmeans and ant-based clustering algorithm.