Non-convex Clusters Research Articles

Constrained Density-Based Spatial Clustering of Applications with Noise (DBSCAN) using hyperparameter optimization

This article proposes a hyperparameter optimization method for density-based spatial clustering of applications with noise (DBSCAN) with constraints, termed HC-DBSCAN. While DBSCAN is effective at creating non-convex clusters, it cannot limit the number of clusters. This limitation is difficult to address with simple adjustments or heuristic methods. We approach constrained DBSCAN as an optimization problem and solve it using a customized alternating direction method of multipliers Bayesian optimization (ADMMBO). Our custom ADMMBO enables HC-DBSCAN to reuse clustering results for enhanced computational efficiency, handle integer-valued parameters, and incorporate constraint functions that account for the degree of violations to improve clustering performance. Furthermore, we propose an evaluation metric, penalized Davies–Bouldin score, with a computational cost of O(N). This metric aims to mitigate the high computational cost associated with existing metrics and efficiently manage noise instances in DBSCAN. Numerical experiments demonstrate that HC-DBSCAN, equipped with the proposed metric, generates high-quality non-convex clusters and outperforms benchmark methods on both simulated and real datasets.

Improving the utility of differentially private clustering through dynamical processing

This study aims to alleviate the trade-off between utility and privacy of differentially private clustering. Existing works focus on simple methods, which show poor performance for non-convex clusters. To fit complex cluster distributions, we propose sophisticated dynamical processing inspired by Morse theory, with which we hierarchically connect the Gaussian sub-clusters obtained through existing methods. Our theoretical results imply that the proposed dynamical processing introduces little to no additional privacy loss. Experiments show that our framework can improve the clustering performance of existing methods at the same privacy level.

Unified Framework for Faster Clustering via Joint Schatten p-Norm Factorization With Optimal Mean.

To enhance the effectiveness and efficiency of subspace clustering in visual tasks, this work introduces a novel approach that automatically eliminates the optimal mean, which is embedded in the subspace clustering framework of low-rank representation (LRR) methods, along with the computationally factored formulation of Schatten p -norm. By addressing the issues related to meaningful computations involved in some LRR methods and overcoming biased estimation of the low-rank solver, we propose faster nonconvex subspace clustering methods through joint Schatten p -norm factorization with optimal mean (JS p NFOM), forming a unified framework for enhancing performance while reducing time consumption. The proposed approach employs tractable and scalable factor techniques, which effectively address the disadvantages of higher computational complexity, particularly when dealing with large-scale coefficient matrices. The resulting nonconvex minimization problems are reformulated and further iteratively optimized by multivariate weighting algorithms, eliminating the need for singular value decomposition (SVD) computations in the developed iteration procedures. Moreover, each subproblem can be guaranteed to obtain the closed-form solver, respectively. The theoretical analyses of convergence properties and computational complexity further support the applicability of the proposed methods in real-world scenarios. Finally, comprehensive experimental results demonstrate the effectiveness and efficiency of the proposed nonconvex clustering approaches compared to existing state-of-the-art methods on several publicly available databases. The demonstrated improvements highlight the practical significance of our work in subspace clustering tasks for visual data analysis. The source code for the proposed algorithms is publicly accessible at https://github.com/ZhangHengMin/TRANSUFFC.

Intelligent mapping of geochemical anomalies: Adaptation of DBSCAN and mean-shift clustering approaches

Cluster analysis can be used to organize samples and generate ideas regarding the multivariate geochemistry of given dataset. Traditional clustering techniques have the drawbacks of high computational complexity and poor adaptability to big data. Hence, there has recently been much focus on creating better clustering algorithms. Although many clustering algorithms have been applied, and some produce notable clustering results, the performance efficiency of algorithms is often highly dependent on the values the user chooses for the parameters. Currently, density-based spatial clustering of applications with noise clustering (DBSCAN) is widely utilized in image processing, bioinformatics, and social network analysis owing to its ability to detect clusters of various shapes. Even though partitional clustering techniques may be effective when the number of clusters K is known in advance, they cannot implement non-convex clustering and rapidly return to a local optimum. This study proposes the concept of DBSCAN clustering for stream sediment geochemical data. In this respect, the geochemical data collected from Varcheh district, SW Iran, were processed using the clr transformation before applying DBSCAN. Then, PCA was used to minimize the dimension of variables and specify the mineralization-related elements. In the following, one of the PCs connected with mineralization (PC2) was chosen for further analysis. DBSCAB, Mean-shift and Fuzzy K-means algorithms were used to monitor the multi-element geochemical anomalies linked to MVT PbZn deposits in the study area. According to Davies-Bouldin and Silhouette as two validation metrics, it can be deduced that the three SCB models are advantageous, however, the model generated by DBSCAN is preferable to the model generated by Mean-shift and Fuzzy K-means.

An internal validity index for arbitrarily shaped clusters

We examine in this paper some internal CVIs (cluster validity indices) especially designed for the validation of arbitrarily shaped clusters, for example nonconvex clusters or clusters that nearly touch each other or are embedded into other clusters. They are based on the identification of multi-representative points for each cluster and on density considerations. In general, they target clusters characterized by cores of high density surrounded by regions of low density. Such a characterization is exploited to evaluate the separation among clusters, but can be a serious limitation for example when the clusters have high density regions in peripheral positions close to individual borders or have internal regions of non uniform density. Among the CVIs taken into consideration, we especially single out the SSDD index introduced in Liang et al. (2020) and propose some modifications for extending its applicability field. A numerical experimentation on both artificial and real-world datasets has been performed, confirming the effectiveness of the proposed modified index with respect to SSDD index and to other multi-representative CVIs described in literature.

An Effective Clustering Optimization Method for Unsupervised Linear Discriminant Analysis

The recent work Unsupervised Linear Discriminant Analysis (Un-LDA) completes its clustering process during the alternating optimization by converting equivalently the objective and finally using the K-means algorithm. However, the K-means algorithm has its inherent drawbacks. It is hard for the K-means algorithm to deal well with some complex clustering cases where there are too many real clusters or non-convex clusters. In this paper, a novel clustering optimization method is presented to accomplish the clustering process in Un-LDA and the resulting method can be named Un-LDA(CD). Specifically, instead of the K-means algorithm, an elaborately designed coordinate descent algorithm is adopted to obtain the clusters after the objective function goes through a series of simple but deft equivalent conversions. Extensive experiments have demonstrated that the coordinate descent clustering solution for Un-LDA can outperform the original K-means based solution on the tested data sets especially those complex data sets with a pretty large number of real clusters.

A two-phase strategy for nonconvex clusters integrating a spectral clustering with a merging technique

In this paper we address the problem of arbitrarily shaped clustering of points belonging to a linear space. Among the several approaches to clustering proposed in literature, the spectral clustering has become more and more popular for the case of arbitrarily shaped clusters. It depends on parameters whose choice can be critical. In order to reduce these criticalities, in this paper we propose a two-phase strategy that integrates the partitioning obtained by the spectral clustering with a merging technique which makes use only of the information already produced in the first phase by the spectral clustering. The main novelty of this approach is precisely the fact that no geometric tool is exploited in the merging phase. A numerical experimentation on artificial and real-world datasets has been performed to compare the proposed method with the spectral clustering and with two other widely used algorithms for non-convex problems, namely DBSCAN and Chameleon 2.

Spectral Clustering of Mixed-Type Data

Cluster analysis seeks to assign objects with similar characteristics into groups called clusters so that objects within a group are similar to each other and dissimilar to objects in other groups. Spectral clustering has been shown to perform well in different scenarios on continuous data: it can detect convex and non-convex clusters, and can detect overlapping clusters. However, the constraint on continuous data can be limiting in real applications where data are often of mixed-type, i.e., data that contains both continuous and categorical features. This paper looks at extending spectral clustering to mixed-type data. The new method replaces the Euclidean-based similarity distance used in conventional spectral clustering with different dissimilarity measures for continuous and categorical variables. A global dissimilarity measure is than computed using a weighted sum, and a Gaussian kernel is used to convert the dissimilarity matrix into a similarity matrix. The new method includes an automatic tuning of the variable weight and kernel parameter. The performance of spectral clustering in different scenarios is compared with that of two state-of-the-art mixed-type data clustering methods, k-prototypes and KAMILA, using several simulated and real data sets.

Effective density-based clustering algorithms for incomplete data

Density-based clustering is an important category among clustering algorithms. In real applications, many datasets suffer from incompleteness. Traditional imputation technologies or other techniques for handling missing values are not suitable for density-based clustering and decrease clustering result quality. To avoid these problems, we develop a novel density-based clustering approach for incomplete data based on Bayesian theory, which conducts imputation and clustering concurrently and makes use of intermediate clustering results. To avoid the impact of low-density areas inside non-convex clusters, we introduce a local imputation clustering algorithm, which aims to impute points to high-density local areas. The performances of the proposed algorithms are evaluated using ten synthetic datasets and five real-world datasets with induced missing values. The experimental results show the effectiveness of the proposed algorithms.

Application of the novel harmony search optimization algorithm for DBSCAN clustering

At present, the DBSCAN clustering algorithm has been commonly used principally due to its ability in discovering clusters with arbitrary shapes. When the cluster number K is predefined, though the partitional clustering methods can perform efficiently, they cannot process the non-convex clustering and easily fall into local optimum. Thereby the concept of K-DBSCAN clustering is proposed in this paper. But the basic DBSCAN has a crucial defect, that is, difficult to predict the suitable clustering parameters. Here, the well-known harmony search (HS) optimization algorithm is considered to deal with this problem. By modifying the original HS, the novel harmony search (novel-HS) is put forward, which can improve the accuracy of results as well as enhance the robustness of optimization. In K-DBSCAN, the novel-HS is used to optimize the clustering parameters of DBSCAN to obtain better clustering effect with the number of K classifications. Experimental results show that the designed clustering method has superior performance to others and can be successfully considered as a new clustering scheme for further research.

KNN-SC: Novel Spectral Clustering Algorithm Using k-Nearest Neighbors

Spectral clustering is a well-known graph-theoretic clustering algorithm. Although spectral clustering has several desirable advantages (such as the capability of discovering non-convex clusters and applicability to any data type), it often leads to incorrect clustering results because of high sensitivity to noise points. In this study, we propose a robust spectral clustering algorithm known as KNN-SC that can discover exact clusters by decreasing the influence of noise points. To achieve this goal, we present a novel approach that filters out potential noise points by estimating the density difference between data points using $k$ -nearest neighbors. In addition, we introduce a novel method for generating a similarity graph in which various densities of data points are effectively represented by expanding the nearest neighbor graph. Experimental results on synthetic and real-world datasets demonstrate that KNN-SC achieves significant performance improvement over many state-of-the-art spectral clustering algorithms.

Relax-and-split method for nonconvex inverse problems

We develop and analyze a new ‘relax-and-split’ (RS) approach for inverse problems modeled using nonsmooth nonconvex optimization formulations. RS uses a relaxation technique together with partial minimization, and brings classic techniques including direct factorization, matrix decompositions, and fast iterative methods to bear on nonsmooth nonconvex problems. We also extend the approach to robustify any such inverse problem through trimming, a mechanism that robustifies inverse problems to measurement outliers. We then show practical performance of RS and trimmed RS (TRS) on a diverse set of problems, including: (1) phase retrieval, (2) semi-supervised classification, (3) stochastic shortest path problems, and (4) nonconvex clustering. RS/TRS are easy to implement, competitive with existing methods, and show promising results on difficult inverse problems with nonsmooth and nonconvex features.

A methodology for automatic parameter-tuning and center selection in density-peak clustering methods

The density-peak clustering algorithm, which we refer to as DPC, is a novel and efficient density-based clustering approach. The method has the advantage of allowing non-convex clusters, and clusters of variable size and density, to be grouped together, but it also has some limitations, such as the visual location of centers and the parameter tuning. This paper describes an optimization-based methodology for automatic parameter/center selection applicable both to the DPC and to other algorithms derived from it. The objective function is an internal/external cluster validity index, and the decisions are the parameterization of the algorithm and the choice of centers. The internal validation measures lead to an automatic parameter-tuning process, and the external validation measures lead to the so-called optimal rules, which are a tool to bound the performance of a given algorithm from above on the set of parameterizations. A numerical experiment with real data was performed for the DPC and for the fuzzy weighted k-nearest neighbor (FKNN-DPC) which validates the automatic parameter-tuning methodology and demonstrates its efficiency compared to the state of the art.

Mechanism and a new algorithm for nonconvex clustering

ABSTRACTNonconvex clustering is based on over-parameterization and uses a shrinkage term to achieve sparsity in the differences among individual centroids. In this paper, we elaborate the mechanism of nonconvex clustering from a new perspective by reformulating the basic alternating direction method of multipliers (ADMM) algorithm, and explain for the first time how nonconvex clustering can identify outliers that are far from the nearest cluster centre [Knorr EM, Ng RT. Algorithms for mining distance-based outliers in large datasets. Proceedings of the 24th VLDB Conference; New York, USA: 1998. p. 392–403]. To overcome the computational burden owing to complexity, inspired by the mechanism of nonconvex clustering, we develop another novel and accelerating algorithm called aBlock_ADMM, which combines adaptive block clustering and the ADMM algorithm. We evaluate the clustering performance of the new algorithm on both simulated and real data examples.

Forward-stagewise clustering: An algorithm for convex clustering

This paper proposes an exceptionally simple algorithm, called forward-stagewise clustering, for convex clustering. Convex clustering has drawn recent attention since it nicely addresses the instability issue of traditional non-convex clustering methods. While existing algorithms can precisely solve convex clustering problems, they are sophisticated and produce (agglomerative) clustering paths that contain splits. This motivates us to propose an algorithm that only produces no-split clustering paths. The approach undertaken here follows the line of research initiated in the area of regression. Specifically, we apply the forward-stagewise technique to clustering problems and prove that the algorithm can only produce no-split clustering paths. We then modify the forward-stagewise clustering algorithm to deal with noise and outliers. We further suggest rules of thumb for the algorithm to be applicable to cases where clusters are non-convex. The performance of the proposed algorithm is evaluated through simulations and a real data application.

Approximate spectral clustering density-based similarity for noisy datasets

Approximate spectral clustering (ASC) was developed to overcome heavy computational demands of spectral clustering (SC). It maintains SC ability in predicting non-convex clusters. Since it involves a preprocessing step, ASC defines new similarity measures to assign weights on graph edges. Connectivity matrix (CONN) is an efficient similarity measure to construct graphs for ASC. It defines the weight between two vertices as the number of points assigned to them during vector quantization training. However, this relationship is undirected, where it is not clear which of the vertices is contributing more to that edge. Also, CONN could be tricked by noisy density between clusters. We defined a directed version of CONN, named DCONN, to get insights on vertices contributions to edges. Also, we provided filtering schemes to ensure CONN edges are highlighting potential clusters. Experiments reveal that the proposed filtering was highly efficient when noise cannot be tolerated by CONN.

The SpectACl of Nonconvex Clustering: A Spectral Approach to Density-Based Clustering

When it comes to clustering nonconvex shapes, two paradigms are used to find the most suitable clustering: minimum cut and maximum density. The most popular algorithms incorporating these paradigms are Spectral Clustering and DBSCAN. Both paradigms have their pros and cons. While minimum cut clusterings are sensitive to noise, density-based clusterings have trouble handling clusters with varying densities. In this paper, we propose SPECTACL: a method combining the advantages of both approaches, while solving the two mentioned drawbacks. Our method is easy to implement, such as Spectral Clustering, and theoretically founded to optimize a proposed density criterion of clusterings. Through experiments on synthetic and real-world data, we demonstrate that our approach provides robust and reliable clusterings.

Non‐alternating stochastic K ‐means based on probabilistic representation of solution space

A non-alternating (NA) form of K-means is proposed to improve the performance for the most fundamental, yet highly non-convex clustering problem. The motivation of this Letter is that the non-convex nature of K-means can be better handled by stochastic optimisation. However, the alternating update of the Lloyd's algorithm prohibits an effective stochastic optimisation. In order to fully realise the idea, a probabilistic representation is provided for the solution space of K-means, which leads to a simple yet efficient NA update. Experiments show that the proposed method outperforms the existing variants, especially for large numbers of clusters, with reasonable time complexity.

A General Hybrid Clustering Technique

Here, we propose a clustering technique for general clustering problems including those that have nonconvex clusters. For a given desired number of clusters K, we use three stages to find clusters. The first stage uses a hybrid clustering technique to produce a series of clusterings of various sizes (randomly selected). The key step in this stage is to find a K-means clustering using clusters where and then join these small clusters by using single linkage clustering. The second stage stabilizes the result of stage one by reclustering via the “membership matrix” under Hamming distance to generate a dendrogram. The third stage is to cut the dendrogram to get clusters where and then prune back to K to give a final clustering. A variant on our technique also gives a reasonable estimate for KT, the true number of clusters. We provide arguments to justify the steps in the stages of our methods and we provide examples involving simulated and published data to compare our technique with other techniques. An R library, GHC, implementing our method is available from Github.

Fixed-order ℋ∞ controller design for MIMO systems via polynomial approach

This paper presents a fixed-order [Formula: see text]∞ controller design based on linear matrix inequalities for multi-input–multi-output systems. The main difficulty in the development of a fixed-order controller design is that the associated solution set of the problem is defined in a non-convex cluster, and that makes the problem computationally intractable. The convex inner approximation is used to deal with this non-convexity. The proposed controller design approach is applied to some elegant numerical problems taken from various previous works. To show the effectiveness of the proposed method, the full-order [Formula: see text]∞ controller and fixed-order controllers are constructed for these models using the traditional method and popular toolboxes, respectively. Furthermore, in this paper, some strategies for choosing the central polynomial, which is the main conservatism of the proposed method, are discussed.