Clustering Objective Function Research Articles

A robust multi-view knowledge transfer-based rough fuzzy C-means clustering algorithm

Rough fuzzy clustering algorithms have received extensive attention due to the excellent ability to handle overlapping and uncertainty of data. However, existing rough fuzzy clustering algorithms generally consider single view clustering, which neglects the clustering requirements of multiple views and results in the failure to identify diverse data structures in practical applications. In addition, rough fuzzy clustering algorithms are always sensitive to the initialized cluster centers and easily fall into local optimum. To solve the above problems, the multi-view and transfer learning are introduced into rough fuzzy clustering and a robust multi-view knowledge transfer-based rough fuzzy c-means clustering algorithm (MKT-RFCCA) is proposed in this paper. First, multiple distance metrics are adopted as multiple views to effectively recognize different data structures, and thus positively contribute to clustering. Second, a novel multi-view transfer-based rough fuzzy clustering objective function is constructed by using fuzzy memberships as transfer knowledge. This objective function can fully explore and utilize the potential information between multiple views and characterize the uncertainty information. Then, combining the statistical information of color histograms, an initialized centroids selection strategy is presented for image segmentation to overcome the instability and sensitivity caused by the random distribution of the initialized cluster centers. Finally, to reduce manual intervention, a distance-based adaptive threshold determination mechanism is designed to determine the threshold parameter for dividing the lower approximation and boundary region of rough fuzzy clusters during the iteration process. Experiments on synthetic datasets, real-world datasets, and noise-contaminated Berkeley and Weizmann images show that MKT-RFCCA obtains favorable clustering results. Especially, it provides satisfactory segmentation results on images with different types of noise and preserves more specific detail information of images.

Spectral clustering with linear embedding: A discrete clustering method for large-scale data

In recent decades, spectral clustering has found widespread applications in various real-world scenarios, showcasing its effectiveness. Traditional spectral clustering typically follows a two-step procedure to address the optimization problem. However, this approach may result in substantial information loss and performance decline. Furthermore, the eigenvalue decomposition, a key step in spectral clustering, entails cubic computational complexity. This paper incorporates linear embedding into the objective function of spectral clustering and proposes a direct method to solve the indicator matrix. Moreover, our method achieves a linear time complexity with respect to the input data size. Our method, referred to as Spectral Clustering with Linear Embedding (SCLE), achieves a direct and efficient solution and naturally handles out-of-sample data. SCLE initiates the process with balanced and hierarchical K-means, effectively partitioning the input data into balanced clusters. After generating anchors, we compute a similarity matrix based on the distances between the input data points and the generated anchors. In contrast to the conventional two-step spectral clustering approach, we directly solve the cluster indicator matrix at a linear time complexity. Extensive experiments across multiple datasets underscore the efficiency and effectiveness of our proposed SCLE method.

Attention guided multi-scale cluster refinement with extended field of view for amodal nuclei segmentation

Nuclei segmentation plays a crucial role in disease understanding and diagnosis. In whole slide images, cell nuclei often appear overlapping and densely packed with ambiguous boundaries due to the underlying 3D structure of histopathology samples. Instance segmentation via deep neural networks with object clustering is able to detect individual segments in crowded nuclei but suffers from a limited field of view, and does not support amodal segmentation. In this work, we introduce a dense feature pyramid network with a feature mixing module to increase the field of view of the segmentation model while keeping pixel-level details. We also improve the model output quality by adding a multi-scale self-attention guided refinement module that sequentially adjusts predictions as resolution increases. Finally, we enable clusters to share pixels by separating the instance clustering objective function from other pixel-related tasks, and introduce supervision to occluded areas to guide the learning process. For evaluation of amodal nuclear segmentation, we also update prior metrics used in common modal segmentation to allow the evaluation of overlapping masks and mitigate over-penalization issues via a novel unique matching algorithm. Our experiments demonstrate consistent performance across multiple datasets with significantly improved segmentation quality.

Insider Threat Detection Based on Deep Clustering of Multi-Source Behavioral Events

With the continuous advancement of enterprise digitization, insider threats have become one of the primary cybersecurity concerns for organizations. Therefore, it is of great significance to develop an effective insider threat detection mechanism to ensure the security of enterprises. Most methods rely on artificial feature engineering and input the extracted user behavior features into a clustering-based unsupervised machine learning model for insider threat detection. However, feature extraction is independent of clustering-based unsupervised machine learning. As a result, user behavior features are not the most appropriate for clustering-based unsupervised machine learning, and thus, they reduce the insider threat detection accuracy. This paper proposes an insider threat detection method based on the deep clustering of multi-source behavioral events. On the one hand, the proposed method constructs an end-to-end deep clustering network and automatically learns the user behavior feature expression from multi-source behavioral event sequences. On the other hand, a deep clustering objective function is presented to jointly optimize the learning of feature representations and the clustering task for insider threat detection. This optimization can adjust the optimal user behavior features for the clustering model to improve the insider threat detection accuracy. The experimental results show that the proposed end-to-end insider threat detection model can accurately identify insider threats based on abnormal multi-source user behaviors in enterprise networks.

Double-Constrained Consensus Clustering with Application to Online Anti-Counterfeiting

Semi-supervised consensus clustering is a promising strategy to compensate for the subjectivity of clustering and its sensitivity to design factors, with various techniques being recently proposed to integrate domain knowledge and multiple clustering partitions. In this article, we present a new approach that makes double use of domain knowledge, namely to build the initial partitions, as well as to combine them. In particular, we show how to model and integrate must-link and cannot-link constraints into the objective function of a generic consensus clustering (CC) framework that maximizes the similarity between the consensus partition and the input partitions, which have, in turn, been enriched with the same constraints. In addition, borrowing from the theory of functional dependencies, the integrated framework exploits the notions of deductive closure and minimal cover to take full advantage of the logical implication between constraints. Using standard UCI benchmarks, we found that the resulting algorithm, termed CCC double-constrained consensus clustering), was more effective than plain CC at combining base-constrained partitions, with an average performance improvement of 5.54%. We then argue that CCC is especially well-suited for profiling counterfeit e-commerce websites, as constraints can be acquired by leveraging specific domain features, and demonstrate its potential for detecting affiliate marketing programs. Taken together, our experiments suggest that CCC makes the process of clustering more robust and able to withstand changes in clustering algorithms, datasets, and features, with a remarkable improvement in average performance.

An Optimization-Based Order-and-Cut Approach for Fair Clustering of Data Sets

Machine learning algorithms have been increasingly integrated into applications that significantly affect human lives. This surged an interest in designing algorithms that train machine learning models to minimize training error and imposing a certain level of fairness. In this paper, we consider the problem of fair clustering of data sets. In particular, given a set of items each associated with a vector of nonsensitive attribute values and a categorical sensitive attribute (e.g., gender, race, etc.), our goal is to find a clustering of the items that minimizes the loss (i.e., clustering objective) function and imposes fairness measured by Rényi correlation. We propose an efficient and scalable in-processing algorithm, driven by findings from the field of combinatorial optimization, that heuristically solves the underlying optimization problem and allows for regulating the trade-off between clustering quality and fairness. The approach does not restrict the analysis to a specific loss function, but instead considers a more general form that satisfies certain desirable properties. This broadens the scope of the algorithm’s applicability. We demonstrate the effectiveness of the algorithm for the specific case of k-means clustering as it is one of the most extensively studied and widely adopted clustering schemes. Our numerical experiments reveal the proposed algorithm significantly outperforms existing methods by providing a more effective mechanism to regulate the trade-off between loss and fairness. History: Rema Padman served as the senior editor for this article. Data Ethics & Reproducibility Note: The code capsule is available on Code Ocean at https://codeocean.com/capsule/9556728/tree and in the e-Companion to the this article (available at https://doi.org/10.1287/ijds.2022.0005 ).

Deep discriminative clustering and structural constraint for cross-domain fault diagnosis of rotating machinery

With the rapid development of intelligent manufacturing, fault diagnostic methods based on deep learning have achieved impressive results. However, most methods require plentiful annotated samples and are based on the assumption that data from the source and target domains has the same distribution. These two conditions are difficult to satisfy in practical engineering. In light of these problems, an unsupervised domain adaptation approach named Deep Discriminative Clustering network with Structural Constraint (DDCSC) is proposed in this article. In our method, a Convolutional Neural Network (CNN) module is exploited for learning feature representations of raw data. Then a softmax module is employed to simultaneously predict class probabilities and cluster assignments of the source and target data, respectively. The learnable cluster centroids are introduced into the latent feature space to alleviate the data distribution discrepancy while better capturing the discriminative structure of the target data. In addition, geometric properties of the source data in a feature space are constrained to expand the scope of each category, which facilitates to improve prediction accuracy. An information-theoretic metric is considered as the objective function of discriminative clustering. Diagnostic experiments on a rolling bearing dataset demonstrate that our approach outperforms other popular intelligent approaches and confirms the effectiveness of discriminative clustering.

K-Plus anticlustering: An improved k-means criterion for maximizing between-group similarity.

Anticlustering refers to the process of partitioning elements into disjoint groups with the goal of obtaining high between-group similarity and high within-group heterogeneity. Anticlustering thereby reverses the logic of its better known twin-cluster analysis-and is usually approached by maximizing instead of minimizing a clustering objective function. This paper presents k-plus, an extension of the classical k-means objective of maximizing between-group similarity in anticlustering applications. K-plus represents between-group similarity as discrepancy in distribution moments (means, variance, and higher-order moments), whereas the k-means criterion only reflects group differences with regard to means. While constituting a new criterion for anticlustering, it is shown that k-plus anticlustering can be implemented by optimizing the original k-means criterion after the input data have been augmented with additional variables. A computer simulation and practical examples show that k-plus anticlustering achieves high between-group similarity with regard to multiple objectives. In particular, optimizing between-group similarity with regard to variances usually does not compromise similarity with regard to means; the k-plus extension is therefore generally preferred over classical k-means anticlustering. Examples are given on how k-plus anticlustering can be applied to real norming data using the open source R package anticlust, which is freely available via CRAN.

A Differentiable Perspective for Multi-View Spectral Clustering With Flexible Extension.

Multi-view clustering aims to discover common patterns from multi-source data, whose generality is remarkable. Compared with traditional methods, deep learning methods are data-driven and have a larger search space for solutions, which may find a better solution to the problem. In addition, more considerations can be introduced by loss functions, so deep models are highly reusable. However, compared with deep learning methods, traditional methods have better interpretability, whose optimization is relatively stable. In this paper, we propose a multi-view spectral clustering model, combining the advantages of traditional methods and deep learning methods. Specifically, we start with the objective function of traditional spectral clustering, perform multi-view extension, and then obtain the traditional optimization process. By partially parameterizing this process, we further design corresponding differentiable modules, and finally construct a complete network structure. The model is interpretable and extensible to a certain extent. Experiments show that the model performs better than other multi-view clustering algorithms, and its semi-supervised classification extension also has excellent performance compared to other algorithms. Further experiments also show the stability and fewer iterations of the model training.

SKIFF: Spherical K-means with iterative feature filtering for text document clustering

Text clustering has been an overlooked field of text mining that requires more attention. Several applications require automatic text organisation which relies on an information retrieval system based on organised search results. Spherical k-means is a successful adaptation of the classic k-means algorithm for text clustering. However, conventional methods to accelerate k-means may not apply to spherical k-means due to the different nature of text document data. The proposed work introduces an iterative feature filtering technique that reduces the data size during the process of clustering which further produces more feature-relevant clusters in less time compared to classic spherical k-means. The novelty of the proposed method is that feature assessment is distinct from the objective function of clustering and derived from the cluster structure. Experimental results show that the proposed scheme achieves computation speed without sacrificing cluster quality over popular text corpora. The demonstrated results are satisfactory and outperform compared to recent works in this domain.

Self-constrained Spectral Clustering.

As a leading graph clustering technique, spectral clustering is one of the most widely used clustering methods to capture complex clusters in data. Some additional prior information can help it to further reduce the difference between its clustering results and users' expectations. However, it is hard to get the prior information under unsupervised scene to guide the clustering process. To solve this problem, we propose a self-constrained spectral clustering algorithm. In this algorithm, we extend the objective function of spectral clustering by adding pairwise and label self-constrained terms to it. We provide the theoretical analysis to show the roles of the self-constrained terms and the extensibility of the proposed algorithm. Based on the new objective function, we build an optimization model for self-constrained spectral clustering so that we can simultaneously learn the clustering results and constraints. Furthermore, we propose an iterative method to solve the new optimization problem. Compared to other existing versions of spectral clustering algorithms, the new algorithm can discover a high-quality cluster structure of a data set without prior information. Extensive experiments on benchmark data sets illustrate the effectiveness of the proposed algorithm.

A self-training subspace clustering algorithm based on adaptive confidence for gene expression data.

Gene clustering is one of the important techniques to identify co-expressed gene groups from gene expression data, which provides a powerful tool for investigating functional relationships of genes in biological process. Self-training is a kind of important semi-supervised learning method and has exhibited good performance on gene clustering problem. However, the self-training process inevitably suffers from mislabeling, the accumulation of which will lead to the degradation of semi-supervised learning performance of gene expression data. To solve the problem, this paper proposes a self-training subspace clustering algorithm based on adaptive confidence for gene expression data (SSCAC), which combines the low-rank representation of gene expression data and adaptive adjustment of label confidence to better guide the partition of unlabeled data. The superiority of the proposed SSCAC algorithm is mainly reflected in the following aspects. 1) In order to improve the discriminative property of gene expression data, the low-rank representation with distance penalty is used to mine the potential subspace structure of data. 2) Considering the problem of mislabeling in self-training, a semi-supervised clustering objective function with label confidence is proposed, and a self-training subspace clustering framework is constructed on this basis. 3) In order to mitigate the negative impact of mislabeled data, an adaptive adjustment strategy based on gravitational search algorithm is proposed for label confidence. Compared with a variety of state-of-the-art unsupervised and semi-supervised learning algorithms, the SSCAC algorithm has demonstrated its superiority through extensive experiments on two benchmark gene expression datasets.

A new deep clustering method with application to customer selection for demand response program

Demand response (DR) is regarded as a promising solution to the problem of renewable energy integration, while it remains one of the key barriers for DR to target the right customers, i.e. the customers with high potential in peak reduction. Exploring the electricity consumption behaviors of customers can help DR program practitioners evaluate customers’ peak reduction potential. Clustering has been the most common approach to describe customers’ consumption behaviors in the literature. However, the existing clustering methods suffer from either unscalability or poor performance when handling a large number of daily load curves. To overcome these issues, a new deep learning-based clustering method is proposed in this paper, which integrates both intra-cluster compactness and inter-cluster separation into the objective function of clustering for consideration. Based on the clustering results, a new customer selection framework is developed to identify the potential good candidates for DR programs, which takes the stability of consumption behaviors and the characteristics at peak hours into account. Case studies on a London dataset demonstrate the superiority of the proposed clustering method over classical K-means and two state-of-the-art deep clustering methods in terms of clustering validity indexes. Furthermore, the simulations performed on an actual price-based pilot show that the customers selected by the proposed customer selection framework contribute to 59 % of total peak reduction with only accounting for 23 % of population.

Study on Intelligent Analysis and Processing Technology of Computer Big Data Based on Clustering Algorithm

Aim: Clustering belongs to unsupervised learning, which divides the data objects into the data set into multiple clusters or classes, so that the objects in the same cluster have high similarity Background: The clustering of spatial data objects can be solved by optimization based on the clustering objective function. Objective: Study on intelligent analysis and processing technology of computer big data based on clustering algorithm. Methods: First, a new dynamic self-organizing feature mapping model is proposed, and the training algorithm of the model is given. Then, the spectral clustering technology and related concepts are introduced. The spectral clustering algorithm is studied and analyzed, and a spectral clustering algorithm that automatically determines the number of clusters is proposed. Furthermore, an algorithm for constructing a discrete Morse function to find the optimal solution is proposed, proving that the constructed function is the optimal discrete Morse function. At the same time, two optimization models based on the discrete Morse theory are constructed. Finally, the optimization model based on discrete Morse theory is applied to cluster analysis, and a density clustering algorithm based on the discrete Morse optimization model is proposed. Results: This study is focused on designing and implementing a partitional-based clustering algorithm based on big data, that is suitable for clustering huge datasets to meet low computational requirements. The experiments are conducted in terms of time and space complexity and it is observed that the measure of clustering quality and the run time is capable of running in very less time without negotiating the quality of clustering. The results show that the experiments are carried out on the artificial data set and the UCI data set. Conclusion: The efficiency and superiority of the new model, are verified by comparing it with the clustering results of the DBSCAN algorithm.

Graph Enhanced Fuzzy Clustering for Categorical Data Using a Bayesian Dissimilarity Measure

Categorical data are widely available in many real-world applications, and to discover valuable patterns in such data by clustering is of great importance. However, the lack of a decent quantitative relationship among categorical values makes traditional clustering approaches, which are usually developed for numerical data, perform poorly on categorical datasets. To solve this problem and boost the performance of clustering for categorical data, we propose a novel fuzzy clustering model in this article. At first, by approximating the maximum a posteriori (MAP) estimation of a discrete distribution of data partition, a new fuzzy clustering objective function is designed for categorical data. The Bayesian dissimilarity measure is formulated in this objective to tackle the subtle relationships between categorical values efficiently. Then, to further enhance the performance of clustering, a novel Kullback–Leibler divergence-based graph regularization is integrated into the clustering objective to exploit the prior knowledge on datasets, for example, the information about correlations of data points. The proposed model is solved by the alternative optimization and the experimental results on the synthetic and real-world datasets show that it outperforms the classical and relevant state-of-the-art algorithms. We also present the parameter analysis of our approach, and conduct a comprehensive study on the effectiveness of the Bayesian dissimilarity measure and the KL divergence-based graph regularization.

MSC-CSMC: A multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints for gene expression data.

Many clustering techniques have been proposed to group genes based on gene expression data. Among these methods, semi-supervised clustering techniques aim to improve clustering performance by incorporating supervisory information in the form of pairwise constraints. However, noisy constraints inevitably exist in the constraint set obtained on the practical unlabeled dataset, which degenerates the performance of semi-supervised clustering. Moreover, multiple information sources are not integrated into multi-source constraints to improve clustering quality. To this end, the research proposes a new multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints (MSC-CSMC) for unlabeled gene expression data. The proposed method first uses the gene expression data and the gene ontology (GO) that describes gene annotation information to form multi-source constraints. Then, the multi-source constraints are applied to the clustering by improving the constraint violation penalty weight in the semi-supervised clustering objective function. Furthermore, the constraints selection and cluster prototypes are put into the multi-objective evolutionary framework by adopting a mixed chromosome encoding strategy, which can select pairwise constraints suitable for clustering tasks through synergistic optimization to reduce the negative influence of noisy constraints. The proposed MSC-CSMC algorithm is testified using five benchmark gene expression datasets, and the results show that the proposed algorithm achieves superior performance.

Chimp Optimization Algorithm Influenced Type-2 Intuitionistic Fuzzy C-Means Clustering-Based Breast Cancer Detection System.

In recent years, breast cancer detection is an important area of concentration towards curative image dispensation and exploration. Detection of a disease at an early stage is an important factor in taking it to the next level of treatment. Accuracy plays an important role in the detection of disease. COA-T2FCM (Chimp Optimization Algorithm Based Type-2 Intuitionistic Fuzzy C-Means Clustering) is constructed for detection of such malignancy with the highest accuracy in this paper. The proposed detection process is designed with the combination of type-2 intuitionistic fuzzy c-means clustering in addition to oppositional function. In the type-2 intuitionistic fuzzy c-means clustering, the efficient cluster center can be preferred using the chimp optimization algorithm. Initially, the objective function of the type-2 intuitionistic fuzzy c-means clustering is considered. The chimp optimization algorithm is utilized to optimize the cluster center and fuzzifier in the clustering method. The projected technique is implemented, and in addition, performance metrics such as specificity, sensitivity, accuracy, Jaccard Similarity Index (JSI), and Dice Similarity Coefficient (DSC) are assessed. The projected technique is compared with the conventional technique such as fuzzy c means clustering and k mean clustering methods. The resulting method was also compared with existing methods to ensure the accuracy in the proposed method. The proposed algorithm is tested for its effectiveness on the mammogram images of the three different datasets collected from the Mini-Mammographic Image Analysis Society (Mini-MIAS), the Digital Database for Screening Mammography (DDSM), and Inbreast. The accuracy and Jaccard index score are generally used to measure the similarity between the proposed output and the actual cancer affected regions from the image considered. On an average the proposed method achieved an accuracy of 97.29% and JSI of 95.

Kmeans-CM Algorithm With Spectral Angle Mapper for Hyperspectral Image Classification

Hyperspectral images possess the characteristics of high dimensionality, which causes “dimensional disaster” and low classification accuracy, in response to the problems, based on traditional k-means algorithm and considering the importance of different bands for classification, and also combining both intra-class and inter-class information, a Kmeans-CM (K-means with correlation coefficient and maximize inter-class distance) algorithm with spectral angle mapper for hyperspectral image classification is proposed. First, weights of bands are defined by introducing coefficient of variation and spectral angle mapping, which measures the importance of bands for classification. Second, in order to intensify correlation between pixels in the same category, correlation coefficient is introduced to reset intra-class distance. Then, inter-class information is introduced for clustering by maximizing the distance between class centers and global center to reduce effect of local optimum of clustering effect. Finally, the K-means clustering objective function is redefined according to the band weights and intra-class, inter-class information, and also solving optimally. The overall classification accuracy of the method reached 84.47%, 90.08% and 80.45% on classical hyperspectral data sets Pavia University, Salinas and Botswana respectively, and the results show that the proposed algorithm owns good classification performance.

Finding compact and well-separated clusters: Clustering using silhouette coefficients

Finding compact and well-separated clusters in data sets is a challenging task. Most clustering algorithms try to minimize certain clustering objective functions. These functions usually reflect the intra-cluster similarity and inter-cluster dissimilarity. However, the use of such functions alone may not lead to the finding of well-separated and, in some cases, compact clusters. Therefore additional measures, called cluster validity indices, are used to estimate the true number of well-separated and compact clusters. Some of these indices are well-suited to be included into the optimization model of the clustering problem. Silhouette coefficients are among such indices. In this paper, a new optimization model of the clustering problem is developed where the clustering function is used as an objective and silhouette coefficients are used to formulate constraints. Then an algorithm, called CLUSCO (CLustering Using Silhouette COefficients), is designed to construct clusters incrementally. Three schemes are discussed to reduce the computational complexity of the algorithm. Its performance is evaluated using fourteen real-world data sets and compared with that of three state-of-the-art clustering algorithms. Results show that the CLUSCO is able to compute compact clusters which are significantly better separable in comparison with those obtained by other algorithms.

New feature analysis-based elastic net algorithm with clustering objective function

Cluster analysis, as one of the core methods of data mining, is critical in discovering the natural structure of data to obtain useful information from massive amounts of data. However, many existing clustering algorithms have problems such as poor clustering accuracy and high sensitivity to noise points. These problems are particularly prominent when solving high-dimensional and large-data clustering problems. To overcome these problems, a new feature analysis-based elastic net algorithm with a clustering objective function (FAENC) is proposed in this paper. The new algorithm redefines a cost function based on the goal of clustering, and a new energy function of the clustering elastic net is presented based on the cost function and maximum entropy principle. The proposed model is an unsupervised optimization method. By minimizing the energy function, clustering problems can be solved through self-learning, without manual training or intervention. Additionally, a method for calculating the dispersion degree of the feature attributes is proposed, and the noise attributes can be identified. Each feature attribute is weighted automatically according to the weighting strategy, which can eliminate the influence of noise variables and improve the clustering quality and efficiency. The proposed FAENC algorithm can significantly reduce the impact of the internal structure of the dataset, identify clusters of different sizes, shapes, and densities, and obtain higher clustering quality. Compared with several classical and state-of-the-art clustering methods, FAENC substantially improves the accuracy of clustering results on a large number of synthetic and real-world datasets.