Clustering Performance Research Articles

Distance-based Clustering of Functional Data with Derivative Principal Component Analysis

Functional data analysis (FDA) is an important modern paradigm for handling infinite-dimensional data. An important task in FDA is clustering, which identifies subgroups based on the shapes of measured curves. Considering that derivatives can provide additional useful information about the shapes of functionals, we propose a novel L 2 distance between two random functions by incorporating the functions and their derivative information to determine the dissimilarity of curves under a unified scheme for dense observations. The Karhunen–Loève expansion is used to approximate the curves and their derivatives. Cluster membership prediction for each curve intends to minimize the new distances between the observed and predicted curves through subspace projection among all possible clusters. We provide consistent estimators for the curves, curve derivatives, and the proposed distance. Identifiability issues of the clustering procedure are also discussed. The utility of the proposed method is illustrated via simulation studies and applications to two real datasets. The proposed method can considerably improve cluster performance compared with existing functional clustering methods. Supplementary materials for the article are available online.

Semi-supervised pivotal-aware nonnegative matrix factorization with label and pairwise constraint propagation for data clustering

Semi-supervised nonnegative matrix factorization (NMF) methods have found extensive utility in data clustering applications. However, these existing methods encounter challenges in effectively leveraging the limited supervisory information to enhance the performance of clustering. In particular, the majority of these methods tend to solely utilize either the label information or the pairwise constraint information, neglecting the potential benefits of their joint utilization. To address this issue, a novel algorithm named semi-supervised pivotal-aware nonnegative matrix factorization (SPNMF) is proposed in this paper, which utilizes the label information and the pairwise constraint information simultaneously to enhance the performance in data clustering tasks. The proposed method has two main innovations: (1) adopting the dual constraint propagation (DCP) algorithm (i.e., label propagation and pairwise constraint propagation) to effectively utilize a meager amount of label information for learning a compact data representation; (2) incorporating the pivotal-aware technique to mitigate the negative impact of outliers. Notably, the DCP algorithm not only propagates limited label information to a multitude of unlabeled samples, but also disseminates the pairwise constraint supervisory information obtained from the labels to unconstrained samples, allowing for the acquisition of richer supervisory information in the form of pointwise and pairwise constraints. Furthermore, a comprehensive analysis of SPNMF is conducted. Extensive experimental results are executed on eight real-world image datasets. The results underscore the superior performance of SPNMF in comparison to several state-of-the-art methods.

A simple multi-constraint fusion based semi-supervised non-negative matrix decomposition for image clustering

Semi-supervised non-negative matrix decomposition (SNMF) is a highly interpretable image clustering algorithm. We typically incorporate graph learning to enhance the model’s representation ability, while adding positive and negative labeling information and pairwise constraints to prevent overfitting and improve the discriminative performance. However, embedding positive and negative labels and pairwise constraints simultaneously in the SNMF model is challenging due to their dimension mismatch. Thus We propose a method named multi-constraint fusion based semi-supervised non-negative matrix decomposition (MCFSNMF), which effectively solves the above problem by constructing symmetric labeling matrices embedded with positive and negative labeling information and fusing them with pairwise constraint matrices. In addition, SNMF is essentially a feature extraction method, which cannot obtain clustering results directly. Therefore, we combine the label propagation algorithm to achieve feature learning and label assignment interaction, which not only avoids the impact of clustering post-processing operations but also enhances the representation of labeled data relative to unlabeled data. We compare multiple SNMF algorithms on six image datasets and show that the proposed algorithm can effectively enhance the clustering performance and validate the algorithm’s convergence.

Improving the utility of differentially private clustering through dynamical processing

This study aims to alleviate the trade-off between utility and privacy of differentially private clustering. Existing works focus on simple methods, which show poor performance for non-convex clusters. To fit complex cluster distributions, we propose sophisticated dynamical processing inspired by Morse theory, with which we hierarchically connect the Gaussian sub-clusters obtained through existing methods. Our theoretical results imply that the proposed dynamical processing introduces little to no additional privacy loss. Experiments show that our framework can improve the clustering performance of existing methods at the same privacy level.

ScParser: sparse representation learning for scalable single-cell RNA sequencing data analysis

The rapid rise in the availability and scale of scRNA-seq data needs scalable methods for integrative analysis. Though many methods for data integration have been developed, few focus on understanding the heterogeneous effects of biological conditions across different cell populations in integrative analysis. Our proposed scalable approach, scParser, models the heterogeneous effects from biological conditions, which unveils the key mechanisms by which gene expression contributes to phenotypes. Notably, the extended scParser pinpoints biological processes in cell subpopulations that contribute to disease pathogenesis. scParser achieves favorable performance in cell clustering compared to state-of-the-art methods and has a broad and diverse applicability.

Health-care app detection using optimized clustering

Medical health-care apps have become ubiquitous in today’s world, enhancing health-care quality at affordable costs. The continuous development of new apps underscores their high acceptance and popularity. Machine learning techniques offer effective app identification owing to their high prediction accuracy, particularly with a training dataset of known apps. Although machine learning techniques provide high detection accuracy for known apps, they exhibit abysmal accuracy in detecting unknown and novel apps. This research proposes a novel approach to optimizing the K-means clustering algorithm for detecting zero-day apps. The proposed technique integrates a perceptron feed-forward neural network to determine the coordinates of the centroids of the clusters in K-means clustering. Experimental evaluations demonstrate the efficacy of the proposed approach in enhancing the performance of K-means clustering, providing improved detection for both known and unknown medical health-care apps. A total of 30 health-care apps was utilized in this evaluation. This research enhances the detection accuracy of medical healthcare apps, particularly zero-day apps. The intercluster similarity of the benign class improved to 0.99, and that of the malicious class improved to 0.91, highlighting the improved classification of the apps. The major contribution of this work is achieving an intercluster similarity of 0.89 for detecting novel apps.

Co-regularized optimal high-order graph embedding for multi-view clustering

Real-world applications frequently involve multiple data modalities in the same samples, which are regarded as multi-view data. Multi-view clustering has been studied extensively in recent years to demonstrate embedded heterogeneity. However, most existing methods emphasize low-order correlation in multiple views, whereas approaches that incorporate high-order correlation are limited by the equal view-specific significance problem or a trade-off between global and local consistency. In this paper, we propose a co-regularized optimal graph-based clustering method known as Co-MSE, which integrates the correlation of different orders. By integrating the first-order and second-order similarities, the local structure is preserved, while an optimized embedding representation for multi-view data is obtained simultaneously through co-regularization. We demonstrate that Co-MSE can aid in providing a more suitable embedding representation and further enable satisfactory clustering performance. Extensive experiments on real-world datasets confirm the effectiveness and advantages of the proposed method.

SPGMVC: Multiview Clustering via Partitioning the Signed Prototype Graph.

Multiview clustering (MVC) has been widely studied in machine learning and data mining for its capability of improving clustering performance by fusing the information from multiview data. In the past decade, a large number of MVC methods have made impressive progress, but most of them suffer from computational burdens, especially in large-scale tasks. Binary MVC (BMVC) is proposed to address this issue by representing the large-scale high-dimensional dataset as a group of consensus and low-dimensional binary codes. However, current BMVC-based approaches generate the clustering by executing binary k -means on the obtained binary codes, which fail to capture the embedded geometric information, leading to poor clustering performance. In addition, parameter selection is another "mission impossible" in unsupervised learning tasks including MVC. To tackle these challenges, a framework of multiview clustering via partitioning the signed prototype graph (SPGMVC) is proposed in this work. The SPGMVC framework offers several contributions. First, SPGMVC is designed as a unified framework for MVC. It combines effective technologies, such as consensus binary coding, code compression (CC), signed prototype graph (SPG) partitioning, and prototype-based cluster assignment. Second, SPGMVC partitions the signed graph (SG) based on the relationships between positive and negative edges. By capturing the underlying structure of the data, this partitioning strategy improves clustering accuracy (ACC). CC techniques are applied to reduce the graph's scale, enabling further partitioning and enhancing computational efficiency. Third, SPGMVC employs an alternate minimizing strategy to efficiently handle the optimization problem. This strategy has nearly linear time and space complexity with respect to the data volume, making it suitable for large-scale tasks. Fourth, SPGMVC proposes an automatic parameter selection strategy, eliminating the need for extensive parameter exploration. Comprehensive experiments illustrate the superiority of our model. The implementation of SPGMVC is available at: https://github.com/gepingyang/PSGMVC.

A deep generative model for selecting representative periods in renewable energy-integrated power systems

The extensive integration of renewable energies into power systems has led to a challenging and computationally demanding scenario for the system planning. This is due to the increased number of time series involved and the greater complexity of time series data integrated into power systems. In this paper, a new deep learning-based time aggregation method is proposed for selecting representative periods for renewable energy-integrated power system datasets where multiple variable energy resources are present. Relying on the capabilities of deep generative models, especially Generative Adversarial Network (GAN), the proposed method selects a representative period, including one or more representative days, obtained from multiple time series with spatio-temporal correlations among them. The proposed approach contains the Long Short-Term Memory (LSTM) Network in both generator and discriminator parts. Also, it includes a loss term, specifically designed for clustering tasks, in the minimax objective of the vanilla GAN loss function to enhance the clustering performance in the latent space. Furthermore, the learning rate of the proposed model, as the most important parameter of the learning algorithm, is adaptively fine-tuned during the training process, based on the training error, to enhance its learning performance. Two real-world test cases with various datasets and time series are used for data-based and model-based evaluations. Results obtained on these two real-world test cases confirm the superiority of the proposed model with respect to the state-of-the-art conventional and deep learning-based models, obtaining the performance improvement in the range of [38.55 %, 46.14 %] in terms of Mean Absolute Percentage Error (MAPE), in the range of [45.75 %, 70.16 %] in terms of Root Mean Squared Error (RMSE), and in the range of [65.81 %, 74.58 %] in terms of Mean Absolute Error (MAE). The proposed model can significantly enhance the tractability and scalability of the planning problem of renewable energy-integrated power systems, facilitating renewable energy integration into power systems which is a key issue for net zero transitioning of communities.

Clustering algorithm for remote information transmission of IoT sensor nodes

Internet of Things development is of great significance for modern society progress. However, the limited information in some areas with incomplete infrastructure restricts Internet of Things development, so the long-distance information transmission task of sensor nodes needs to be put on the agenda. The research introduces beamforming technology for clustering wireless sensor nodes, and proposes a clustering algorithm based on wireless sensor node’s energy consumption rate for nodes energy management to achieve remote information sharing and transmission. The results confirm that the success rate of clustering algorithm based on beamforming event triggering increases with node density increasing, and the success rate is infinitely close to 1. In addition, when the sensor node is 120, the average charging delay time based on machine learning energy consumption prediction is only 946 seconds, which is reduced by 521 seconds compared to the Mean-shift algorithm. When sensor node is 120, the algorithm has a successful access count of up to 1288 times. These two clustering algorithms have good clustering performance and significant practical application effects, providing reliable technical support for remote data transmission in the modern Internet of Things.

Multi-view clustering via high-order bipartite graph fusion

Multi-view clustering is widely applied in engineering and scientific research. It helps reveal the underlying structures and correlations behind complex multi-view data. Graph-based multi-view clustering stands as a prominent research frontier within the multi-view clustering field, yet faces persistent challenges. Firstly, typically constructed initial input graphs for each view yields sparse clustering structure, hindering clustering performance. Secondly, as data sources proliferate, algorithms encounter escalating time complexities, notably in methods relying on n×n fully connected graphs. Thirdly, prevailing graph fusion strategies struggle to mitigate the impact of low-quality graphs, impeding overall efficacy. In this paper, we present a novel Multi-View Clustering method based on High-Order Bipartite Graph fusion (MCHBG). For the first two challenges, the introduced high-order bipartite graphs in MCHBG reveal richer clustering structures, effectively alleviating sparse clustering structure of the input graph, while keeping the overall algorithm’s computational complexity controlled within O(n). For the third challenge, our graph fusion mechanism selectively integrates high-order bipartite graphs, and implicitly weights the selected bipartite graphs to mitigate the impact of low-quality bipartite graphs. MCHBG learns a structured fusion bipartite graph under the Laplacian rank constraint, which directly indicates the clusters of data. Extensive experimental results demonstrate the effectiveness and superiority of MCHBG. Code available: https://anonymous.4open.science/r/MCHBG.

Revisiting Possibilistic Fuzzy C-Means Clustering Using the Majorization-Minimization Method.

Possibilistic fuzzy c-means (PFCM) clustering is a kind of hybrid clustering method based on fuzzy c-means (FCM) and possibilistic c-means (PCM), which not only has the stability of FCM but also partly inherits the robustness of PCM. However, as an extension of FCM on the objective function, PFCM tends to find a suboptimal local minimum, which affects its performance. In this paper, we rederive PFCM using the majorization-minimization (MM) method, which is a new derivation approach not seen in other studies. In addition, we propose an effective optimization method to solve the above problem, called MMPFCM. Firstly, by eliminating the variable V∈Rp×c, the original optimization problem is transformed into a simplified model with fewer variables but a proportional term. Therefore, we introduce a new intermediate variable s∈Rc to convert the model with the proportional term into an easily solvable equivalent form. Subsequently, we design an iterative sub-problem using the MM method. The complexity analysis indicates that MMPFCM and PFCM share the same computational complexity. However, MMPFCM requires less memory per iteration. Extensive experiments, including objective function value comparison and clustering performance comparison, demonstrate that MMPFCM converges to a better local minimum compared to PFCM.

MSDQ: Multi-Scheduling Dual-Queues coflow scheduling without prior knowledge

Coflow scheduling is crucial for enhancing application-level communication performance in data-parallel clusters. While schemes like Varys can potentially achieve optimal performance, their dependence on a prior information about coflows poses practical challenges. Existing non-clairvoyant solutions, such as Aalo, approximate the classical online Shortest-Job-First (SJF) scheduling but fail to identify bottleneck flows in coflows. Consequently, they often allocate excessive bandwidth to non-bottleneck flows, leading to bandwidth wastage and reduced overall performance. In this paper, we introduce MSDQ, a coflow scheduling mechanism that operates without prior knowledge, utilizing multi-scheduling dual-priority queues, and using width estimates. This method adjusts coflow queue priorities and scheduling sequences based on the coflow’s width and the volume of data transmitted. By reallocating unused network bandwidth at multiple points during the scheduling process, MSDQ maximizes the bandwidth usage and significantly reduces the average coflow completion time. Our evaluation, using a publicly available production cluster trace from Facebook, demonstrates that MSDQ reduces the average coflow completion time by 1.42× compared to Aalo.

Enhancing clustering representations with positive proximity and cluster dispersion learning

Contemporary deep clustering approaches often rely on contrastive or non-contrastive techniques to acquire effective representations for clustering tasks. Contrastive methods apply negative pairs to achieve homogenous representations but can introduce class collision problems, potentially compromising clustering performance. In contrast, non-contrastive techniques prevent class collisions but may produce non-uniform representations that lead to clustering collapse. In this work, we propose a novel end-to-end deep clustering approach named PIPCDR, designed to harness the strengths of both approaches while mitigating their limitations. PIPCDR incorporates a positive instance proximity loss and a cluster dispersion regularizer. The positive instance proximity loss ensures alignment between augmented views of instances and their sampled neighbors, enhancing within-cluster compactness by selecting genuinely positive pairs within the embedding space. Meanwhile, the cluster dispersion regularizer maximizes inter-cluster distances while minimizing within-cluster compactness, promoting uniformity in the learned representations. PIPCDR excels in producing well-separated clusters, generating uniform representations, avoiding class collision issues, and enhancing within-cluster compactness. We extensively validate the effectiveness of PIPCDR within an end-to-end Majorize-Minimization framework, demonstrating its competitive performance on moderate-scale clustering benchmark datasets and establishing new state-of-the-art results on large-scale datasets.

Multi-order graph clustering with adaptive node-level weight learning

Current graph clustering methods emphasize individual node and edge connections, while ignoring higher-order organization at the level of motif. Recently, higher-order graph clustering approaches have been designed by motif-based hypergraphs. However, these approaches often suffer from hypergraph fragmentation issue seriously, which degrades the clustering performance greatly. Moreover, real-world graphs usually contain diverse motifs, with nodes participating in multiple motifs. A key challenge is how to achieve precise clustering results by integrating information from multiple motifs at the node level. In this paper, we propose a multi-order graph clustering model (MOGC) to integrate multiple higher-order structures and edge connections at node level. MOGC employs an adaptive weight learning mechanism to automatically adjust the contributions of different motifs for each node. This not only tackles hypergraph fragmentation issue but enhances clustering accuracy. MOGC is efficiently solved by an alternating minimization algorithm. Experiments on seven real-world datasets illustrate the effectiveness of MOGC.

Projective Incomplete Multi-View Clustering.

Due to the rapid development of multimedia technology and sensor technology, multi-view clustering (MVC) has become a research hotspot in machine learning, data mining, and other fields and has been developed significantly in the past decades. Compared with single-view clustering, MVC improves clustering performance by exploiting complementary and consistent information among different views. Such methods are all based on the assumption of complete views, which means that all the views of all the samples exist. It limits the application of MVC, because there are always missing views in practical situations. In recent years, many methods have been proposed to solve the incomplete MVC (IMVC) problem and a kind of popular method is based on matrix factorization (MF). However, such methods generally cannot deal with new samples and do not take into account the imbalance of information between different views. To address these two issues, we propose a new IMVC method, in which a novel and simple graph regularized projective consensus representation learning model is formulated for incomplete multi-view data clustering task. Compared with the existing methods, our method not only can obtain a set of projections to handle new samples but also can explore information of multiple views in a balanced way by learning the consensus representation in a unified low-dimensional subspace. In addition, a graph constraint is imposed on the consensus representation to mine the structural information inside the data. Experimental results on four datasets show that our method successfully accomplishes the IMVC task and obtain the best clustering performance most of the time. Our implementation is available at https://github.com/Dshijie/PIMVC.

Ensemble Clustering via Co-Association Matrix Self-Enhancement.

Ensemble clustering integrates a set of base clustering results to generate a stronger one. Existing methods usually rely on a co-association (CA) matrix that measures how many times two samples are grouped into the same cluster according to the base clusterings to achieve ensemble clustering. However, when the constructed CA matrix is of low quality, the performance will degrade. In this article, we propose a simple, yet effective CA matrix self-enhancement framework that can improve the CA matrix to achieve better clustering performance. Specifically, we first extract the high-confidence (HC) information from the base clusterings to form a sparse HC matrix. By propagating the highly reliable information of the HC matrix to the CA matrix and complementing the HC matrix according to the CA matrix simultaneously, the proposed method generates an enhanced CA matrix for better clustering. Technically, the proposed model is formulated as a symmetric constrained convex optimization problem, which is efficiently solved by an alternating iterative algorithm with convergence and global optimum theoretically guaranteed. Extensive experimental comparisons with 12 state-of-the-art methods on ten benchmark datasets substantiate the effectiveness, flexibility, and efficiency of the proposed model in ensemble clustering. The codes and datasets can be downloaded at https://github.com/Siritao/EC-CMS.

Multiview Subspace Clustering via Low-Rank Symmetric Affinity Graph.

Multiview subspace clustering (MVSC) has been used to explore the internal structure of multiview datasets by revealing unique information from different views. Most existing methods ignore the consistent information and angular information of different views. In this article, we propose a novel MVSC via low-rank symmetric affinity graph (LSGMC) to tackle these problems. Specifically, considering the consistent information, we pursue a consistent low-rank structure across views by decomposing the coefficient matrix into three factors. Then, the symmetry constraint is utilized to guarantee weight consistency for each pair of data samples. In addition, considering the angular information, we utilize the fusion mechanism to capture the inherent structure of data. Furthermore, to alleviate the effect brought by the noise and the high redundant data, the Schatten p-norm is employed to obtain a low-rank coefficient matrix. Finally, an adaptive information reduction strategy is designed to generate a high-quality similarity matrix for spectral clustering. Experimental results on 11 datasets demonstrate the superiority of LSGMC in clustering performance compared with ten state-of-the-art multiview clustering methods.

Selective multiple kernel fuzzy clustering with locality preserved ensemble

Multiple kernel fuzzy clustering (MKFC) has demonstrated promising performance in capturing the non-linear relationships within data. However, its effectiveness relies heavily on the appropriate selection of the fuzzification coefficient and kernel functions. To address this challenge, this paper proposes a novel clustering ensemble approach for improving the robustness and accuracy of the MKFC algorithm. The proposed method employs a multi-objective evolutionary optimization approach to heuristically select the optimal fuzzification and kernel coefficients. By employing the selected coefficients, the MKFC algorithm generates a diverse set of accurate candidate clustering results. Subsequently, a locality preserved ensemble mechanism is introduced to derive the final partition matrix, ensuring that all candidate clusterings within the Pareto non-dominated set contribute to the final consensus matrix. Moreover, this mechanism incorporates the knowledge about the locality of the dataset via a graph regularization term, thereby further enhancing the clustering performance. Comprehensive experiments conducted on widely adopted benchmark datasets demonstrate the superiority of the proposed method over the state-of-the-art approaches.

Attribute graph clustering via transformer and graph attention autoencoder

Graph clustering is a crucial technique for partitioning graph data. Recent research has concentrated on integrating topology and attribute information from attribute graphs to generate node embeddings, which are subsequently clustered using classical algorithms. However, these methods have some limitations, such as insufficient information inheritance in shallow networks or inadequate quality of reconstructed nodes, leading to suboptimal clustering performance. To tackle these challenges, we introduce two normalization techniques within the graph attention autoencoder framework, coupled with an MSE loss, to facilitate node embedding learning. Furthermore, we integrate Transformers into the self-optimization module to refine node embeddings and clustering outcomes. Our model can induce appropriate node embeddings for graph clustering in a shallow network. Our experimental results demonstrate that our proposed approach outperforms the state-of-the-art in graph clustering over multiple benchmark datasets. In particular, we achieved 76.3% accuracy on the Pubmed dataset, an improvement of at least 7% compared to other methods.