Clustering Performance Research Articles

Bidirectional Probabilistic Subspaces Approximation for Multiview Clustering.

The existing multiview clustering models learn a consistent low-dimensional embedding either from multiple feature matrices or multiple similarity matrices, which ignores the interaction between the two procedures and limits the improvement of clustering performance on multiview data. To address this issue, a bidirectional probabilistic subspaces approximation (BPSA) model is developed in this article to learn a consistently orthogonal embedding from multiple feature matrices and multiple similarity matrices simultaneously via the disturbed probabilistic subspace modeling and approximation. A skillful bidirectional fusion strategy is designed to guarantee the parameter-free property of the BPSA model. Two adaptively weighted learning mechanisms are introduced to ensure the inconsistencies among multiple views and the inconsistencies between bidirectional learning processes. To solve the optimization problem involved in the BPSA model, an iterative solver is derived, and a rigorous convergence guarantee is provided. Extensive experimental results on both toy and real-world datasets demonstrate that our BPSA model achieves state-of-the-art performance even if it is parameter-free.

Deep Tensor Spectral Clustering Network via Ensemble of Multiple Affinity Tensors.

Tensor spectral clustering (TSC) is an emerging approach that explores multi-wise similarities to boost learning. However, two key challenges have yet to be well addressed in the existing TSC methods: (1) The construction and storage of high-order affinity tensors to encode the multi-wise similarities are memory-intensive and hampers their applicability, and (2) they mostly employ a two-stage approach that integrates multiple affinity tensors of different orders to learn a consensus tensor spectral embedding, thus often leading to a suboptimal clustering result. To this end, this paper proposes a tensor spectral clustering network (TSC-Net) to achieve one-stage learning of a consensus tensor spectral embedding, while reducing the memory cost. TSC-Net employs a deep neural network that learns to map the input samples to the consensus tensor spectral embedding, guided by a TSC objective with multiple affinity tensors. It uses stochastic optimization to calculate a small part of the affinity tensors, thereby avoiding loading the whole affinity tensors for computation, thus significantly reducing the memory cost. Through using an ensemble of multiple affinity tensors, the TSC can dramatically improve clustering performance. Empirical studies on benchmark datasets demonstrate that TSC-Net outperforms the recent baseline methods.

An implementation of a plasma physics application for distributed-memory supercomputers using a directive-based programming framework

To extract performance from supercomputers, programmers in the High Performance Computing (HPC) community are often required to use a combination of frameworks to take advantage of the multiple levels of parallelism. However, over the years, efforts have been made to simplify this situation by creating frameworks that can take advantage of multiple levels. This often means that the programmer has to learn a new library. On the other hand, there are frameworks that were created by extending the capabilities of established paradigms. In this paper, we explore one of this libraries, OpenMP Cluster. As its name implies, it extends the OpenMP API, which allows seasoned programmers to take advantage of their experience to use just one API to program in sharedmemory and distributed-memory parallelism. In this paper, we took an existing plasma physics code that was programmed with MPI+OpenMP and ported it over to OpenMP Cluster. We also show that under certain conditions, the performance of OpenMP Cluster is similar to that of the MPI+OpenMP code.

Using machine learning algorithms to cluster and classify stone pine (Pinus pinea L.) populations based on seed and seedling characteristics

AbstractThe phenotype of a woody plant represents its unique morphological properties. Population discrimination and individual classification are crucial for breeding populations and conserving genetic diversity. Machine Learning (ML) algorithms are gaining traction as powerful tools for predicting phenotypes. The present study is focused on classifying and clustering the seeds and seedlings in terms of morphological characteristics using ML algorithms. In addition, the k-means algorithm is used to determine the ideal number of clusters. The results obtained from the k-means algorithm were then compared with reality. The best classification performance achieved by the Random Forest algorithm was an accuracy of 0.648 and an F1-Score of 0.658 for the seed traits. Also, the best classification performance for stone pine seedlings was observed for the k-Nearest Neighbors algorithm (k = 18), for which the accuracy and F1-Score were 0.571 and 0.582, respectively. The best clustering performance was achieved with k = 2 for the seed (average Silhouette index = 0.48) and seedling (average Silhouette Index = 0.51) traits. According to the principal component analysis, two dimensions accounted for 97% and 63% of the traits of seeds and seedlings, respectively. The most important features between the seed and seedling traits were cone weight and bud set, respectively. This study will provide a foundation and motivation for future efforts in forest management practices, particularly regarding reforestation, yield optimization, and breeding programs.

Joint Projected Fuzzy Neighborhood Preserving C-means Clustering with Local Adaptive Learning

Graph-based dimensionality reduction approaches are extensively applied for preserving the intrinsic structures of high-dimensional data. To capture discriminative information from data clusters that lack a clear manifold structure, previous studies have combined clustering algorithms with local manifold structure preservation. These methods assume that two neighbor data points share the same cluster label. However, the assumption may not hold for Fuzzy C-Means which may assign two nearby data points to different clusters depending on the distance between data points and cluster centroids. This inconsistency degrades the performance in clustering and dimensionality reduction. To tackle this issue, we propose a novel approach for dimensionality reduction, termed joint Projected Fuzzy Neighborhood Preserving C-means Clustering with Local Adaptive Learning (PFNPCM). PFNPCM learns cluster centroids, sparse membership grades, embedded subspaces, and structured graphs simultaneously and extracts both global and local structures of data. Through clustering data that preserves the local neighborhood structure in a low-dimensional space instead of the original one, PFNPCM can effectively capture both global and local information from the inherent manifold structure and cluster structure. The employment of ℓ2,1-norm as the distance metric in the clustering and the preservation of neighborhood structure further ensure the robustness of clustering results against noise or outliers. To achieve the consistency of the captured global and local discriminative information, PFNPCM adopts a novel fuzzy local similarity measure based on the manifold structure, and the manifold assumption thus holds when partitioning unlabeled samples into subgroups. Massive experiments on diverse well-known datasets demonstrate that PFNPCM is superior to some state-of-the-art methods.

Coupled double self-expressive subspace clustering with low-rank tensor learning

In recent years, subspace clustering (SC) methods have been widely used in machine learning and computer vision. However, the self-expressive matrix obtained by the existing SC methods is not clean block diagram and still contains some of typical noise in data, which leads to a decrease in clustering performance. In addition, most SC methods do not make full use of the structural information of self-expressive matrices. To solve the above problems, we propose coupled double self-expressive subspace clustering with low-rank tensor learning (CDSESLRTL) method. Specifically, first, we use the self-expressive characteristics of the data to obtain the self-expressive matrix of the data. However, the structural information of the self-expressive matrix is not fully utilized, and the block diagonal structure is often not sufficiently clear. Then, we obtain a new self-expressive matrix by viewing the obtained self-expressive matrix as a feature representation of the data and use self-expressive characteristics again. Third, we stack the two self-expressive matrices into a tensor, which is constrained by the tensor nuclear norm (TNN). Then, the two self-expressive matrices can be learned in a mutually reinforcing manner, which not only gives these matrices a clearer block diagonal structure but also sufficiently utilizes their structural information. Finally, the combination of these three steps forms a framework, which is addressed using the augmented Lagrange multiplier (ALM) method. The experimental results on various datasets demonstrate that the CDSESLRTL algorithm outperforms the state-of-the-art algorithms.

Motor PHM on Edge Computing with Anomaly Detection and Fault Severity Estimation through Compressed Data Using PCA and Autoencoder

The motor is essential for manufacturing industries, but wear can cause unexpected failure. Predictive and health management (PHM) for motors is critical in manufacturing sites. In particular, data-driven PHM using deep learning methods has gained popularity because it reduces the need for domain expertise. However, the massive amount of data poses challenges to traditional cloud-based PHM, making edge computing a promising solution. This study proposes a novel approach to motor PHM in edge devices. Our approach integrates principal component analysis (PCA) and an autoencoder (AE) encoder achieving effective data compression while preserving fault detection and severity estimation integrity. The compressed data is visualized using t-SNE, and its ability to retain information is assessed through clustering performance metrics. The proposed method is tested on a custom-made experimental platform dataset, demonstrating robustness across various fault scenarios and providing valuable insights for practical applications in manufacturing.

Screening of Methyl-β-cyclodextrins as an Antifading Agent for Cyanine Dye-Labeled Streptavidin to Improve the Performance of Genotyping Chips.

As a photostabilizing agent for cyanine dye, methyl-β-cyclodextrin (MβCD) was investigated as a possible antifading agent for cyanine dye-labeled proteins. Cyanine-3 (Cy3)-labeled streptavidin (SA-Cy3) solutions containing MβCD exhibited improved resistance against photobleaching. Further research revealed that MβCD can be used as a coating material on the surface of gene chips. Chips loaded with cyanine dye (Cy3 and Cyanine-5 (Cy5))-conjugated model microbeads exhibited resistance against photobleaching with MβCD coatings. MβCD coatings improved the imaging quality of model chips and resulted in higher discrimination ratios (DR) of single base recognition by a set of control beads (NP68). In a whole genome genotyping assay for human samples, the MβCD-coated samples were found to have a better clustering performance than the noncoated ones for a group of randomly picked single nucleotide polymorphisms (SNPs).

Novel Alzheimer's disease subtypes based on functional brain connectivity in human connectome project

The pathogenesis of Alzheimer's disease (AD) remains unclear, but revealing individual differences in functional connectivity (FC) may provide insights and improve diagnostic precision. A hierarchical clustering-based autoencoder with functional connectivity was proposed to categorize 82 AD patients from the Alzheimer's Disease Neuroimaging Initiative. Compared to directly performing clustering, using an autoencoder to reduce the dimensionality of the matrix can effectively eliminate noise and redundant information in the data, extract key features, and optimize clustering performance. Subsequently, subtype differences in clinical and graph theoretical metrics were assessed. Results indicate a significant inter-subject heterogeneity in the degree of FC disruption among AD patients. We have identified two neurophysiological subtypes: subtype I exhibits widespread functional impairment across the entire brain, while subtype II shows mild impairment in the Limbic System region. What is worth noting is that we also observed significant differences between subtypes in terms of neurocognitive assessment scores associations with network functionality, and graph theory metrics. Our method can accurately identify different functional disruptions in subtypes of AD, facilitating personalized treatment and early diagnosis, ultimately improving patient outcomes.

Projected fuzzy [formula omitted]-means clustering algorithm with instance penalty

At present, high-dimensional data clustering has become a vital research field in machine learning. Traditional clustering algorithms cannot perform well on high-dimensional data, where the clustering task is usually divided into two stages: dimensionality reduction first and clustering later. In general, the existing high-dimensional clustering methods usually have the following shortcomings: (1) the two-stage strategy splits the connection between clustering and dimensionality reduction; (2) these algorithms do not consider the impact of anomalous instances in high-dimensional data on clustering performance. Therefore, to address these problems, a projected fuzzy c-means clustering algorithm with instance penalty (PCIP) is proposed. Firstly, we construct an instance penalty matrix and assign an instance penalty coefficient to each sample. Secondly, a model for clustering high-dimensional data is constructed by integrating fuzzy c-means clustering (FCM) and principal component analysis (PCA). The proposed model can perform dimensionality reduction and clustering simultaneously. In addition, the time complexity of the proposed algorithm is linearly related to the number of samples n, which can efficiently deal with large data sets. The proposed PCIP algorithm is verified by experiments using clustering accuracy and normalized mutual information (NMI) as evaluation metrics. The experimental results on 10 image datasets show that the average accuracy and average NMI of the PCIP algorithm are improved by 0.0375 and 0.0275, respectively, compared to the second-ranked algorithm.

Utilizing excitation-emission matrix fluorescence spectroscopy and convolutional neural networks for dark tea brand and aging period identification

This study utilized excitation-emission matrix (EEM) fluorescence spectroscopy in conjunction with convolutional neural networks (CNNs) to identify the dark tea brands and their aging periods, which were both key factors of market value. The adaptation of EEM data to the CNN model was examined, encompassing the configuration of the data structure and the implementation of data augmentation. Three typical CNN models including AlexNet, GoogLeNet, and ResNet were evaluated. ResNet was selected because it demonstrated the highest classification accuracy on the test set and superior clustering performance in the t-SNE visualization of extracted features. Analysis of feature contribution suggested that the CNN model exhibited enhanced sensitivity towards the boundaries of the fluorescence region. Furthermore, transfer learning was proved to be effective in facilitating adaptation of dark tea recognition models to databases with different sample sizes, which was meaningful for the application of this method in dark tea market supervision. Finally, the multi-task model was developed and achieved a 96.9 % accuracy in brand identification and 87.5 % in aging period determination, suggesting the advantages of multi-task learning in enhancing model generalization and training efficiency.

Deep image clustering: A survey

Deep image clustering networks have the capability to categorize unlabeled images, thereby effectively utilizing them. This paper synthesizes recent researches about deep image clustering network and summarizes them within a general framework. Image Preprocessing part transforms collected images into a dataset accepted by the network, then Image Embedding part embeds images from the dataset into vectors which represents image features. After Feature Processing part further reduces the dimensionality or enhances these features, the following Feature Clustering part divides the images into several categories. Cluster Result Processing part treats the clustering outcomes to implement weighted, multi-view, subspace, multimodal, or self-supervised methods. Downstream Applications part employ the clustering results to address various real-world problems. The performance of common baseline clustering methods and several feature extraction architectures are also compared and analyzed. The results indicate that within deep image clustering networks, the choice of clustering algorithm has a relatively minor impact on clustering performance, whereas the selection of the feature extraction network architecture is decisive for clustering metrics. The choice of architecture should be customized based on the characteristics of the dataset. Finally, we provide suggestions for potential research directions in deep image clustering networks.

Dual contrastive learning for multi-view clustering

Self-representation subspace clustering, based on the self-representation property of linear subspaces, has demonstrated superior performance in subspace clustering. However, many existing methods learn self-representation in the embedding space, which fails to accurately capture the clustering structure of the data. Additionally, in the data embedding process, only the shared features among clusters are removed, and the discriminative features that distinguish different clusters cannot be effectively selected. To address these limitations, we propose a contrastive learning-based self-representation subspace clustering method named Multi-view Contrastive Subspace Clustering (MCSC). This method integrates feature selection and self-representation learning into a unified framework. It learns a shared self-representation coefficient in both the embedding space and the original space using contrastive learning, which allows for a more accurate description of the linear relationship between samples and a better depiction of the inter-sample structure simultaneously. Moreover, we employ the ℓ1,2-norm for feature selection, which eliminates shared redundancies while preserving important features specific to each cluster, making the samples more distinguishable. Extensive experiments on six datasets verify the validity of our proposed model.

Multi-view clustering via latent consistency multi-graph fusion

Multi-view clustering (MVC) algorithms are prevalent in machine learning tasks due to their superiority in mining complementary information from multi-view data. Despite their widespread use, existing MVC approaches ignore the presence of inconsistencies in the multi-view features during multi-view data fusion, which leads to suboptimal clustering performance. To address this challenge, the paper presents a novel approach called multi-view clustering via latent consistency multi-graph fusion. Specifically, a novel latent representation learning model is devised to learn a unified representation from multi-view features, simultaneously capturing the high-order relationships within the multi-view data. Additionally, to alleviate the adverse effects caused by the inconsistency multi-view feature, a consistent multi-graph fusion module is proposed to provide consistent constraints for latent representations. To be specific, the graphs that reflect the manifold structure for each view are fused to guide a consistent latent representation learning. After that, an alternating optimization algorithm is designed to update variables iteratively and alternatively, among which the fusion weights are assigned adaptively. Finally, comprehensive experiments conducted on seven widely recognized benchmark multi-view datasets affirm the efficacy of the proposed method, showcasing its superiority over state-of-the-art approaches.

Federated Momentum Contrastive Clustering

Self-supervised representation learning and deep clustering are mutually beneficial to learn high-quality representations and cluster data simultaneously in centralized settings. However, it is not always feasible to gather large amounts of data at a central entity, considering data privacy requirements and computational resources. Federated Learning (FL) has been developed successfully to aggregate a global model while training on distributed local data, respecting the data privacy of edge devices. However, most FL research effort focuses on supervised learning algorithms. A fully unsupervised federated clustering scheme has not been considered in the existing literature. We present federated momentum contrastive clustering (FedMCC), a generic federated clustering framework that can not only cluster data automatically but also extract discriminative representations training from distributed local data over multiple users. In FedMCC, we demonstrate a two-stage federated learning paradigm where the first stage aims to learn differentiable instance embeddings and the second stage accounts for clustering data automatically. The experimental results show that FedMCC not only achieves superior clustering performance but also outperforms several existing federated self-supervised methods for linear evaluation and semi-supervised learning tasks. Additionally, FedMCC can easily be adapted to ordinary centralized clustering through what we call momentum contrastive clustering (MCC). We show that MCC achieves state-of-the-art clustering accuracy results in certain datasets such as STL-10 and ImageNet-10. We also present a method to reduce the memory footprint of our clustering schemes.

Exploring road safety using alignment perspective features in real driving images: A case study on mountain freeways.

While driving, drivers frequently adapt their driving behaviors according to their perception of the road's alignment features. However, traditional two-dimensional alignment methods lack the ability to capture these features from the driver's perspective. This study introduces a novel method for road alignment recognition, employing image recognition technology to extract alignment perspective features, namely alignment perspective skewness (APS) and alignment perspective kurtosis (APK), from in-real driving images. Subsequently, the K-means clustering algorithm is utilized for road segment classification based on APS and APK indicators. Various sliding step length for clustering are employed, with step length ranging from 100m to 400m. Furthermore, the accident rates for different segment clusters are analyzed to explore the relationship between alignment perspective features and traffic safety. A 150 km mountain road section of the Erlianhaote-Guangzhou freewway from Huaiji to Sihui is selected as a case study. The results demonstrate that using alignment perspective features as classification criteria produces favorable clustering outcomes, with superior clustering performance achieved using shorter segment lengths and fewer cluster centers. The road segment classification based on alignment perspective features reveals notable differences in accident rates across categories; while traditional two-dimensional parameters-based classification methods fail to capture these differences. The most significant differences in accident rates across categories are observed with segment length of 100m, with the significance gradually diminishing as segment length increases and disappearing entirely when the length exceeds 300m. These findings validate the reliability of using alignment perspective features (APS and APK) for road alignment classification and road safety analysis, providing valuable insights for road safety management.

An adaptive kernel dictionary-based low-rank representation method for subspace clustering

Low-rank representation (LRR) is a classic subspace clustering (SC) algorithm, and many LRR-based methods have been proposed. Generally, LRR-based methods use denoized data as dictionaries for data reconstruction purpose. However, the dictionaries used in LRR-based algorithms are fixed, leading to poor clustering performance. In addition, most of these methods assume that the input data are linearly correlated. However, in practice, data are mostly nonlinearly correlated. To address these problems, we propose a novel adaptive kernel dictionary-based LRR (AKDLRR) method for SC. Specifically, to explore nonlinear information, the given data are mapped to the Hilbert space via the kernel technique. The dictionary in AKDLRR is not fixed; it adaptively learns from the data in the kernel space, making AKDLRR robust to noise and yielding good clustering performance. To solve the AKDLRR model, an efficient procedure including an alternative optimization strategy is proposed. In addition, a theoretical analysis of the convergence performance of AKDLRR is presented, which reveals that AKDLRR can converge in at most three iterations under certain conditions. The experimental results show that AKDLRR can achieve the best clustering performance and has excellent speed in comparison with other algorithms.

The MD-BK-means Construction Method for Library Reader Portraits

Due to the rapid development of internet technology, knowledge acquisition has become more convenient and efficient in network operations. University libraries serve as important resources for readers to acquire knowledge, and online resources and services in libraries have become the main direction for readers to acquire knowledge at present. Research the use of binary K-means clustering algorithm and library reader portrait technology to optimize the design of the reader portrait module and construct a multidimensional and multi perspective reader feature system. Reuse Spark programming language and support vector machine to perform computational processing on reader profile data to ensure accurate segmentation of the dataset. Finally, three datasets were used to test the accuracy and efficiency of the algorithm. The experimental comparison shows that the mining and precision segmentation of parallel SVM on the dataset are 93.20%, 85.16%, and 79.35% on the sample set, respectively, in order to optimize the mining performance of the data. The MD multi view binary K-means algorithm has a total Mahalanobis distance of 3.543, 5.268, and 22.385 on the sample dataset, respectively, to demonstrate its superiority in clustering performance. Therefore, the multi view binary K-means algorithm based on Mahalanobis distance has high advantages in reader portrait technology design, and provides technical support and theoretical reference for library reader portrait technology.

G2-SCANN: Gaussian-kernel graph-based SLD clustering algorithm with natural neighbourhood

For most clustering methods, not only the number of clusters must be set in advance, but also various hyperparameters such as initial centroids, number of nearest neighbours, the minimum number of points, neighbourhood radius, and cutoff distance all require pre-specification. As one of the most promising unsupervised learning methods in machine intelligence, existing clustering methods cannot simultaneously handle datasets with arbitrary shapes, different densities, distinct sizes, and overlapping. Background outliers and high dimensionality make clustering problems more challenging. In this paper, we propose a novel universal clustering methodology, called G2-SCANN, which yields the best clustering performance for all 30 synthetic and real datasets without any hyperparameter tuning if the exact number of clusters is known. Firstly, the shortest path length (SPL) in complex network or graph-based geodesic distance is used to give a locally backbone-structured description of graph vertex similarity. Accordingly, SPL-weighted local degree (SLD) is defined as vertex attributes of a SPL-weighted graph expressed by G2-SPL adjacency matrix with ε-natural neighbourhood. Secondly, the process of calculating SLD for every data point in a bottom-up way directly leads to division from a complete graph constituted by all data points to a group of SLD trees. This brings the interpretability and the elimination of lone trees. Thirdly, contrastive learning of largest SLD values for finding root vertices of each divisive tree is conducted and top-down category message is then transmitted from the root vertices to all the leaf ones of a SLD tree. It eventually produces tree-like clusters. Totally, the proposed G2-SCANN method leverages both local neighbouring similarity of data points and global information about data distribution and makes it perform better than other methods.

Meta-Embedded Clustering (MEC): A new method for improving clustering quality in unlabeled bird sound datasets

In recent years, ecoacoustics has offered an alternative to traditional biodiversity monitoring techniques with the development of passive acoustic monitoring (PAM) systems allowing, among others, to detect and identify species that are difficult to detect by human observers, automatically. PAM systems typically generate large audio datasets, but using these monitoring techniques to infer ecologically meaningful information remains challenging. In most cases, several thousand hours of recordings need to be manually labeled by experts limiting the operability of the systems. Based on recent developments of meta-learning algorithms and unsupervised learning techniques, we propose here Meta-Embedded Clustering (MEC), a new method with high potential for improving clustering quality in unlabeled bird sound datasets. MEC method is organized in two main steps, with: (a) fine-tuning of a pretrained convolutional neural network (CNN) backbone with different meta-learning algorithms using pseudo-labeled data, and (b) clustering of manually-labeled bird sounds in the latent space based on vector embeddings extracted from the fine-tuned CNN. The MEC method significantly enhanced average clustering performance from less than 1% to more than 80%, greatly outperforming the traditional approach of relying solely on CNN features extracted from a general neotropical audio database. However, this enhanced performance came with the cost of excluding a portion of the data categorized as noise. By improving the quality of clustering in unlabeled bird sound datasets, the MEC method should facilitate the work of ecoacousticians in managing acoustic units of bird song/call clustered according to their similarities, and in identifying potential clusters of species undetected using traditional approaches.