Multiple Clusters Research Articles

Multiple Kernel Clustering With Adaptive Multi-Scale Partition Selection

Multiple Kernel Clustering With Adaptive Multi-Scale Partition Selection

A Clustering Ensemble Method for Drug Safety Signal Detection in Post-Marketing Surveillance.

Post-marketing surveillance refers to the process of monitoring the safety of drugs once they reach the market, after the successful completion of clinical trials. In this work, we investigate a computational approach using data mining tools to detect safety signals from post-market safety databases, or in other words, to identify adverse events (AEs) with disproportionately high reporting rates compared to other AEs, associated with a particular drug or a drug class. Essentially, we view this as a problem of cluster analysis-based anomaly detection on post-market safety data, where the goal is to 'unsupervisedly' detect the anomalous or the signal AEs. Our findings demonstrate the potential of using a clustering ensemble method to detect drug safety signals. It employs multiple clustering or anomaly detection algorithms, followed by a performance comparison of the candidate algorithms based on a collection of appropriate measures of goodness of clustering results. The method is general enough to include any number of clustering or anomaly detection algorithms and goodness measures, and performs better than any of the candidate algorithms in identifying the signal AEs. Extensive simulation studies illustrate that the ensemble method detects the AE signals from synthetic post-market safety datasets pretty accurately across the different scenarios explored. Based on the cases reported to the FDA Adverse Event Reporting System (FAERS) between 2013 and 2022, we further demonstrate that the ensemble method successfully identifies and confirms most of the adverse events that are known to occur most frequently in reaction to antiepileptic drugs and -lactam antibiotics.

A clustering ensemble algorithm for handling deep embeddings using cluster confidence

Abstract Clustering ensemble, which aims to learn a robust consensus clustering from multiple weak base clusterings, has achieved promising performance on various applications. With the development of big data, the scale and complexity of data is constantly increasing. However, most existing clustering ensemble methods typically employ shallow clustering algorithms to generate base clusterings. When confronted with high-dimensional complex data, these shallow algorithms fail to fully utilize the intricate features present in the latent data space. As a result, the quality and diversity of the generated base clusterings are insufficient, thus affecting the subsequent ensemble performance. To address this issue, we propose a novel clustering ensemble algorithm for handling deep embeddings using cluster confidence (CEDECC) to improve the robustness and performance. Instead of simply combining deep clustering with clustering ensembles, we take into consideration that the performance of existing deep clustering methods heavily relies on the quality of low-dimensional embeddings generated during the pre-training stage. The quality of embeddings is unstable due to the influence of different initialization parameters. In CEDECC, specifically, we first construct a cluster confidence measure to evaluate the quality of low-dimensional embeddings. Typically, high-quality low-dimensional embeddings yield accurate clustering results with the same model parameters. Then, we utilize multiple high-quality embeddings to generate the base partitions. In the ensemble strategy phase, we consider the cluster-wise diversity and propose a novel ensemble cluster estimation to improve the overall consensus performance of the model. Extensive experiments on three benchmark datasets and four real-world biological datasets have demonstrated that the proposed CEDECC consistently outperforms the state-of-the-art clustering ensemble methods.

On the Consistency and Large-Scale Extension of Multiple Kernel Clustering.

Existing multiple kernel clustering (MKC) algorithms have two ubiquitous problems. From the theoretical perspective, most MKC algorithms lack sufficient theoretical analysis, especially the consistency of learned parameters, such as the kernel weights. From the practical perspective, the high complexity makes MKC unable to handle large-scale datasets. This paper tries to address the above two issues. We first make a consistency analysis of an influential MKC method named Simple Multiple Kernel k-Means (SimpleMKKM). Specifically, suppose that ∧γn are the kernel weights learned by SimpleMKKM from the training samples. We also define the expected version of SimpleMKKM and denote its solution as γ*. We establish an upper bound of ||∧γn-γ*||∞ in the order of ~O(1/√n), where n is the sample number. Based on this result, we also derive its excess clustering risk calculated by a standard clustering loss function. For the large-scale extension, we replace the eigen decomposition of SimpleMKKM with singular value decomposition (SVD). Consequently, the complexity can be decreased to O(n) such that SimpleMKKM can be implemented on large-scale datasets. We then deduce several theoretical results to verify the approximation ability of the proposed SVD-based method. The results of comprehensive experiments demonstrate the superiority of the proposed method.

A novel approach to energy management in the dairy industry using performance indicators and load profiles: Application to a cheese dairy plant in Tuscany, Italy

The available energy monitoring information in the dairy industry reveals significant gaps regarding the impact of operational factors on performance indicators and the generation of typical days from load profiles. This paper presents a method for predicting performance indicators and load profiles in the dairy industry based on multiple regression and clustering. The method is applied to a cheese dairy plant located in Tuscany, Italy, providing actionable insights for energy efficiency and renewable integration. With regard to performance indicators, predictions using multiple regression achieved accuracies within 8 % for electricity consumption and within 20 % for steam generation, mainly due to limited data availability. The combination of k-means clustering with multiple regression yielded an overall accuracy within approximately 10 % for electricity load profiles, enabling the labelling of clusters based on production and meteorological variables. The analysis of improved production planning and the compatibility of energy demand with solar resources showed potential reductions in the electrical performance indicator by 27 % and self-consumption rates between 14 % and 42 %, respectively. Validation with data from other dairy contexts confirms the accuracy of the method and the considerable potential for improvement, suggesting further implementation towards effective energy management in the dairy industry.

Auto-weighted Graph Reconstruction for efficient ensemble clustering

Ensemble clustering (EC) is an important field in data clustering, which aims to combine multiple base clustering results of a given dataset into a single consensus result. Previous studies have shown that one of the effective methods is to convert the base clustering results into a graph partitioning problem. Existing spectral ensemble clustering methods typically obtain a spectral embedding matrix for the graph and then perform k-means, separating the graph construction and clustering tasks into two distinct stages. To address these challenges, we propose a novel and efficient ensemble clustering method named Auto-weighted Graph Reconstruction for efficient ensemble clustering. This method integrates weighted base clusters, consensus distance graph construction, and clustering into a unified framework. A consensus distance graph is constructed, from which a stable and discrete label matrix is derived using spectral clustering. An effective optimization algorithm is then employed to address the resulting problem. Finally, the experimental results on eight real-world datasets verify the effectiveness and superiority of the proposed method.

Joint consensus kernel learning and adaptive hypergraph regularization for graph-based clustering

Recent advancements in multiple kernel learning-based graph clustering methods have demonstrated significant promise in effectively learning a consensus kernel matrix from various candidate kernel matrices. This approach enables the creation of a low-dimensional representation in the kernel space of high-dimensional data through self-expressiveness. However, a key challenge remains in capturing the latent geometric properties embedded within different kernel matrices to enhance data representation. In this paper, we propose a novel method called joint consensus kernel learning and adaptive hypergraph regularization for graph-based clustering (JKHR). Our approach integrates an innovative adaptive hypergraph Laplacian regularizer, which is characterized by the fusion of multiple nearest neighbor kernel graphs, into the multiple kernel learning-based graph clustering framework. JKHR jointly and adaptively optimizes both the consensus kernel matrix and the hypergraph Laplacian regularizer to achieve a low-dimensional representation that effectively preserves the intrinsic geometry of the data. Experimental results on both synthetic and real benchmark datasets demonstrate that JKHR outperforms state-of-the-art self-expressiveness-based graph clustering methods as well as traditional clustering techniques.

Tuning Vision-Language Models with Multiple Prototypes Clustering.

Benefiting from advances in large-scale pre-training, foundation models, have demonstrated remarkable capability in the fields of natural language processing, computer vision, among others. However, to achieve expert-level performance in specific applications, such models often need to be fine-tuned with domain-specific knowledge. In this paper, we focus on enabling vision-language models to unleash more potential for visual understanding tasks under few-shot tuning. Specifically, we propose a novel adapter, dubbed as lusterAdapter, which is based on trainable multiple prototypes clustering algorithm, for tuning the CLIP model. It can not only alleviate the concern of catastrophic forgetting of foundation models by introducing anchors to inherit common knowledge, but also improve the utilization efficiency of few annotated samples via bringing in clustering and domain priors, thereby improving the performance of few-shot tuning. We have conducted extensive experiments on 11 common classification benchmarks. The results show our method significantly surpasses the original CLIP and achieves state-of-the-art (SOTA) performance under all benchmarks and settings. For example, under the 16-shot setting, our method exhibits a remarkable improvement over the original CLIP by 19.6%, and also surpasses TIP-Adapter and GraphAdapter by 2.7% and 2.2%, respectively, in terms of average accuracy across the 11 benchmarks. Code is available at https://github.com/uyzhang/Cluster-Adapter.

3D indoor area recognition for personnel security using integrated UWB and barometer approach

Real-time location tracking is essential for personal security in industrial scenes, e.g. monitoring worker safety in power substations. Ultra wideband (UWB) technology is suitable for indoor positioning thanks to its high penetration capability, high ranging accuracy and low power consumption. However, UWB based trilateration positioning requires high workload for field deployment of base stations. Indoor complex topology results in multipath and non-line of sight (NLOS) conditions of UWB signals, and degrades the positioning performance in terms of accuracy and reliability. This paper proposes a three-dimensional (3D) area recognition solution by integrating UWB time of flight (TOF) ranging and barometer measurements. The proposed solution utilizes a multi-tier distributed joint probabilistic inference model, which accomplishes the indoor 3D area recognition exploiting multiple clustering and prediction algorithms of machine learning. The field experiments showed that the proposed method can achieve an accuracy of 3D area recognition of more than 99.2%. The proposed method improves the computing efficiency by 93%. The errors of improved differential barometric height estimation method are less than 1 m, which means a success rate of 100% for floor identification, given a floor separation of 3–4 m. The proposed solution is suitable for personnel security applications of industrial scenes, which requires reliable real-time area information rather than just coordinates.

Mixtures of Probit Regression Models with Overlapping Clusters

Studies with binary outcomes on a heterogeneous population are quite common. Typically, the heterogeneity is modelled through varying effect coefficients within some binary regression setting combined with a clustering procedure. Most of the existing methods assign statistical units to distinct and non-overlapping clusters. However, there are scenarios where units exhibit a more complex organization and the clusters can be thought as partially overlapping. In this case, the standard approach does not work. In this paper, we define a mixture of regression models that allows overlapping clusters. This approach involves an overlap function that maps the regression coefficients, either at the unit or response level, of the parent clusters into the coefficients of the multiple allocation clusters. In order to deal with this intrinsic heterogeneity, regression analyses have to be stratified for different groups of observations or clusters. We present a computationally efficient Monte Carlo Markov Chain (MCMC) scheme for the case of a mixture of probit regressions. A simulation study shows the overall performance of the method. We conclude with two illustrative examples of modelling voting behavior, involving United States (US) Supreme Court justices over a number of topics and members of the United Kingdom (UK) parliament over divisions related to Brexit. These applications provide insights on the usefulness of the method in real applications. The method described can be extended to the case of a generic mixture of multivariate generalized linear models under overlapping clusters.

Active Clustering Ensemble With Self-Paced Learning.

A clustering ensemble provides an elegant framework to learn a consensus result from multiple prespecified clustering partitions. Though conventional clustering ensemble methods achieve promising performance in various applications, we observe that they may usually be misled by some unreliable instances due to the absence of labels. To tackle this issue, we propose a novel active clustering ensemble method, which selects the uncertain or unreliable data for querying the annotations in the process of the ensemble. To fulfill this idea, we seamlessly integrate the active clustering ensemble method into a self-paced learning framework, leading to a novel self-paced active clustering ensemble (SPACE) method. The proposed SPACE can jointly select unreliable data to label via automatically evaluating their difficulty and applying easy data to ensemble the clusterings. In this way, these two tasks can be boosted by each other, with the aim to achieve better clustering performance. The experimental results on benchmark datasets demonstrate the significant effectiveness of our method. The codes of this article are released in https://Doctor-Nobody.github.io/codes/space.zip.

Optimisation of multiple clustering based undersampling using artificial bee colony: Application to improved detection of obfuscated patterns without adversarial training

Attack detection is one of the main features required in modern defence systems. Despite the ongoing research, it remains challenging for a typical mechanism like network-based intrusion detection system (NIDS) to catch up with evolving adversarial attacks. They specifically aim to confuse a machine-learning based predictor. Without the knowledge of adversarial patterns, the best approach is generalising signatures learned from a dataset of legitimate connections and known intrusions. This work focuses on analysing non-payload traffics so that the resulting techniques can be exploited to a range of network-based applications. It investigates a novel means to deal with the problem of imbalanced classes. An optimised undersampling method is introduced to select a subset of majority-class representatives initially created through an ensemble clustering procedure. A weighted combination of criteria representing distributions within and between classes is proposed as the objective function for a global optimisation using the artificial bee colony (ABC). This approach usually outperforms its baselines and other state-of-the-art undersampling models, with ABC being more effective using the global best strategy than a random selection of solutions or an iterative greedy search. The paper also details the parameter analysis offering a heuristic guide for potential taking up of the proposed techniques.

Comparison of gene-by-gene and genome-wide short nucleotide sequence-based approaches to define the global population structure of Streptococcus pneumoniae.

Defining the population structure of a pathogen is a key part of epidemiology, as genomically related isolates are likely to share key clinical features such as antimicrobial resistance profiles and invasiveness. Multiple different methods are currently used to cluster together closely related genomes, potentially leading to inconsistency between studies. Here, we use a global dataset of 26 306 Streptococcus pneumoniae genomes to compare four clustering methods: gene-by-gene seven-locus MLST, core genome MLST (cgMLST)-based hierarchical clustering (HierCC) assignments, life identification number (LIN) barcoding and k-mer-based PopPUNK clustering (known as GPSCs in this species). We compare the clustering results with phylogenetic and pan-genome analyses to assess their relationship with genome diversity and evolution, as we would expect a good clustering method to form a single monophyletic cluster that has high within-cluster similarity of genomic content. We show that the four methods are generally able to accurately reflect the population structure based on these metrics and that the methods were broadly consistent with each other. We investigated further to study the discrepancies in clusters. The greatest concordance was seen between LIN barcoding and HierCC (adjusted mutual information score=0.950), which was expected given that both methods utilize cgMLST, but have different methods for defining an individual cluster and different core genome schema. However, the existence of differences between the two methods shows that the selection of a core genome schema can introduce inconsistencies between studies. GPSC and HierCC assignments were also highly concordant (AMI=0.946), showing that k-mer-based methods which use the whole genome and do not require the careful selection of a core genome schema are just as effective at representing the population structure. Additionally, where there were differences in clustering between these methods, this could be explained by differences in the accessory genome that were not identified in cgMLST. We conclude that for S. pneumoniae, standardized and stable nomenclature is important as the number of genomes available expands. Furthermore, the research community should transition away from seven-locus MLST, whilst cgMLST, GPSC and LIN assignments should be used more widely. However, to allow for easy comparison between studies and to make previous literature relevant, the reporting of multiple clustering names should be standardized within the research.

TFMKC: Tuning-Free Multiple Kernel Clustering Coupled With Diverse Partition Fusion.

Clustering is a popular research pipeline in unsupervised learning to find potential groupings. As a representative paradigm in multiple kernel clustering (MKC), late fusion-based models learn a consistent partition across multiple base kernels. Despite their promising performance, a common concern is the limited representation capacity caused by the inflexible fusion mechanism. Concretely, the representations are constrained by truncated- k Eigen-decomposition (EVD) without fully exploiting potential information. An intuitive idea to alleviate this concern is to generate a set of augmented partitions and then select the optimal partition by fine-tuning. However, this is overlimited by: 1) introducing undesired hyperparameters and dataset-related consequences; 2) neglecting rich information across diverse partitions; and 3) expensive parameter-tuning costs. To address these problems, we propose transforming the challenging problem of directly determining the optimal partition (optimal parameter) into a diverse partition fusion (parameter ensemble) problem. We design a novel flexible fusion mechanism called tuning-free multiple kernel clustering coupled with diverse partition fusion (TFMKC) by reweighting diverse partitions through optimization, achieving an optimal consensus partition by integrating diverse and complementary information rather than traditional fine-tuning, and distinguishing our work from existing methods. Extensive experiments verify that TFMKC achieves competitive effectiveness and efficiency over comparison baselines. The code can be accessed at https://github.com/ZJP/TFMKC.

MICFOA: A Novel Improved Catch Fish Optimization Algorithm with Multi-Strategy for Solving Global Problems.

Catch fish optimization algorithm (CFOA) is a newly proposed meta-heuristic algorithm based on human behaviors. CFOA shows better performance on multiple test functions and clustering problems. However, CFOA shows poor performance in some cases, and there is still room for improvement in convergence accuracy, getting rid of local traps, and so on. To further enhance the performance of CFOA, a multi-strategy improved catch fish optimization algorithm (MICFOA) is proposed in this paper. In the exploration phase, we propose a Lévy-based differential independent search strategy to enhance the global search capability of the algorithm while minimizing the impact on the convergence speed. Secondly, in the exploitation phase, a weight-balanced selection mechanism is used to maintain population diversity, enhance the algorithm's ability to get rid of local optima during the search process, and effectively boost the convergence accuracy. Furthermore, the structure of CFOA is also modified in this paper. A fishermen position replacement strategy is added at the end of the algorithm as a way to strengthen the robustness of the algorithm. To evaluate the performance of MICFOA, a comprehensive comparison with nine other metaheuristic algorithms is performed on the 10/30/50/100 dimensions of the CEC 2017 test functions and the 10/20 dimensions of the CEC2022 test functions. Statistical experiments show that MICFOA has more significant dominance in numerical optimization problems, and its overall performance outperforms the CFOA, PEOA, TLBO, COA, ARO, EDO, YDSE, and other state-of-the-art algorithms such as LSHADE, JADE, IDE-EDA, and APSM-jSO.

Machine learning identifies clusters of the normal adolescent spine based on sagittal balance.

This study applied a machine learning semi-supervised clustering approach to radiographs of adolescent sagittal spines from a single pediatric institution to identify patterns of sagittal alignment in the normal adolescent spine. We sought to explore the inherent variability found in adolescent sagittal alignment using machine learning to remove bias and determine whether clusters of sagittal alignment exist. Multiple semi-supervised machine learning clustering algorithms were applied to 111 normal adolescent sagittal spines. Sagittal parameters for resultant clusters were determined. Machine learning analysis found that the spines did cluster into distinct groups with an optimal number of clusters ranging from 3 to 5. We performed an analysis on both 3 and 5-cluster groups. The 3-cluster groups analysis found good consistency between methods with 96 of 111, while the analysis of 5-cluster groups found consistency with 105 of 111 spines. When assessing for differences in sagittal parameters between the groups for both analyses, there were differences in T4-12 TK, L1-S1 LL, SS, SVA, PI-LL mismatch, and TPA. However, the only parameter that was statistically different for all groups was SVA. Based on machine learning, the adolescent sagittal spine alignments do cluster into distinct groups. While there were distinguishing features with TK and LL, the most important parameter distinguishing these groups was SVA. Further studies may help to understand these findings in relation to spinal deformities.

Leveraging ensemble clustering for privacy-preserving data fusion: Analysis of big social-media data in tourism

Discovering knowledge from social media becomes a trend in many domains such as tourism, where users' feedback and rating are the basis of recommendation systems. In this context, cluster analysis has been a major tool to disclose user groups by which the process of collaborative filtering can better determine a personalised suggestion. Matching this to the curse of big data is a challenge with previous studies either implementing conventional techniques on a distributed system or making use of data sampling. Specific to ensemble clustering, only a few aim to obtain both scalability and privacy preserving that are significant to handling social data. This paper presents a new bi-level framework of ensemble clustering in which an instance-segment based analysis is adopted to ensure data privacy and reduce the complexity of clustering the whole dataset. Unlike existing studies, instead of drawing a single clustering from each segment, multiple clusterings are selected to better represent instances therein. Based on published tourism datasets and different experimental settings, the new approach usually outperforms its baselines whilst being competitive to related methods found in the literature. Additional case studies on simulated big datasets and noisy variations are reported and discussed in addition to the analysis of algorithmic parameters.

Optimised multiple data partitions for cluster-wise imputation of missing values in gene expression data

It is commonly agreed that the quality of data analysis may be degraded by the presence of missing data. In various domains such as bioinformatics, an effective tool is required for the discovery of knowledge from gene expression datasets. One may simply ignore defected samples, while others attempt to either make an algorithm robust to the problem or develop an imputation technique to fill in missing values. This research focuses on the latter and introduces a new hybridisation of cluster- and neighbour-based references to generate an accurate estimate. It also proposes a novel exploitation of multiple clusterings as the source of cluster-wise information, instead of a single data partition that has been studied by existing methods. These data partitions are selected from a pool of base clusterings with respect to both quality and diversity criteria. Another hybridisation is thus established between swarm intelligence and this search problem. In particular, the algorithm of artificial bee colony (ABC) is explored, with two new operators being invented to allow an evolution of solutions, or food sources for bees. Also, two different imputation strategies are provided to generate entries of missing entries in a data matrix, called cluster-only and cluster-neighbour. Based on published gene expression datasets and different experimental settings, the resulting models usually outperform their baselines and recent approaches, which make use of cluster analysis or devise an intelligent determination of nearest neighbours. Furthermore, they have proven competitive to a benchmark technique belonging to the global approach, especially with high missing ratios. Further extensions to iterative refinement and supervised imputation are discussed in addition to parameter analysis.

Building shape-focused pharmacophore models for effective docking screening

The performance of molecular docking can be improved by comparing the shape similarity of the flexibly sampled poses against the target proteins’ inverted binding cavities. The effectiveness of these pseudo-ligands or negative image-based models in docking rescoring is boosted further by performing enrichment-driven optimization. Here, we introduce a novel shape-focused pharmacophore modeling algorithm O-LAP that generates a new class of cavity-filling models by clumping together overlapping atomic content via pairwise distance graph clustering. Top-ranked poses of flexibly docked active ligands were used as the modeling input and multiple alternative clustering settings were benchmark-tested thoroughly with five demanding drug targets using random training/test divisions. In docking rescoring, the O-LAP modeling typically improved massively on the default docking enrichment; furthermore, the results indicate that the clustered models work well in rigid docking. The C+ +/Qt5-based algorithm O-LAP is released under the GNU General Public License v3.0 via GitHub (https://github.com/jvlehtonen/overlap-toolkit).Scientific contributionThis study introduces O-LAP, a C++/Qt5-based graph clustering software for generating new type of shape-focused pharmacophore models. In the O-LAP modeling, the target protein cavity is filled with flexibly docked active ligands, the overlapping ligand atoms are clustered, and the shape/electrostatic potential of the resulting model is compared against the flexibly sampled molecular docking poses. The O-LAP modeling is shown to ensure high enrichment in both docking rescoring and rigid docking based on comprehensive benchmark-testing.Graphical

One-Stage Shifted Laplacian Refining for Multiple Kernel Clustering.

Graph learning can effectively characterize the similarity structure of sample pairs, hence multiple kernel clustering based on graph learning (MKC-GL) achieves promising results on nonlinear clustering tasks. However, previous methods confine to a "three-stage" scheme, that is, affinity graph learning, Laplacian construction, and clustering indicator extracting, which results in the information distortion in the step alternating. Meanwhile, the energy of Laplacian reconstruction and the necessary cluster information cannot be preserved simultaneously. To address these problems, we propose a one-stage shifted Laplacian refining (OSLR) method for multiple kernel clustering (MKC), where using the "one-stage" scheme focuses on Laplacian learning rather than traditional graph learning. Concretely, our method treats each kernel matrix as an affinity graph rather than ordinary data and constructs its corresponding Laplacian matrix in advance. Compared to the traditional Laplacian methods, we transform each Laplacian to an approximately shifted Laplacian (ASL) for refining a consensus Laplacian. Then, we project the consensus Laplacian onto a Fantope space to ensure that reconstruction information and clustering information concentrate on larger eigenvalues. Theoretically, our OSLR reduces the memory complexity and computation complexity to O(n) and O(n2) , respectively. Moreover, experimental results have shown that it outperforms state-of-the-art MKC methods on multiple benchmark datasets.