Multiple Clustering Methods Research Articles

A novel combined dynamic ensemble selection model for imbalanced data to detect COVID-19 from complete blood count

BackgroundAs blood testing is radiation-free, low-cost and simple to operate, some researchers use machine learning to detect COVID-19 from blood test data. However, few studies take into consideration the imbalanced data distribution, which can impair the performance of a classifier. MethodA novel combined dynamic ensemble selection (DES) method is proposed for imbalanced data to detect COVID-19 from complete blood count. This method combines data preprocessing and improved DES. Firstly, we use the hybrid synthetic minority over-sampling technique and edited nearest neighbor (SMOTE-ENN) to balance data and remove noise. Secondly, in order to improve the performance of DES, a novel hybrid multiple clustering and bagging classifier generation (HMCBCG) method is proposed to reinforce the diversity and local regional competence of candidate classifiers. ResultsThe experimental results based on three popular DES methods show that the performance of HMCBCG is better than only use bagging. HMCBCG+KNE obtains the best performance for COVID-19 screening with 99.81% accuracy, 99.86% F1, 99.78% G-mean and 99.81% AUC. ConclusionCompared to other advanced methods, our combined DES model can improve accuracy, G-mean, F1 and AUC of COVID-19 screening.

VRPDiv: A Divide and Conquer Framework for Large Vehicle Routing Problems

The Vehicle Routing Problem (VRP) is an NP hard problem where we need to optimize itineraries for agents to visit multiple targets. When considering real-world travel (road-network topology, speed limits and traffic), modern VRP solvers can only process small instances with a few hundred targets. We propose a framework (VRPDiv) that can scale any solver to support larger VRP instances with up to ten thousand targets (10k) by dividing them into smaller clusters. VRPDiv supports the multiple VRP scenarios and contains a pool of clustering algorithms from which it chooses the ideal one depending on properties of the instance. VRPDiv assigns agents based on cluster demand and targets compatibility (i.e. realizable time-windows and capacity limitations). We incorporate the framework into the Bing Maps Multi-Itinerary Optimization (MIO) 1 online service. This architecture allows MIO to scale up from solving instances with a few hundred to over 10k targets in under 10 minutes. We evaluate our framework on public datasets and publish a new dataset ourselves, as large enough instances supporting real-world travel were impossible to find. We investigate multiple clustering methods and show that choosing the correct one is critical with differences of up to 60% in quality. We compare with relevant baselines and report a 40% improvement in target allocation and a 9.8% improvement in itinerary durations. We compare with existing scores and report an average delta of 10%, with lower values (<5%) in instances with low workload (few targets per agent), which are acceptable for an online service.

Multiple kernel clustering with late fusion consensus local graph preserving

Multiple kernel clustering (MKC) methods aim at integrating an optimal partition from a set of precalculated kernel matrices. Though achieving success in various applications, we observe that existing MKC methods: (i) lack of representation flexibility; and (ii) do not considerably preserve the locality structure in partition space. These issues may adversely affect the learning procedure of MKC, leading to unsatisfying clustering performance. In this paper, we propose a late fusion MKC method with local graph refinement to address the aforementioned issues. Different from existing MKC mechanisms, our method unifies the traditional weighted multiple kernel k-means, kernel partition, and graph construction into a single optimization procedure. The local graph is utilized to preserve the locality information in partition space and therefore all of the counterparts can be boosted for mutual clustering improvements. By this way, our approach enhances the local graph structure in partition space and enjoys more flexible kernel representations, leading to significant clustering improvements. Moreover, a three-step alternate algorithm is developed to solve the resultant optimization problem with proved convergence. Extensive experiments are conducted on several multiple kernel benchmark datasets to compare the proposed algorithm with the state-of-the-art ones, and the results well demonstrate its effectiveness and superiority.

Clustering of Multiple Psychiatric Disorders Using Functional Connectivity in the Data-Driven Brain Subnetwork

Recently, the dimensional approach has attracted much attention, bringing a paradigm shift to a continuum of understanding of different psychiatric disorders. In line with this new paradigm, we examined whether there was common functional connectivity related to various psychiatric disorders in an unsupervised manner without explicitly using diagnostic label information. To this end, we uniquely applied a newly developed network-based multiple clustering method to resting-state functional connectivity data, which allowed us to identify pairs of relevant brain subnetworks and subject cluster solutions accordingly. Thus, we identified four subject clusters, which were characterized as major depressive disorder (MDD), young healthy control (young HC), schizophrenia (SCZ)/bipolar disorder (BD), and autism spectrum disorder (ASD), respectively, with the relevant brain subnetwork represented by the cerebellum-thalamus-pallidum-temporal circuit. The clustering results were validated using independent datasets. This study is the first cross-disorder analysis in the framework of unsupervised learning of functional connectivity based on a data-driven brain subnetwork.

Robust subspace clustering based on automatic weighted multiple kernel learning

Multiple kernel learning (MKL), which combines a set of prespecified basic kernels to improve the clustering performance, has become an important research topic. Unfortunately, the current methods have the following defects in noisy circumstances. 1) Their clustering performance may be significantly reduced due to the noise in the kernel, which is caused by the lack of a reliable discriminant guideline for basic kernel combinations. 2) The noise from corrupted data or occlusion may destroy the block-diagonal structures of the affinity matrices they obtained, which will affect the clustering performance when using spectral clustering. In this work, to solve the above problems, we propose an automatic weighted multikernel learning-based robust subspace clustering (AWLKSC) algorithm. The model integrates multikernel learning strategies, the Correntropy-Induced Metric (CIM), low rank approximation technology and block diagonal constraints. In addition, an effective AM&GST algorithm, which is integrated by alternating minimization and generalized soft-thresholding, is developed to optimize the AWLKSC. Seven types of noise are considered in the experiments, and the experimental results illustrate that AWLKSC is more effective and robust than several up-to-date single kernel and multiple kernel clustering methods.

Multiple clustering for identifying subject clusters and brain sub-networks using functional connectivity matrices without vectorization

In neuroscience, the functional magnetic resonance imaging (fMRI) is a vital tool to non-invasively access brain activity. Using fMRI, the functional connectivity (FC) between brain regions can be inferred, which has contributed to a number of findings of the fundamental properties of the brain. As an important clinical application of FC, clustering of subjects based on FC recently draws much attention, which can potentially reveal important heterogeneity in subjects such as subtypes of psychiatric disorders. In particular, a multiple clustering method is a powerful analytical tool, which identifies clustering patterns of subjects depending on their FC in specific brain areas. However, when one applies an existing multiple clustering method to fMRI data, there is a need to simplify the data structure, independently dealing with elements in a FC matrix, i.e., vectorizing a correlation matrix. Such a simplification may distort the clustering results. To overcome this problem, we propose a novel multiple clustering method based on Wishart mixture models, which preserves the correlation matrix structure without vectorization. The uniqueness of this method is that the multiple clustering of subjects is based on particular networks of nodes (or regions of interest, ROIs), optimized in a data-driven manner. Hence, it can identify multiple underlying pairs of associations between a subject cluster solution and a ROI sub-network. The key assumption of the method is independence among sub-networks, which is effectively addressed by whitening correlation matrices. We applied the proposed method to synthetic and fMRI data, demonstrating the usefulness and power of the proposed method.

Autoencoder-like semi-NMF multiple clustering

Clustering is performed to partition samples into disjoint groups for facilitating the discovery of hidden patterns in the data. Many real-world applications involve various clustering methods , most of which only produce a single clustering. As a response to this issue, multiple clustering that aims to generate diverse and high-quality clustering, has emerged recently. This study proposes a novel autoencoder-like semi-nonnegative matrix factorization (NMF) multiple clustering (ASNMFMC) model that generates multiple non-redundant, high-quality clustering. The nonnegative property of the semi-NMF is utilized by the algorithm to enforce non-redundancy. Extensive experimental results demonstrate that the ASNMFMC is superior to the existing multiple clustering methods and can explore diverse high-quality clustering.

Combination of Immune-Related Genomic Alterations Reveals Immune Characterization and Prediction of Different Prognostic Risks in Ovarian Cancer.

With the highest case-fatality rate among women, the molecular pathological alterations of ovarian cancer (OV) are complex, depending on the diversity of genomic alterations. Increasing evidence supports that immune infiltration in tumors is associated with prognosis. Therefore, we aim to assess infiltration in OV using multiple methods to capture genomic signatures regulating immune events to identify reliable predictions of different outcomes. A dataset of 309 ovarian serous cystadenocarcinoma patients with overall survival >90 days from The Cancer Genome Atlas (TCGA) was analyzed. Multiple estimations and clustering methods identified and verified two immune clusters with component differences. Functional analyses pointed out immune-related alterations underlying internal genomic variables potentially. After extracting immune genes from a public database, the LASSO Cox regression model with 10-fold cross-validation was used for selecting genes associated with overall survival rate significantly, and a risk score model was then constructed. Kaplan–Meier survival and Cox regression analyses among cohorts were performed systematically to evaluate prognostic efficiency among the risk score model and other clinical pathological parameters, establishing a predictive ability independently. Furthermore, this risk score model was compared among identified signatures in previous studies and applied to two external cohorts, showing better prediction performance and generalization ability, and also validated as robust in association with immune cell infiltration in bulk tissues. Besides, a transcription factor regulation network suggested upper regulatory mechanisms in OV. Our immune risk score model may provide gyneco-oncologists with predictive values for the prognosis and treatment management of patients with OV.

School motivation profiles of Dutch 9th graders

The aim of this study was to identify school motivation profiles of Dutch 9th grade students in a four-dimensional motivation space, including mastery, performance, social and extrinsic motivation. Multiple clustering methods (K-means, K-medoids, restricted latent profile analysis) and multiple indices for selecting the optimal number of clusters were applied. The statistical selection methods did not completely concur on the optimal number of clusters, but a clear common denominator was provided by the Calinski-Harabasz index and the minimum and mean Silhouette values. All three indices indicated two clusters as the optimal number, regardless of the clustering method used: one cluster of 9th graders with high average scores on all dimensions and one cluster with low mean scores on all dimensions. In addition, we explored the substantive interpretation of multiple cluster solutions. It was discovered that most students are in clusters that can be classified into one of three profile types that may differ in level: (1) approximately equal mean scores on all dimensions, (2) relative high mean scores on mastery and social motivation, and (3) a relatively low mean score on performance motivation. The latter profile type is believed to be a new discovery.

A Fast and Effective Multiple Kernel Clustering Method on Incomplete Data

Multiple kernel clustering is an unsupervised data analysis method that has been used in various scenarios where data is easy to be collected but hard to be labeled. However, multiple kernel clustering for incomplete data is a critical yet challenging task. Although the existing absent multiple kernel clustering methods have achieved remarkable performance on this task, they may fail when data has a high value-missing rate, and they may easily fall into a local optimum. To address these problems, in this paper, we propose an absent multiple kernel clustering (AMKC) method on incomplete data. The AMKC method first clusters the initialized incomplete data. Then, it constructs a new multiple-kernel-based data space, referred to as <i>K</i>-space, from multiple sources to learn kernel combination coefficients. Finally, it seamlessly integrates an incomplete-kernel-imputation objective, a multiple-kernel-learning objective, and a kernel-clustering objective in order to achieve absent multiple kernel clustering. The three stages in this process are carried out simultaneously until the convergence condition is met. Experiments on six datasets with various characteristics demonstrate that the kernel imputation and clustering performance of the proposed method is significantly better than state-of-the-art competitors. Meanwhile, the proposed method gains fast convergence speed.

Adaptive Spectral Rotation via Joint Cluster and Pairwise Structure

Density structure and pairwise structure serve as two different but complementary perspectives for clustering. Either side of road is frequently visited and explored by multiple clustering methods. However, there are seldom approaches, which could mutually exploit both structures for clustering. To address this problem, in this paper, we develop a novel adaptive joint clustering algorithm, which combines unsupervised discrete orthogonal least squares discriminant analysis (DOLSDA) and discrete spectral clustering (DSC) with adaptive neighbors and side information into a unified model. Firstly, we extend supervised OLSDA to a discrete kernel clustering problem. To further achieve a clear pairwise structure, a new similarity with adaptive neighbors is then derived to establish sparse Laplacian matrix. In addition, side information could be incorporated to formulate clearer graph by modifying the proposed similarity. Based on the constructed graph, DSC is embedded with the discrete kernel OLSDA (DKOLSDA) clustering to exploit both cluster and pairwise data structures. Equipped with the proposed framework regarding quadratic weighted optimization, adaptive weight can be obtained automatically to leverage both unsupervised DKOLSDA and DSC. Since the unified problem is still discrete, we develop an increment scheme to achieve the optimal spectral rotation for the approximate solution to the predicted indicator.

GCAnno: a graph-based single cell type annotation method

BackgroundCurrent single cell analysis methods annotate cell types at cluster-level rather than ideally at single cell level. Multiple exchangeable clustering methods and many tunable parameters have a substantial impact on the clustering outcome, often leading to incorrect cluster-level annotation or multiple runs of subsequent clustering steps. To address these limitations, methods based on well-annotated reference atlas has been proposed. However, these methods are currently not robust enough to handle datasets with different noise levels or from different platforms.ResultsHere, we present gCAnno, a graph-based Cell type Annotation method. First, gCAnno constructs cell type-gene bipartite graph and adopts graph embedding to obtain cell type specific genes. Then, naïve Bayes (gCAnno-Bayes) and SVM (gCAnno-SVM) classifiers are built for annotation. We compared the performance of gCAnno to other state-of-art methods on multiple single cell datasets, either with various noise levels or from different platforms. The results showed that gCAnno outperforms other state-of-art methods with higher accuracy and robustness.ConclusionsgCAnno is a robust and accurate cell type annotation tool for single cell RNA analysis. The source code of gCAnno is publicly available at https://github.com/xjtu-omics/gCAnno.

A systematic performance evaluation of clustering methods for single-cell RNA-seq data.

Subpopulation identification, usually via some form of unsupervised clustering, is a fundamental step in the analysis of many single-cell RNA-seq data sets. This has motivated the development and application of a broad range of clustering methods, based on various underlying algorithms. Here, we provide a systematic and extensible performance evaluation of 14 clustering algorithms implemented in R, including both methods developed explicitly for scRNA-seq data and more general-purpose methods. The methods were evaluated using nine publicly available scRNA-seq data sets as well as three simulations with varying degree of cluster separability. The same feature selection approaches were used for all methods, allowing us to focus on the investigation of the performance of the clustering algorithms themselves. We evaluated the ability of recovering known subpopulations, the stability and the run time and scalability of the methods. Additionally, we investigated whether the performance could be improved by generating consensus partitions from multiple individual clustering methods. We found substantial differences in the performance, run time and stability between the methods, with SC3 and Seurat showing the most favorable results. Additionally, we found that consensus clustering typically did not improve the performance compared to the best of the combined methods, but that several of the top-performing methods already perform some type of consensus clustering. All the code used for the evaluation is available on GitHub ( https://github.com/markrobinsonuzh/scRNAseq_clustering_comparison). In addition, an R package providing access to data and clustering results, thereby facilitating inclusion of new methods and data sets, is available from Bioconductor ( https://bioconductor.org/packages/DuoClustering2018).

Multiple kernel low-rank representation-based robust multi-view subspace clustering

Owing to the presence of complex noise, it is extremely challenging to learn a low-dimensional subspace structure directly from the original data. In addition, the nonlinear structure of the data makes multi-view subspace clustering more difficult. In this paper, we propose a multiple kernel low-rank representation-based robust multi-view subspace clustering method (MKLR-RMSC) that combines a learnable low-rank multiple kernel trick with co-regularization. MKLR-RMSC mainly conducts the following four tasks: 1) fully mining the complementary information provided by the different views in the feature spaces, 2) the containment of multiple low-dimensional subspaces in the feature space data, 3) allowing all view-specific representations towards a common centroid, and 4) effectively dealing with non-Gaussian noise in data. In our model, the weighted Schatten p-norm is applied to fully explore the effects of different ranks while approaching the original low-rank hypothesis. Moreover, different predefined learning kernel matrices are designed for different views, which is more conducive to mining the unique and complementary information of different views. In addition, as a robust measure, correntropy is applied in MKLR-RMSC. Our method is more effective and robust than several of the most advanced methods on six commonly used datasets.

Multiple kernel clustering with pure graph learning scheme

Existing graph-based multiple kernel clustering (GMKC) methods usually adopt a hybridization scheme, termed HGMKC for short, which binds multiple kernel learning (MKL) and graph-based clustering (GBC) together, the former aims to construct a consensus kernel and the latter is used to learn an affinity graph based on the consensus kernel. Obviously, HGMKC performs graph learning and spectral clustering separately. Additionally, the performance of this hybridization is largely determined by MKL, which is empirically contrary to the fact that graph learning is the key of graph-based methods, rather than kernel learning. In this paper, we propose a pure GMKC method, dubbed PGMKC, which contains two parts, including candidate kernel graphs learning (CKGL) and kernel graph fusion (KGF). Specially, CKGL concentrates on constructing multiple candidate kernel graphs in Hilbert kernel space relying on kernel self-expressiveness; KGF leverages a flexibly auto-weighted graph fusion strategy and a connectivity regularizer to yield a consensus kernel graph directly. Compared to HGMKC, PGMKC avoids paying too much attention to consensus kernel construction, thus can focus more on affinity graph learning. Moreover, an efficient and effective optimization algorithm is developed to solve the proposed model. Experimental results on ten benchmark datasets demonstrate that the proposed PGMKC performs better than the state-of-the-art HGMKC competitors.

Leveraging TCGA gene expression data to build predictive models for cancer drug response

BackgroundMachine learning has been utilized to predict cancer drug response from multi-omics data generated from sensitivities of cancer cell lines to different therapeutic compounds. Here, we build machine learning models using gene expression data from patients’ primary tumor tissues to predict whether a patient will respond positively or negatively to two chemotherapeutics: 5-Fluorouracil and Gemcitabine.ResultsWe focused on 5-Fluorouracil and Gemcitabine because based on our exclusion criteria, they provide the largest numbers of patients within TCGA. Normalized gene expression data were clustered and used as the input features for the study. We used matching clinical trial data to ascertain the response of these patients via multiple classification methods. Multiple clustering and classification methods were compared for prediction accuracy of drug response. Clara and random forest were found to be the best clustering and classification methods, respectively. The results show our models predict with up to 86% accuracy; despite the study’s limitation of sample size. We also found the genes most informative for predicting drug response were enriched in well-known cancer signaling pathways and highlighted their potential significance in chemotherapy prognosis.ConclusionsPrimary tumor gene expression is a good predictor of cancer drug response. Investment in larger datasets containing both patient gene expression and drug response is needed to support future work of machine learning models. Ultimately, such predictive models may aid oncologists with making critical treatment decisions.

Inter-Subject Clustering of Brain Fibers from Whole-Brain Tractography.

This work presents an effective multiple subject clustering method using whole-brain tractography datasets. The method is able to obtain fiber clusters that are representative of the population. The proposed approach first applies a fast intra-subject clustering algorithm on each subject obtaining the cluster centroids for all subjects. Second, it compresses the collection of centroids to a latent space through the encoder of a trained autoencoder. Finally, it uses a modified HDBSCAN with adjusted parameters on the encoded centroids of all subjects to obtain the final inter-subject clusters. The results shows that the proposed method outperforms other clustering strategies, and it is able to retrieve known fascicles in a reasonable execution time, achieving a precision over 87% and F1 score above 86% on a collection of 20 simulated subjects.

Tensor-Based Multiple Clustering Approaches for Cyber-Physical-Social Applications

In multiple analysis tasks and personalized services, tremendous challenges in Cyber-Physical-Social Systems (CPSS) are clustering large-scale multi-source data and generating multiple distinct clusterings dependent on different applications. To address these challenges, this paper first presents two simple multiple clustering methods which can produce different clustering results according to arbitrarily selected combinations of features, one is similarity matrices-based multiple clusterings which computes the weighted average of similarity matrices for selected feature spaces, another is Euclidean distance-based multiple clusterings which fuses different feature spaces using selective weighted Euclidean distance. Furthermore, a tensor decomposition-based multiple clusterings is presented for efficiently clustering high-dimensional data, and a multi-relational attribute ranking method is further proposed to improve the clustering performance. This paper illustrates and evaluates the proposed methods on a design example and a real world data set. Experimental results show that the proposed methods can effectively cluster big data to provide enhanced knowledge extractions and services in CPSS.

Discovering Multiple Co-Clusterings With Matrix Factorization.

Clustering is a fundamental data exploration task which aims at discovering the hidden grouping structure in the data. The traditional clustering methods typically compute a single partition. However, there often exist different and equally meaningful clusterings in complex data. To solve this issue, multiple clustering approaches have emerged with the goal of exploring alternative clusterings from different perspectives. Existing solutions to this problem mainly focus on one-way clustering, that is, they cluster either the samples or the features. However, for many practical tasks, it is meaningful and desirable to explore alternative two-way clusterings (or co-clusterings), which capture not only the sample cluster structure but also the feature cluster structure. To tackle this interesting and unresolved task, we introduce an approach, called multiple co-clusterings (MultiCCs), to generate multiple alternative co-clusterings at the same time. MultiCC takes advantage of matrix tri-factorization to seek the co-clustering indicator matrices for samples and features and defines the row and column redundancy quantification terms to enforce diversity among co-clusterings based on these indicator matrices. After that, it integrates matrix tri-factorization and two nonredundancy terms into a unified objective function and gives an alternative optimization procedure to optimize the objective function. Extensive experimental results demonstrate that MultiCC performs significantly better than the existing multiple clustering methods. In addition, MultiCC can find out interesting co-clusters, which cannot be made by those comparing methods.

Privacy-preserving clustering for big data in cyber-physical-social systems: Survey and perspectives

Clustering technique plays a critical role in data mining, and has received great success to solve application problems like community analysis, image retrieval, personalized recommendation, activity prediction, etc. This paper first reviews the traditional clustering and the emerging multiple clustering methods, respectively. Although the existing methods have superior performance on some small or certain datasets, they fall short when clustering is performed on CPSS big data because of the high cost of computation and storage. With the powerful cloud computing, this challenge can be effectively addressed, but it brings enormous threat to individual or company’s privacy. Currently, privacy preserving data mining has attracted widespread attention in academia. Compared to other reviews, this paper focuses on privacy preserving clustering technique, guiding a detailed overview and discussion. Specifically, we introduce a novel privacy-preserving tensor-based multiple clustering, propose a privacy-preserving tensor-based multiple clustering analytic and service framework, and give an illustrated case study on the public transportation dataset. Furthermore, we indicate the remaining challenges of privacy preserving clustering and discuss the future significant research in this area.