Similarity Graph Construction Research Articles

Noise-aware clustering based on maximum correntropy criterion and adaptive graph regularization

Graph-based clustering is a basic subject in the field of machine learning, but most of them still have the following deficiencies. First, similarity graph construction and data division into corresponding classes are always divided into two independent steps. Second, noise contained in real data may cause the learned similarity graph to be inaccurate. Third, the traditional metrics based on Euclidean distance is difficult to tackle non-Gaussian noise. In order to eliminate these limitations, a noise-aware clustering based on correntropy and adaptive graph regularization method (NCCAGR) is proposed. 1) In order to change the problem from two-steps to single-step, we formulate a joint clustering learning framework that simultaneously learns a robust similarity graph and performs data clustering; 2) To overcome the influence of noise, we construct a Laplacian matrix and perform adaptive graph regularization based on clean data; 3) By introducing the correntropy to solve the problem of non-Gaussian noise and heavy tail in the original data. Furthermore, a half-quadratic optimization method is used to transform the problem into a quadratic form to facilitate subsequent solutions. Finally, experiments show that the proposed method not only has high performance, but also outperforms both classical methods and state-of-the-art methods in robustness.

Consensus similarity graph construction for clustering

Consensus similarity graph construction for clustering

Multi-view clustering with constructed bipartite graph in embedding space

Multi-view clustering is a hot topic in the field of machine learning and data mining, which aims to assign objects into clusters with multiple heterogeneous features. Additionally, considerable efforts have been devoted to this issue. However, current methods perform clustering in the original feature space, consequently failing to capture the latency of heterogeneous features. These algorithms assume the independence of feature selection and similarity graph construction, thereby resulting in an undesirable performance of algorithms. Furthermore, the construction of similarity graph only exploits the relations on vertex-level, ignoring the relation of cluster structure. To address these problems, we propose a novel Multi-view Clustering with Constructed Bipartite Graph in Embedding Space (MC2BGES) algorithm by jointly learning the common graph embedding representation among various views and similarity structure of vertices. Distinct from the current algorithms, MC2BGES learns the clustering structure of objects, which is transformed into the bipartite graph partition problem, thereby achieving a better performance. Moreover, the proposed model is formulated as a constrained optimization problem, which can be efficiently solved by an alternating optimization strategy. Extensive experiments on artificial and real-world datasets demonstrate that the proposed algorithm significantly outperforms state-of-the-art methods in terms of accuracy and robustness, implying that joint learning is promising for multi-view clustering.

Fast Fuzzy Clustering Based on Anchor Graph

Fuzzy clustering is one of the most popular clustering approaches and has attracted considerable attention in many fields. However, high computational cost has become a bottleneck which limits its applications in large-scale problems. Moreover, most fuzzy clustering algorithms are sensitive to noise. To address these issues, a novel fuzzy clustering algorithm, called fast fuzzy clustering based on anchor graph (FFCAG), is proposed. The FFCAG algorithm integrates anchor-based similarity graph construction and membership matrix learning into a unified framework, such that the prior knowledge of anchors can be further utilized to improve clustering performance. Specifically, FFCAG first constructs an anchor-based similarity graph with a parameter-free neighbor assignment strategy. Then, it designs a quadratic programming model to learn the membership matrix of anchors, which is very different from traditional fuzzy clustering algorithms. More importantly, a novel balanced regularization term is introduced into the objective function to produce more accurate clustering results. Finally, we adopt an alternating optimization algorithm with guaranteed convergence to solve the proposed method. Experimental results performed on synthetic and real-world datasets demonstrate the proposed FFCAG can significantly reduce the computational time with comparable, even superior, clustering performance, compared with state-of-the-art algorithms.

A new affinity matrix weighted k-nearest neighbors graph to improve spectral clustering accuracy.

Researchers have thought about clustering approaches that incorporate traditional clustering methods and deep learning techniques. These approaches normally boost the performance of clustering. Getting knowledge from large data-sets is quite an interesting task. In this case, we use some dimensionality reduction and clustering techniques. Spectral clustering is gaining popularity recently because of its performance. Lately, numerous techniques have been introduced to boost spectral clustering performance. One of the most significant part of these techniques is to construct a similarity graph. We introduced weighted k-nearest neighbors technique for the construction of similarity graph. Using this new metric for the construction of affinity matrix, we achieved good results as we tested it both on real and artificial data-sets.

Learning a maximized shared latent factor for cross-modal hashing

Due to the enormous advantages of high efficiency and low storage, hashing has already gained wide attention in the field of cross-modal search. Currently, most hashing methods learn unified hash codes for retrieval tasks by leveraging the inner characteristics of original data or the similarity graphs constructed by supervised information. However, both the two schemes neglect the discrepancy between distinct modalities in real-world scenarios that leads to poor search performance as well as limited search scalability. Toward this end, in this paper, we propose a novel supervised hashing method, termed Maximized Shared Latent Factor (MSLF) wherein the common property and unique properties of different modalities are taken into account simultaneously. Specifically, the common property (i.e., shared latent factor) generated by available label supervision without the similarity graph construction, can reduce the computation cost of hashing methods and thus make it suitable for cross-modal search applications. Meanwhile, the unique property (i.e., individual latent factor) obtained by each modality and the cross-correlation matching from data instances are designed to alleviate the structure difference between modalities. Furthermore, we combine these properties to learn a maximized shared latent factor for obtaining the hash codes, which enhances the search performance of MSLF. Extensive quantitative experiments on three popular datasets prove that the proposed MSLF outperforms the state-of-the-art hashing methods in both search performance and scalability.

Multi-Label Chest X-Ray Image Classification via Semantic Similarity Graph Embedding

Automated multi-label chest X-ray (CXR) image classification has recently made significant progress in clinical diagnosis based on the advanced deep learning techniques. However, most existing methods mainly focus on analyzing locality visual cues from a single image but fail to leverage the underlying explicit correlations among different images for precise disease diagnosis. By contrast, an experienced radiologist expertizes in transferring knowledge from previous tasks to diagnose the present radiograph. To enable the machine like a radiologist, this paper proposes a novel Semantic Similarity Graph Embedding (SSGE) framework, which explicitly explores the semantic similarities among images to optimize the visual feature embedding for improving the performance of multi-label CXR images classification. Specifically, the proposed SSGE framework contains three main components: the image feature embedding (IFE) module, similarity graph construction (SGC) module, and semantic similarity learning (SSL) module. To realize interactive teaching and learning between visual and semantic information, the proposed SSGE framework is built on the “Teacher-Student” (semantic-visual) learning mechanism. With the guidance and supervision of the cross-image similarity graph generated by the SGC module, the SSL module leverages Graph Convolutional Network (GCN) to adaptively recalibrate the multi-image feature representations extracted from the IFE module, which guarantees their semantic consistency. Furthermore, we propose a novel re-weighting strategy to learn a more optimal semantic-similarity graph for the information propagation of the GCN layers. Extensive experiments on two benchmark datasets demonstrate the effectiveness of the proposed method in comparison with some state-of-the-art baselines.

Robust Structured Graph Clustering.

Graph-based clustering methods have achieved remarkable performance by partitioning the data samples into disjoint groups with the similarity graph that characterizes the sample relations. Nevertheless, their learning scheme still suffers from two important problems: 1) the similarity graph directly constructed from the raw features may be unreliable as real-world data always involves adverse noises, outliers, and irrelevant information and 2) most graph-based clustering methods adopt two-step learning strategy that separates the similarity graph construction and clustering into two independent processes. Under such circumstance, the generated graph is unstructured and fixed. It may suffer from a low-quality clustering structure and thus lead to suboptimal clustering performance. To alleviate these limitations, in this article we propose a robust structured graph clustering (RSGC) model. We formulate a novel learning framework to simultaneously learn a robust structured similarity graph and perform clustering. Specifically, the structured graph with proper probabilistic neighborhood assignment is adaptively learned on a robust latent representation that resists the noises and outliers. Furthermore, an explicit rank constraint is imposed on the Laplacian matrix to structurize the graph such that the number of the connected components is exactly equal to the ground-truth cluster number. To solve the challenging objective formulation, we propose to first transform it into an equivalent one that can be tackled more easily. An iterative solution based on the augmented Lagrangian multiplier is then derived to solve the model. In RSGC, the discrete cluster labels can be directly obtained by partitioning the learned similarity graph without reliance on label discretization strategy as most graph-based clustering methods. Experiments on both synthetic and real data sets demonstrate the superiority of the proposed method compared with the state-of-the-art clustering techniques.

A spectral clustering with self-weighted multiple kernel learning method for single-cell RNA-seq data.

Single-cell RNA-sequencing (scRNA-seq) data widely exist in bioinformatics. It is crucial to devise a distance metric for scRNA-seq data. Almost all existing clustering methods based on spectral clustering algorithms work in three separate steps: similarity graph construction; continuous labels learning; discretization of the learned labels by k-means clustering. However, this common practice has potential flaws that may lead to severe information loss and degradation of performance. Furthermore, the performance of a kernel method is largely determined by the selected kernel; a self-weighted multiple kernel learning model can help choose the most suitable kernel for scRNA-seq data. To this end, we propose to automatically learn similarity information from data. We present a new clustering method in the form of a multiple kernel combination that can directly discover groupings in scRNA-seq data. The main proposition is that automatically learned similarity information from scRNA-seq data is used to transform the candidate solution into a new solution that better approximates the discrete one. The proposed model can be efficiently solved by the standard support vector machine (SVM) solvers. Experiments on benchmark scRNA-Seq data validate the superior performance of the proposed model. Spectral clustering with multiple kernels is implemented in Matlab, licensed under Massachusetts Institute of Technology (MIT) and freely available from the Github website, https://github.com/Cuteu/SMSC/.

ADEPT: a domain independent sequence alignment strategy for gpu architectures

BackgroundBioinformatic workflows frequently make use of automated genome assembly and protein clustering tools. At the core of most of these tools, a significant portion of execution time is spent in determining optimal local alignment between two sequences. This task is performed with the Smith-Waterman algorithm, which is a dynamic programming based method. With the advent of modern sequencing technologies and increasing size of both genome and protein databases, a need for faster Smith-Waterman implementations has emerged. Multiple SIMD strategies for the Smith-Waterman algorithm are available for CPUs. However, with the move of HPC facilities towards accelerator based architectures, a need for an efficient GPU accelerated strategy has emerged. Existing GPU based strategies have either been optimized for a specific type of characters (Nucleotides or Amino Acids) or for only a handful of application use-cases.ResultsIn this paper, we present ADEPT, a new sequence alignment strategy for GPU architectures that is domain independent, supporting alignment of sequences from both genomes and proteins. Our proposed strategy uses GPU specific optimizations that do not rely on the nature of sequence. We demonstrate the feasibility of this strategy by implementing the Smith-Waterman algorithm and comparing it to similar CPU strategies as well as the fastest known GPU methods for each domain. ADEPT’s driver enables it to scale across multiple GPUs and allows easy integration into software pipelines which utilize large scale computational systems. We have shown that the ADEPT based Smith-Waterman algorithm demonstrates a peak performance of 360 GCUPS and 497 GCUPs for protein based and DNA based datasets respectively on a single GPU node (8 GPUs) of the Cori Supercomputer. Overall ADEPT shows 10x faster performance in a node-to-node comparison against a corresponding SIMD CPU implementation.ConclusionsADEPT demonstrates a performance that is either comparable or better than existing GPU strategies. We demonstrated the efficacy of ADEPT in supporting existing bionformatics software pipelines by integrating ADEPT in MetaHipMer a high-performance denovo metagenome assembler and PASTIS a high-performance protein similarity graph construction pipeline. Our results show 10% and 30% boost of performance in MetaHipMer and PASTIS respectively.

Clustering with similarity preserving

Graph-based clustering has shown promising performance in many tasks. A key step of graph-based approach is the similarity graph construction. In general, learning graph in kernel space can enhance clustering accuracy due to the incorporation of nonlinearity. However, most existing kernel-based graph learning mechanisms is not similarity-preserving, hence leads to sub-optimal performance. To overcome this drawback, we propose a more discriminative graph learning method which can preserve the pairwise similarities between samples in an adaptive manner for the first time. Specifically, we require the learned graph be close to a kernel matrix, which serves as a measure of similarity in raw data. Moreover, the structure is adaptively tuned so that the number of connected components of the graph is exactly equal to the number of clusters. Finally, our method unifies clustering and graph learning which can directly obtain cluster indicators from the graph itself without performing further clustering step. The effectiveness of this approach is examined on both single and multiple kernel learning scenarios in several datasets.

Discrete Optimal Graph Clustering.

Graph-based clustering is one of the major clustering methods. Most of it works in three separate steps: 1) similarity graph construction; 2) clustering label relaxing; and 3) label discretization with k -means (KM). Such common practice has three disadvantages: 1) the predefined similarity graph is often fixed and may not be optimal for the subsequent clustering; 2) the relaxing process of cluster labels may cause significant information loss; and 3) label discretization may deviate from the real clustering result since KM is sensitive to the initialization of cluster centroids. To tackle these problems, in this paper, we propose an effective discrete optimal graph clustering framework. A structured similarity graph that is theoretically optimal for clustering performance is adaptively learned with a guidance of reasonable rank constraints. Besides, to avoid the information loss, we explicitly enforce a discrete transformation on the intermediate continuous label, which derives a tractable optimization problem with a discrete solution. Furthermore, to compensate for the unreliability of the learned labels and enhance the clustering accuracy, we design an adaptive robust module that learns the prediction function for the unseen data based on the learned discrete cluster labels. Finally, an iterative optimization strategy guaranteed with convergence is developed to directly solve the clustering results. Extensive experiments conducted on both real and synthetic datasets demonstrate the superiority of our proposed methods compared with several state-of-the-art clustering approaches.

Auto-weighted multi-view clustering via kernelized graph learning

Datasets are often collected from different resources or comprised of multiple representations (i.e., views). Multi-view clustering aims to analyze the multi-view data in an unsupervised way. Owing to the efficiency of uncovering the hidden structures of data, graph-based approaches have been investigated widely for various multi-view learning tasks. However, similarity measurement in these methods is challenging since the construction of similarity graph is impacted by several factors such as the scale of data, neighborhood size, choice of similarity metric, noise and outliers. Moreover, nonlinear relationships usually exist in real-world datasets, which have not been considered by most existing methods. In order to address these challenges, a novel model which simultaneously performs multi-view clustering task and learns similarity relationships in kernel spaces is proposed in this paper. The target optimal graph can be directly partitioned into exact c connected components if there are c clusters. Furthermore, our model can assign ideal weight for each view automatically without additional parameters as previous methods do. Since the performance is often sensitive to the input kernel matrix, the proposed model is further extended with multiple kernel learning ability. With the proposed joint model, three subtasks including construct the most accurate similarity graph, automatically allocate optimal weight for each view and find the cluster indicator matrix can be simultaneously accomplished. By this joint learning, each subtask can be mutually enhanced. Experimental results on benchmark datasets demonstrate that our model outperforms other state-of-the-art multi-view clustering algorithms.

When collaborative representation meets subspace projection: A novel supervised framework of graph construction augmented by anti-collaborative representation

Abstract Collaborative representation (CR) offers an appealing way for similarity graph construction in subspace projection (SP) methods. However, the dissimilarities between samples from different classes under the CR framework have been far less investigated. In this paper, we try to integrate the full-length CR with graph embedding based SP, and gradually arrive at the following findings: (1) The methods involving both CR and SP can be formulated as a two-stage framework, where CR servers as a feature extraction step while SP further constructs the similarity graph for feature embedding. This uncovers the essence of this type of methods, as well as their limitations due to the unsupervised property of CR. (2) A novel concept of anti-collaborative representation (Anti-CR), as a counterpart of CR, is introduced to characterize the coding ability of each sample by the samples from different classes. In this way, the label information could be fully exploited to capture the sample-by-sample dissimilarities derived from Anti-CR at the first stage. (3) At the second stage, the above two representations are incorporated into the construction of within-class and between-class graphs, respectively, so that in the projected subspace, the collaborative relationship of the samples in the same class could be strengthened while the collaborative relationship of the samples from different classes would be largely inhibited. Straightforwardly, the above analysis leads to a novel SP method, coined Complete Representation based supervised Feature Extraction and Embedding (CRFEE), as well as its accelerated version (CRFEE-A). Extensive experiments on benchmark image datasets show that the proposed methods outperform several state-of-the-art SP and CR algorithms in terms of discriminant power, indicating the benefits of exploring the anti-collaborative representation for feature description and projection.

Unified Spectral Clustering With Optimal Graph

Spectral clustering has found extensive use in many areas. Most traditional spectral clustering algorithms work in three separate steps: similarity graph construction; continuous labels learning; discretizing the learned labels by k-means clustering. Such common practice has two potential flaws, which may lead to severe information loss and performance degradation. First, predefined similarity graph might not be optimal for subsequent clustering. It is well-accepted that similarity graph highly affects the clustering results. To this end, we propose to automatically learn similarity information from data and simultaneously consider the constraint that the similarity matrix has exact c connected components if there are c clusters. Second, the discrete solution may deviate from the spectral solution since k-means method is well-known as sensitive to the initialization of cluster centers. In this work, we transform the candidate solution into a new one that better approximates the discrete one. Finally, those three subtasks are integrated into a unified framework, with each subtask iteratively boosted by using the results of the others towards an overall optimal solution. It is known that the performance of a kernel method is largely determined by the choice of kernels. To tackle this practical problem of how to select the most suitable kernel for a particular data set, we further extend our model to incorporate multiple kernel learning ability. Extensive experiments demonstrate the superiority of our proposed method as compared to existing clustering approaches.

Sparse Affinity Propagation for Image Analysis

It is challenging to find a small set of data points, so-called exemplars or landmarks, that are nicely representative of itself and other data points. Affinity propagation (AP) is an effective algorithm that identifies exemplars among data points by recursively sending real-valued messages between pairs of data points. AP calculates the message using the similarity among data points. Hence, the construction of similarity graph lies in the heart of the AP algorithm. A common choice for similarity is negative Euclidean distance. However, most data points, especial high-dimensional data, lies into the non-Euclidean space such that Euclidean distance cannot capture the real structure of data. Moreover, Euclidean distance is sensitive to noise and outliers such that the performance of the algorithm will be degraded when data are grossly corrupted. In this paper, we propose an algorithm, named as Sparse Affinity Propagation (SAP), which adopts sparse representation coefficient to depict the relationship among data points. For a given data set, SAP calculates the sparse representation for each data point by solving a convex problem; and then, builds a similarity graph using the representation coefficient; after that, obtains the exemplars by performing AP over the sparse similarity graph. To verify the efficacy of our algorithm, we carried out numerous experiments in the context of data summarization. Empirical studies show that SAP is superior to AP and other baseline algorithms for image analysis in accuracy and robustness.