Nonnegative Embedding Research Articles

Towards a unified framework for graph-based multi-view clustering

Recently, clustering data collected from various sources has become a hot topic in real-world applications. The most common methods for multi-view clustering can be divided into several categories: Spectral clustering algorithms, subspace multi-view clustering algorithms, matrix factorization approaches, and kernel methods. Despite the high performance of these methods, they directly fuse all similarity matrices of all views and separate the affinity learning process from the multiview clustering process. The performance of these algorithms can be affected by noisy affinity matrices. To overcome this drawback, this paper presents a novel method called One Step Multi-view Clustering via Consensus Graph Learning and Nonnegative Embedding (OSMGNE). Instead of directly merging the similarity matrices of different views, which may contain noise, a step of learning a consensus similarity matrix is performed. This step forces the similarity matrices of different views to be too similar, which eliminates the problem of noisy data. Moreover, the use of the nonnegative embedding matrix (soft cluster assignment matrix makes it possible to directly obtain the final clustering result without any extra step. The proposed method can solve five subtasks simultaneously. It jointly estimates the similarity matrix of all views, the similarity matrix of each view, the corresponding spectral projection matrix, the unified clustering indicator matrix, and automatically gives the weight of each view without the use of hyper-parameters. In addition, another version of our method is also studied in this paper. This method differs from the first one by using a consensus spectral projection matrix and a consensus Laplacian matrix over all views. An iterative algorithm is proposed to solve the optimization problem of these two methods. The two proposed methods are tested on several real datasets, which prove their superiority.

Recognizing and detecting COVID-19 in chest X-ray images using constrained multi-view spectral clustering

AbstractMachine learning, particularly classification algorithms, has been widely employed for diagnosing COVID-19 cases. However, these methods typically rely on labeled datasets and analyze a single data view. With the vast amount of patient data available without labels, this paper addresses the novel challenge of unsupervised COVID-19 diagnosis. The goal is to harness the abundant data without labels effectively. In recent times, multi-view clustering has garnered considerable attention in the research community. Spectral clustering, known for its robust theoretical framework, is a key focus. However, traditional spectral clustering methods generate only nonlinear data projections, necessitating additional clustering steps. The quality of these post-processing steps can be influenced by various factors, such as initialization procedures and outliers. This paper introduces an enhanced version of the recent “Multiview Spectral Clustering via integrating Nonnegative Embedding and Spectral Embedding” method. While retaining the benefits of the original technique, the proposed model integrates two essential constraints: (1) a constraint for ensuring the consistent smoothness of the nonnegative embedding across all views and (2) an orthogonality constraint imposed on the columns of the nonnegative embedding matrix. The effectiveness of this approach is demonstrated using COVIDx datasets. Additionally, the method is evaluated on other image datasets to validate its suitability for this study.

Consensus graph and spectral representation for one-step multi-view kernel based clustering

Recently, multi-view clustering has received much attention in the fields of machine learning and pattern recognition. Spectral clustering for single and multiple views has been the common solution. Despite its good clustering performance, it has a major limitation: it requires an extra step of clustering. This extra step, which could be the famous k-means clustering, depends heavily on initialization, which may affect the quality of the clustering result. To overcome this problem, a new method called Multi-view Clustering via Consensus Graph Learning and Nonnegative Embedding (MVCGE) is presented in this paper. In the proposed approach, the consensus affinity matrix (graph matrix), consensus representation and cluster index matrix (nonnegative embedding) are learned simultaneously in a unified framework. Our proposed method takes as input the different kernel matrices corresponding to the different views. The proposed learning model integrates two interesting constraints: (i) the cluster indices should be as smooth as possible over the consensus graph and (ii) the cluster indices are set to be as close as possible to the graph convolution of the consensus representation. In this approach, no post-processing such as k-means or spectral rotation is required. Our approach is tested with real and synthetic datasets. The experiments performed show that the proposed method performs well compared to many state-of-the-art approaches.

Incomplete multiview nonnegative representation learning with multiple graphs

Multiview clustering has become an important research topic during the past decade. However, partial views of many data instances are missing in some realistic multiview learning scenarios. To handle this problem, we develop an effective incomplete multiview nonnegative representation learning (IMNRL) framework, which is suitable for incomplete multiview clustering in various situations. The IMNRL framework performs matrix factorization on multiple incomplete graphs and decomposes these incomplete graphs into a consensus nonnegative representation and view-specific spectral representations, which integrates the advantages of multiview nonnegative representation learning and graph learning. The proposed framework has the following merits: (1) it learns a consensus nonnegative embedding and view-specific embeddings simultaneously; (2) the nonnegative embedding satisfies the neighbor constraint on each incomplete view, which directly reveals the multiview clustering results. Experimental results show that the proposed framework outperforms other state-of-the-art incomplete multiview clustering algorithms.

One-step multi-view spectral clustering by learning common and specific nonnegative embeddings

Multi-view spectral clustering is a hot research area which has attracted increasing attention. Most existing multi-view spectral clustering methods utilize a two-step strategy. The first step obtains a common embedding by fusing spectral embeddings of different views, and the second step conducts hard clustering, such as K-means or spectral rotation, on the common embedding. Because the goal of the first step is not obtaining optimal clustering result, and the requirement to post-processing makes the final clustering result uncertain. In this paper, we propose a novel one-step multi-view spectral clustering method, in which the spectral embedding and nonnegative embedding are unified into one framework. Therefore, our method can avoid the uncertainty brought by post-processing and obtain optimal clustering result. Moreover, the nonnegative embedding is divided into two parts. The common nonnegative embedding indicates the shared cluster structure, and the specific nonnegative embedding indicates the exclusive cluster structure of each view. Hence, our method can well tackle with noises and outliers of different views. Furthermore, an alternating iterative algorithm is used to solve the joint optimization problem. Extensive experimental results on four real-world datasets have demonstrated the effectiveness of the proposed method.

One-step multi-view subspace clustering with incomplete views

Multi-view clustering has gained soar attention in the field of data mining in recent years. Most multi-view clustering methods are considered to be complete for each view. However, in many applications, some views may contain missing instances, resulting in the incomplete multi-view learning problem. Incomplete views cannot be directly dealt with traditional multi-view clustering methods. In this paper, we propose a One-step multi-view subspace clustering with incomplete views (OMVSC-IV) method based on low-rank matrix factorization to overcome this problem. Specifically, the proposed algorithm uses low-rank matrix factorization to learn a consensus representation matrix, and then combined with the objective function of non-negative embedding and spectral embedding subspace clustering proposed in this paper. The whole process is established into the same objective function for joint optimization and requires no post-processing (e.g., K-means), thus avoiding the defect of being sensitive to initial values. Furthermore, the accuracy, parameter, sensitivity and convergence of the algorithm are studied by experiments. Experimental results on nine data sets show that the proposed algorithm performs superior to some of the latest algorithms.

Joint Representation Learning and Clustering: A Framework for Grouping Partial Multiview Data

Partial multi-view clustering has attracted various attentions from diverse fields. Most existing methods adopt separate steps to obtain unified representations and extract clustering indicators. This separate manner prevents two learning processes to negotiate to achieve optimal performance. In this paper, we propose the Joint Representation Learning and Clustering (JRLC) framework to address this issue. The JRLC framework employs representation matrices to extract view-specific clustering information directly from the presence of partial similarity matrices, and rotates them to learn a common probability label matrix simultaneously, which connects representation learning and clustering seamlessly to achieve better clustering performance. Under the guidance of JRLC framework, several new incomplete multi-view clustering methods can be developed by extending existing single-view graph-based representation learning methods. For illustration, within the framework, we propose two specific methods, JRLC with spectral embedding (JRLC-SE) and JRLC via integrating nonnegative embedding and spectral embedding (JRLC-NS). Two iterative algorithms with guaranteed convergence are designed to solve the resultant optimization problems of JRLC-SE and JRLC-NS. Experimental results on various datasets and news topic clustering application demonstrate the effectiveness of the proposed algorithms.

Multi-view spectral clustering via integrating nonnegative embedding and spectral embedding

The application of most existing multi-view spectral clustering methods is generally limited by the following three deficiencies. First, the requirement to post-processing, such as K-means or spectral rotation. Second, the susceptibility to parameter selection. Third, the high computation cost. To this end, in this paper we develop a novel method that integrates nonnegative embedding and spectral embedding into a unified framework. Two promising advantages of proposed method include 1) the learned nonnegative embedding directly reveals the consistent clustering result, such that the uncertainty brought by post-processing can be avoided; 2) the involved model is parameter-free, which makes our method more applicable than existing algorithms that introduce many additional parameters. Furthermore, we develop an efficient inexact Majorization-Minimization method to solve the involved model which is non-convex and non-smooth. Experiments on multiple benchmark datasets demonstrate that our method achieves state-of-the-art performance.

Evaluation of Wu etal.: Comprehending Global and Local Structure of Single-Cell Datasets.

One snapshot of the peer review process for "Visualizing and interpreting Single-Cell Gene Expression Datasets with Similarity Weighted Nonnegative Embedding" (Wu etal., 2018).

Visualizing and Interpreting Single-Cell Gene Expression Datasets with Similarity Weighted Nonnegative Embedding.

High-throughput single-cell gene expression profiling has enabled the definition of new cell types and developmental trajectories. Visualizing these datasets is crucial to biological interpretation, and a popular method is t-stochastic neighbor embedding (t-SNE), which visualizes local patterns well but distorts global structure, such as distances between clusters. We developed similarity weighted nonnegative embedding (SWNE), which enhances interpretation of datasets by embedding the genes and factors that separate cell states on the visualization alongside the cells and maintains fidelity when visualizing local and global structure for both developmental trajectories and discrete cell types. SWNE uses nonnegative matrix factorization to decompose the gene expression matrix into biologically relevant factors; embeds the cells, genes, and factors in a 2D visualization; and uses a similarity matrix to smooth the embeddings. We demonstrate SWNE on single-cell RNA-seq data from hematopoietic progenitors and human brain cells. SWNE is available as an R package at github.com/yanwu2014/swne.

Non-negative embedding for fully unsupervised domain adaptation

Domain adaptation is a field of machine learning that addresses the problem occurring when a classifier is trained and tested on domains from different distributions. This kind of paradigm is of vital importance as it allows a learner to generalize the knowledge across different tasks. In this paper, we present a new method for fully unsupervised domain adaptation that seeks to align two domains using a shared set of basis vectors derived from eigenvectors of each domain. We use non-negative matrix factorization (NMF) techniques to generate a non-negative embedding that minimizes the distance between projections of source and target data. We present a theoretical justification for our approach by showing the consistency of the similarity function defined using the obtained embedding. We also prove a theorem that relates the source and target domain errors using kernel embeddings of distribution functions. We validate our approach on benchmark data sets and show that it outperforms several state-of-art domain adaptation methods.

Effective multiplicative updates for non-negative discriminative learning in multimodal dimensionality reduction

Fisher discriminant analysis gives the unsatisfactory results if points in the same class have within-class multimodality and fails to produce the non-negativity of projection vectors. In this paper, we focus on the newly formulated within and between-class scatters based supervised locality preserving dimensionality reduction problem and propose an effective dimensionality reduction algorithm, namely, Multiplicative Updates based non-negative Discriminative Learning (MUNDL), which optimally seeks to obtain two non-negative embedding transformations with high preservation and discrimination powers for two data sets in different classes such that nearby sample pairs in the original space compact in the learned embedding space, under which the projections of the original data in different classes can be appropriately separated from each other. We also show that MUNDL can be easily extended to nonlinear dimensionality reduction scenarios by employing the standard kernel trick. We verify the feasibility and effectiveness of MUNDL by conducting extensive data visualization and classification experiments. Numerical results on some benchmark UCI and real-world datasets show the MUNDL method tends to capture the intrinsic local and multimodal structure characteristics of the given data and outperforms some established dimensionality reduction methods, while being much more efficient.