Discriminant Cluster Research Articles

CSTrans: cross-subdomain transformer for unsupervised domain adaptation

Unsupervised domain adaptation (UDA) aims to make full use of a labeled source domain data to classify an unlabeled target domain data. With the success of Transformer in various vision tasks, existing UDA methods borrow strong Transformer framework to learn global domain-invariant feature representation from the domain level or category level. Of them, the cross-attention as a key component acts for the cross-domain feature alignment, benefiting from its robustness. Intriguingly, we find that the robustness makes the model insensitive to the sub-grouping property within the same category of both source and target domains, known as the subdomain structure. This is because the robustness regards some fine-grained information as the noises and removes them. To overcome this shortcoming, we propose an end-to-end Cross-Subdomain Transformer framework (CSTrans) to exploit the transferability of subdomain structures and the robustness of cross-attention to calibrate inter-domain features. Specifically, there are two innovations in this paper. First, we devise an efficient Index Matching Module (IMM) to calculate the cross-attention of the same category in different domains and learn the domain-invariant representation. This not only simplifies the traditional daunting image-pair selection but also paves the safer way for guarding fine-grained subdomain information. This is because the IMM implements reliable feature confusion. Second, we introduce discriminative clustering to mine the subdomain structures in the same category and further learn subdomain discrimination. Both aspects cooperates with each other for fewer training stages. We perform extensive studies on five benchmarks, and the respective experimental results show that, as compared to existing UDA siblings, CSTrans attains remarkable results with average classification accuracy of 94.3%, 92.1%, and 85.4% on datasets Office-31, ImageCLEF-DA, and Office-Home, respectively.

Deep Incomplete Multi-view Clustering via Multi-level Imputation and Contrastive Alignment

Deep incomplete multi-view clustering (DIMVC) aims to enhance clustering performance by capturing consistent information from incomplete multiple views using deep models. Most existing DIMVC methods typically employ imputation-based strategies to handle missing views before clustering. However, they often assume complete data availability across all views, overlook potential low-quality views, and perform imputation at a single data level, leading to challenges in accurately inferring missing data. To address these issues, we propose a novel imputation-based approach called Multi-level Imputation and Contrastive Alignment (MICA) to simultaneously improve imputation quality and boost clustering performance. Specifically, MICA employs an individual deep model for each view, which unifies view feature learning and cluster assignment prediction. It leverages the learned features from available instances to construct an adaptive cross-view graph for reliable view selection. Guided by these reliable views, MICA performs multi-level (feature-level, data-level, and reconstruction-level) imputation to preserve topological structures across levels and ensure accurate missing feature inference. The complete features are then used for discriminative cluster assignment learning. Additionally, an instance- and cluster-level contrastive alignment is conducted on the cluster assignments to further enhance semantic consistency across views. Experimental results show the effectiveness and superior performance of the proposed MICA method.

Sparse and geometry-aware generalisation of the mutual information for joint discriminative clustering and feature selection

Sparse and geometry-aware generalisation of the mutual information for joint discriminative clustering and feature selection

Progressive Neighbor-masked Contrastive Learning for Fusion-style Deep Multi-view Clustering

Fusion-style Deep Multi-view Clustering (FDMC) can efficiently integrate comprehensive feature information from latent embeddings of multiple views and has drawn much attention recently. However, existing FDMC methods suffer from the interference of view-specific information for fusion representation, affecting the learning of discriminative cluster structure. In this paper, we propose a new framework of Progressive Neighbor-masked Contrastive Learning for FDMC (PNCL-FDMC) to tackle the aforementioned issues. Specifically, by using neighbor-masked contrastive learning, PNCL-FDMC can explicitly maintain the local structure during the embedding aggregation, which is beneficial to the common semantics enhancement on the fusion view. Based on the consistent aggregation, the fusion view is further enhanced by diversity-aware cluster structure enhancement. In this process, the enhanced cluster assignments and cluster discrepancies are employed to guide the weighted neighbor-masked contrastive alignment of semantic structure between individual views and the fusion view. Extensive experiments validate the effectiveness of the proposed framework, revealing its ability in discriminative representation learning and improving clustering performance.

CAPTURE: Comprehensive anti-cancer peptide predictor with a unique amino acid sequence encoder

Anticancer peptides (ACPs) key properties including bioactivity, high efficacy, low toxicity, and lack of drug resistance make them ideal candidates for cancer therapies. To deeply explore the potential of ACPs and accelerate development of cancer therapies, although 53 Artificial Intelligence supported computational predictors have been developed for ACPs and non ACPs classification but only one predictor has been developed for ACPs functional types annotations. Moreover, these predictors extract amino acids distribution patterns to transform peptides sequences into statistical vectors that are further fed to classifiers for discriminating peptides sequences and annotating peptides functional classes. Overall, these predictors remain fail in extracting diverse types of amino acids distribution patterns from peptide sequences. The paper in hand presents a unique CARE encoder that transforms peptides sequences into statistical vectors by extracting 4 different types of distribution patterns including correlation, distribution, composition, and transition. Across public benchmark dataset, proposed encoder potential is explored under two different evaluation settings namely; intrinsic and extrinsic. Extrinsic evaluation indicates that 12 different machine learning classifiers achieve superior performance with the proposed encoder as compared to 55 existing encoders. Furthermore, an intrinsic evaluation reveals that, unlike existing encoders, the proposed encoder generates more discriminative clusters for ACPs and non-ACPs classes. Across 8 public benchmark ACPs and non-ACPs classification datasets, proposed encoder and Adaboost classifier based CAPTURE predictor outperforms existing predictors with an average accuracy, recall and MCC score of 1%, 4%, and 2% respectively. In generalizeability evaluation case study, across 7 benchmark anti-microbial peptides classification datasets, CAPTURE surpasses existing predictors by an average AU-ROC of 2%. CAPTURE predictive pipeline along with label powerset method outperforms state-of-the-art ACPs functional types predictor by 5%, 5%, 5%, 6%, and 3% in terms of average accuracy, subset accuracy, precision, recall, and F1 respectively. CAPTURE web application is available at https://sds_genetic_analysis.opendfki.de/CAPTURE.

Parameter-Agnostic Deep Graph Clustering

Deep graph clustering, efficiently dividing nodes into multiple disjoint clusters in an unsupervised manner, has become a crucial tool for analyzing ubiquitous graph data. Existing methods have acquired impressive clustering effects by optimizing the clustering network under the parametric condition—predefining the true number of clusters ( K tr ). However, K tr is inaccessible in pure unsupervised scenarios, in which existing methods are incapable of inferring the number of clusters ( K ), causing limited feasibility. This article proposes the first Parameter-Agnostic Deep Graph Clustering method (PADGC), which consists of two core modules: K -guidence clustering and topological-hierarchical inference, to infer K efficiently and gain impressive clustering predictions. Specifically, K -guidence clustering is employed to optimize the cluster assignments and discriminative embeddings in a mutual promotion manner under the latest updated K , even though K may deviate from K tr . In turn, such optimized cluster assignments are utilized to explore more accurate K in the topological-hierarchical inference, which can split the dispersive clusters and merge the coupled ones. In this way, these two modules are complementarily optimized until generating the final convergent K and discriminative cluster assignments. Extensive experiments on several benchmarks, including graphs and images, can demonstrate the superiority of our method. The mean values of our inferred K , in 11 out of 12 datasets, deviates from K tr by less than 1. Our method can also achieve competitive clustering effects with existing parametric deep graph clustering.

Deep Multi-Dictionary Learning for Survival Prediction With Multi-Zoom Histopathological Whole Slide Images.

Survival prediction based on histopathological whole slide images (WSIs) is of great significance for risk-benefit assessment and clinical decision. However, complex microenvironments and heterogeneous tissue structures in WSIs bring challenges to learning informative prognosis-related representations. Additionally, previous studies mainly focus on modeling using mono-scale WSIs, which commonly ignore useful subtle differences existed in multi-zoom WSIs. To this end, we propose a deep multi-dictionary learning framework for cancer survival prediction with multi-zoom histopathological WSIs. The framework can recognize and learn discriminative clusters (i.e., microenvironments) based on multi-scale deep representations for survival analysis. Specifically, we learn multi-scale features based on multi-zoom tiles from WSIs via stacked deep autoencoders network followed by grouping different microenvironments by cluster algorithm. Based on multi-scale deep features of clusters, a multi-dictionary learning method with a post-pruning strategy is devised to learn discriminative representations from selected prognosis-related clusters in a task-driven manner. Finally, a survival model (i.e., EN-Cox) is constructed to estimate the risk index of an individual patient. The proposed model is evaluated on three datasets derived from The Cancer Genome Atlas (TCGA), and the experimental results demonstrate that it outperforms several state-of-the-art survival analysis approaches.

Pick-and-Place Transform Learning for Fast Multi-View Clustering.

To manipulate large-scale data, anchor-based multi-view clustering methods have grown in popularity owing to their linear complexity in terms of the number of samples. However, these existing approaches pay less attention to two aspects. 1) They target at learning a shared affinity matrix by using the local information from every single view, yet ignoring the global information from all views, which may weaken the ability to capture complementary information. 2) They do not consider the removal of feature redundancy, which may affect the ability to depict the real sample relationships. To this end, we propose a novel fast multi-view clustering method via pick-and-place transform learning named PPTL, which could capture insightful global features to characterize the sample relationships quickly. Specifically, PPTL first concatenates all the views along the feature direction to produce a global matrix. Considering the redundancy of the global matrix, we design a pick-and-place transform with ℓ2,p-norm regularization to abandon the poor features and consequently construct a compact global representation matrix. Thus, by conducting anchor-based subspace clustering on the compact global representation matrix, PPTL can learn a consensus skinny affinity matrix with a discriminative clustering structure. Numerous experiments performed on small-scale to large-scale datasets demonstrate that our method is not only faster but also achieves superior clustering performance over state-of-the-art methods across a majority of the datasets.

Multiview Clustering via Hypergraph Induced Semi-Supervised Symmetric Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) based multiview technique has been commonly used in multiview data clustering tasks. However, previous NMF based multiview clustering approaches fail to take advantage of a small amount of supervisory information to effectively improve the clustering performance, and are easily affected by the additional post-processing method in clustering tasks. To cope with these issues, a novel framework named multiview clustering via hypergraph induced semi-supervised symmetric NMF (MVCHSS) is proposed in this paper for multiview data clustering applications. Specifically, the proposed method has the following features: 1) a new multiview based hypergraph pairwise constraints propagation (MHPCP) algorithm is developed in MVCHSS to construct a set of informative similarity matrices, revealing the high-order relationships effectively and fully utilizing the limited pairwise constraint supervisory information among samples of each view data; 2) the obtained similarity matrices with much supervisory information are not only enforced into the symmetric NMF (SNMF) model, but also incorporated into the graph regularization for each view data; 3) the optimization problem of MVCHSS is formulated for multiview data clustering tasks to acquire a more discriminative clustering indicator matrix (or called consensus assignment matrix) without additional post-processing method. Moreover, the proof of convergence and the computational complexity for MVCHSS are presented. Extensive experiments on five multiview datasets demonstrate that the proposed MVCHSS framework outperforms several state-of-the-art multiview clustering methods.

Deep discriminative clustering and structural constraint for cross-domain fault diagnosis of rotating machinery

With the rapid development of intelligent manufacturing, fault diagnostic methods based on deep learning have achieved impressive results. However, most methods require plentiful annotated samples and are based on the assumption that data from the source and target domains has the same distribution. These two conditions are difficult to satisfy in practical engineering. In light of these problems, an unsupervised domain adaptation approach named Deep Discriminative Clustering network with Structural Constraint (DDCSC) is proposed in this article. In our method, a Convolutional Neural Network (CNN) module is exploited for learning feature representations of raw data. Then a softmax module is employed to simultaneously predict class probabilities and cluster assignments of the source and target data, respectively. The learnable cluster centroids are introduced into the latent feature space to alleviate the data distribution discrepancy while better capturing the discriminative structure of the target data. In addition, geometric properties of the source data in a feature space are constrained to expand the scope of each category, which facilitates to improve prediction accuracy. An information-theoretic metric is considered as the objective function of discriminative clustering. Diagnostic experiments on a rolling bearing dataset demonstrate that our approach outperforms other popular intelligent approaches and confirms the effectiveness of discriminative clustering.

GLCC: A General Framework for Graph-Level Clustering

This paper studies the problem of graph-level clustering, which is a novel yet challenging task. This problem is critical in a variety of real-world applications such as protein clustering and genome analysis in bioinformatics. Recent years have witnessed the success of deep clustering coupled with graph neural networks (GNNs). However, existing methods focus on clustering among nodes given a single graph, while exploring clustering on multiple graphs is still under-explored. In this paper, we propose a general graph-level clustering framework named Graph-Level Contrastive Clustering (GLCC) given multiple graphs. Specifically, GLCC first constructs an adaptive affinity graph to explore instance- and cluster-level contrastive learning (CL). Instance-level CL leverages graph Laplacian based contrastive loss to learn clustering-friendly representations while cluster-level CL captures discriminative cluster representations incorporating neighbor information of each sample. Moreover, we utilize neighbor-aware pseudo-labels to reward the optimization of representation learning. The two steps can be alternatively trained to collaborate and benefit each other. Experiments on a range of well-known datasets demonstrate the superiority of our proposed GLCC over competitive baselines.

Dual Mutual Information Constraints for Discriminative Clustering

Deep clustering is a fundamental task in machine learning and data mining that aims at learning clustering-oriented feature representations. In previous studies, most of deep clustering methods follow the idea of self-supervised representation learning by maximizing the consistency of all similar instance pairs while ignoring the effect of feature redundancy on clustering performance. In this paper, to address the above issue, we design a dual mutual information constrained clustering method named DMICC which is based on deep contrastive clustering architecture, in which the dual mutual information constraints are particularly employed with solid theoretical guarantees and experimental validations. Specifically, at the feature level, we reduce the redundancy among features by minimizing the mutual information across all the dimensionalities to encourage the neural network to extract more discriminative features. At the instance level, we maximize the mutual information of the similar instance pairs to obtain more unbiased and robust representations. The dual mutual information constraints happen simultaneously and thus complement each other to jointly optimize better features that are suitable for the clustering task. We also prove that our adopted mutual information constraints are superior in feature extraction, and the proposed dual mutual information constraints are clearly bounded and thus solvable. Extensive experiments on five benchmark datasets show that our proposed approach outperforms most other clustering algorithms. The code is available at https://github.com/Li-Hyn/DMICC.

Priori Anchor Labels Supervised Scalable Multi-View Bipartite Graph Clustering

Although multi-view clustering (MVC) has achieved remarkable performance by integrating the complementary information of views, it is inefficient when facing scalable data. Proverbially, anchor strategy can mitigate such a challenge a certain extent. However, the unsupervised dynamic strategy usually cannot obtain the optimal anchors for MVC. The main reasons are that it does not consider the fairness of different views and lacks the priori supervised guidance. To completely solve these problems, we first propose the priori anchor graph regularization (PAGG) for scalable multi-view bipartite graph clustering, dubbed as SMGC method. Specifically, SMGC learns a few representative consensus anchors to simulate the numerous view data well, and constructs a bipartite graph to bridge the affinities between the anchors and original data points. In order to largely improve the quality of anchors, PAGG predefines prior anchor labels to constrain the anchors with discriminative cluster structure and fair view allocation, such that a better bipartite graph can be obtained for fast clustering. Experimentally, abundant of experiments are accomplished on six scalable benchmark datasets, and the experimental results fully demonstrate the effectiveness and efficiency of our SMGC.

Advancing Imbalanced Domain Adaptation: Cluster-Level Discrepancy Minimization With a Comprehensive Benchmark.

Unsupervised domain adaptation methods have been proposed to tackle the problem of covariate shift by minimizing the distribution discrepancy between the feature embeddings of source domain and target domain. However, the standard evaluation protocols assume that the conditional label distributions of the two domains are invariant, which is usually not consistent with the real-world scenarios such as long-tailed distribution of visual categories. In this article, the imbalanced domain adaptation (IDA) is formulated for a more realistic scenario where both label shift and covariate shift occur between the two domains. Theoretically, when label shift exists, aligning the marginal distributions may result in negative transfer. Therefore, a novel cluster-level discrepancy minimization (CDM) is developed. CDM proposes cross-domain similarity learning to learn tight and discriminative clusters, which are utilized for both feature-level and distribution-level discrepancy minimization, palliating the negative effect of label shift during domain transfer. Theoretical justifications further demonstrate that CDM minimizes the target risk in a progressive manner. To corroborate the effectiveness of CDM, we propose two evaluation protocols according to the real-world situation and benchmark existing domain adaptation approaches. Extensive experiments demonstrate that negative transfer does occur due to label shift, while our approach achieves significant improvement on imbalanced datasets, including Office-31, Image-CLEF, and Office-Home.

Towards Uncovering the Intrinsic Data Structures for Unsupervised Domain Adaptation Using Structurally Regularized Deep Clustering.

Unsupervised domain adaptation (UDA) is to learn classification models that make predictions for unlabeled data on a target domain, given labeled data on a source domain whose distribution diverges from the target one. Mainstream UDA methods strive to learn domain-aligned features such that classifiers trained on the source features can be readily applied to the target ones. Although impressive results have been achieved, these methods have a potential risk of damaging the intrinsic data structures of target discrimination, raising an issue of generalization particularly for UDA tasks in an inductive setting. To address this issue, we are motivated by a UDA assumption of structural similarity across domains, and propose to directly uncover the intrinsic target discrimination via constrained clustering, where we constrain the clustering solutions using structural source regularization that hinges on the very same assumption. Technically, we propose a hybrid model of Structurally Regularized Deep Clustering, which integrates the regularized discriminative clustering of target data with a generative one, and we thus term our method as H-SRDC. Our hybrid model is based on a deep clustering framework that minimizes the Kullback-Leibler divergence between the distribution of network prediction and an auxiliary one, where we impose structural regularization by learning domain-shared classifier and cluster centroids. By enriching the structural similarity assumption, we are able to extend H-SRDC for a pixel-level UDA task of semantic segmentation. We conduct extensive experiments on seven UDA benchmarks of image classification and semantic segmentation. With no explicit feature alignment, our proposed H-SRDC outperforms all the existing methods under both the inductive and transductive settings. We make our implementation codes publicly available at https://github.com/huitangtang/H-SRDC.

Exploring Fine-Grained Cluster Structure Knowledge for Unsupervised Domain Adaptation

Unsupervised domain adaptation aims to leverage knowledge from a labeled source domain to learn an accurate model in an unlabeled target domain. However, many previous approaches propose to learn domain agnostic feature representations using a global distribution alignment objective, which does not consider the fine-grained cluster structures in the source and target domains. As such, the goal of this paper is to address two challenging problems: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1) how to thoroughly explore fine-grained cluster structure knowledge in the source and target domains, 2) how to effectively incorporate these structure knowledge for adaptation.</i> Regarding the first point, we are motivated by structural domain similarity assumption and propose structural representation learning, which is achieved by enforcing structural consistency between the source and target domains while retaining their individual discriminative properties. Regarding the second point, we firstly devise a novel structural centroid-based label prediction method, which explicitly models structural representations to form discriminative source and target cluster centroids, and estimates the label distribution of each target sample through the cosine similarity between its corresponding target cluster centroid and all the other source cluster centroids. Then, we adopt clustering learning to incorporate these discriminative structure knowledge for adaptation by minimizing the KL divergence between the predictive target label distribution and an introduced auxiliary one. Comprehensive experiments and analyses on four benchmark datasets demonstrate the superiority of the proposed discriminative clustering framework.

EOCSA: Predicting prognosis of Epithelial ovarian cancer with whole slide histopathological images

Ovarian cancer is one of the most serious cancers that threaten women around the world. Epithelial ovarian cancer (EOC), as the most commonly seen subtype of ovarian cancer, has rather high mortality rate and poor prognosis among various gynecological cancers. Survival analysis outcome is able to provide treatment advices to doctors. In recent years, with the development of medical imaging technology, survival prediction approaches based on pathological images have been proposed. In this study, we designed a deep framework named EOCSA which analyzes the prognosis of EOC patients based on pathological whole slide images (WSIs). Specifically, we first randomly extracted patches from WSIs and grouped them into multiple clusters. Next, we developed a survival prediction model, named DeepConvAttentionSurv (DCAS), which was able to extract patch-level features, removed less discriminative clusters and predicted the EOC survival precisely. Particularly, channel attention, spatial attention, and neuron attention mechanisms were used to improve the performance of feature extraction. Then patient-level features were generated from our weight calculation method and the survival time was finally estimated using LASSO-Cox model. The proposed EOCSA is efficient and effective in predicting prognosis of EOC and the DCAS ensures more informative and discriminative features can be extracted. As far as we know, our work is the first to analyze the survival of EOC based on WSIs and deep neural network technologies. The experimental results demonstrate that our proposed framework has achieved state-of-the-art performance of 0.980 C-index. The implementation of the approach can be found at https://github.com/RanSuLab/EOCprognosis.

New precise model of studentized principal components

If the sample Mahalanobis distance (SMD) is atypically large, it is essential to statistically estimate or test the contribution of each studentized principal component (SPC) decomposed from the SMD to consider its cause. However, there are no appropriate probability models for the SPCs of small samples. This study proposes a precise probability model for the SPCs of small samples without estimating the population eigenvalues or eigenvectors. The proposed model for an SPC comprises an elementary formula of sample size and the SPC’s index multiplied by one random variable following the t-distribution, which is simpler and requires no further computing compared with previous models. Numerical experiments demonstrated that the proposed model performs well under the weak condition that population eigenvalues are closely distinct with various dimensions and sample size. For practical implementation, the proposed model was applied for correcting the SMD to the population Mahalanobis distance, demonstrating better performance than other models. Additionally, the proposed model enables precise statistical testing of SPCs for discriminant analysis, cluster analysis, and projection pursuit, and the model improves the expectation–maximization algorithm for Gaussian mixture models.

Molecular signatures associated with diuron exposure on rat urothelial mitochondria

Diuron, 3-(3,4-dichlorophenyl)-1,1-dimethylurea, is a worldwide used herbicide whose biotransformation gives rise to the metabolites, 3-(3,4-dichlorophenyl)-1-methylurea (DCPMU) and 3,4-dichloroaniline (DCA). Previous studies indicate that diuron and/or its metabolites are toxic to the bladder urothelium of the Wistar rats where, under certain conditions of exposure, they may induce successively urothelial cell degeneration, necrosis, hyperplasia and eventually tumors. The hypothesis was raised that the molecular initiating event (MIE) of this Adverse Outcome Pathway is the mitochondrial toxicity of those compounds. Therefore, this study aimed to investigate in vitro the metabolic alterations resulting from urothelial mitochondria isolated from male Wistar rats exposure to diuron, DCPMU and DCA at 10 and 100 µM. A non-targeted metabolomic analysis using mass spectrometry showed discriminative clustering among groups and alterations in the intensity abundance of membrane-associated molecules phosphatidylcholine, phosphatidylinositol and phosphatidylserine, in addition to methylhexanoyl-CoA and, particularly for diuron 100 µM, dehydro-L-gulonate, all of them involved in critical mitochondrial metabolism. Collectively, these data indicate the mitochondrial dysfunction as an MIE that triggers cellular damage and death observed in previous studies.

Unsupervised domain adaptation via distilled discriminative clustering

Unsupervised domain adaptation addresses the problem of classifying data in an unlabeled target domain, given labeled source domain data that share a common label space but follow a different distribution. Most of the recent methods take the approach of explicitly aligning feature distributions between the two domains. Differently, motivated by the fundamental assumption for domain adaptability, we re-cast the domain adaptation problem as discriminative clustering of target data, given strong privileged information provided by the closely related, labeled source data. Technically, we use clustering objectives based on a robust variant of entropy minimization that adaptively filters target data, a soft Fisher-like criterion, and additionally the cluster ordering via centroid classification. To distill discriminative source information for target clustering, we propose to jointly train the network using parallel, supervised learning objectives over labeled source data. We term our method of distilled discriminative clustering for domain adaptation as DisClusterDA. We also give geometric intuition that illustrates how constituent objectives of DisClusterDA help learn class-wisely pure, compact feature distributions. We conduct careful ablation studies and extensive experiments on five popular benchmark datasets, including a multi-source domain adaptation one. Based on commonly used backbone networks, DisClusterDA outperforms existing methods on these benchmarks. It is also interesting to observe that in our DisClusterDA framework, adding an additional loss term that explicitly learns to align class-level feature distributions across domains does harm to the adaptation performance, though more careful studies in different algorithmic frameworks are to be conducted.