Deep Clustering Network Research Articles

Self-Supervised Deep Clustering Method for Detecting Abnormal Data of Wastewater Treatment Process

In wastewater treatment process (WWTP), abnormal data seriously reduce data quality rendering the application techniques impractical. The implementation of abnormal data detection is challenging due to the nonlinear nature of WWTP. Typically, constructing an accurate anomaly detector requires large amounts of labeled data, which is difficult in practice. Thus, a self-supervised memory enhanced deep clustering method (SMEL) is proposed to detect abnormal data without using any labels. First, a self-supervised deep clustering network, combining stacked autoencoders and the clustering algorithm, leverages unlabeled data to extract nonlinear features and capture the normal pattern. Second, an adaptive weight objective function, jointly optimizing the reconstruct error and clustering error, is designed to obtain a robust clustering structure. Third, double memory enhanced modules, consisting of a centroid memory and a score memory, are presented to enhance training stability and detection accuracy. Finally, experiments on three WWTP datasets show that SMEL achieves the highest detection accuracy.

Design of intelligent financial data management system based on higher-order hybrid clustering algorithm

Amid the ever-expanding landscape of financial data, the importance of predicting potential risks through artificial intelligence methodologies has steadily risen. To achieve prudent financial data management, this manuscript delves into the domain of intelligent financial risk forecasting within the scope of system design. It presents a data model based on the variational encoder (VAE) enhanced with an attention mechanism meticulously tailored for forecasting a company’s financial peril. The framework called the ATT-VAE embarks on its journey by encoding and enhancing multidimensional data through VAE. It then employs the attention mechanism to enrich the outputs of the VAE network, thereby demonstrating the apex of the model’s clustering capabilities. In the experimentation, we implemented the model to a battery of training tests using diverse public datasets with multimodal features like AWA and CUB and verified with the local finance dataset. The results conspicuously highlight the model’s commendable performance in comparison to publicly available datasets, surpassing numerous deep clustering networks at this juncture. In the realm of financial data, the ATT-VAE model, as presented within this treatise, achieves a clustering accuracy index exceeding 0.7, a feat demonstrably superior to its counterparts in the realm of deep clustering networks. The method outlined herein provides algorithmic foundations and serves as a pivotal reference for the prospective domain of intelligent financial data governance and scrutiny.

Robust scRNA-seq Cell Types Identification by Self-Guided Deep Clustering Network

Robust scRNA-seq Cell Types Identification by Self-Guided Deep Clustering Network

A Multi-Scale Transformer Fusion Deep Clustering Network for Unsupervised Planetary Change Detection

A Multi-Scale Transformer Fusion Deep Clustering Network for Unsupervised Planetary Change Detection

Insider Threat Detection Based on Deep Clustering of Multi-Source Behavioral Events

With the continuous advancement of enterprise digitization, insider threats have become one of the primary cybersecurity concerns for organizations. Therefore, it is of great significance to develop an effective insider threat detection mechanism to ensure the security of enterprises. Most methods rely on artificial feature engineering and input the extracted user behavior features into a clustering-based unsupervised machine learning model for insider threat detection. However, feature extraction is independent of clustering-based unsupervised machine learning. As a result, user behavior features are not the most appropriate for clustering-based unsupervised machine learning, and thus, they reduce the insider threat detection accuracy. This paper proposes an insider threat detection method based on the deep clustering of multi-source behavioral events. On the one hand, the proposed method constructs an end-to-end deep clustering network and automatically learns the user behavior feature expression from multi-source behavioral event sequences. On the other hand, a deep clustering objective function is presented to jointly optimize the learning of feature representations and the clustering task for insider threat detection. This optimization can adjust the optimal user behavior features for the clustering model to improve the insider threat detection accuracy. The experimental results show that the proposed end-to-end insider threat detection model can accurately identify insider threats based on abnormal multi-source user behaviors in enterprise networks.

Unsupervised Continual Learning in Streaming Environments.

A deep clustering network (DCN) is desired for data streams because of its aptitude in extracting natural features thus bypassing the laborious feature engineering step. While automatic construction of deep networks in streaming environments remains an open issue, it is also hindered by the expensive labeling cost of data streams rendering the increasing demand for unsupervised approaches. This article presents an unsupervised approach of DCN construction on the fly via simultaneous deep learning and clustering termed autonomous DCN (ADCN). It combines the feature extraction layer and autonomous fully connected layer in which both network width and depth are self-evolved from data streams based on the bias-variance decomposition of reconstruction loss. The self-clustering mechanism is performed in the deep embedding space of every fully connected layer, while the final output is inferred via the summation of cluster prediction score. Furthermore, a latent-based regularization is incorporated to resolve the catastrophic forgetting issue. A rigorous numerical study has shown that ADCN produces better performance compared with its counterparts while offering fully autonomous construction of ADCN structure in streaming environments in the absence of any labeled samples for model updates. To support the reproducible research initiative, codes, supplementary material, and raw results of ADCN are made available in https://github.com/andriash001/AutonomousDCN.git.

Integrating somatic mutation profiles with structural deep clustering network for metabolic stratification in pancreatic cancer: a comprehensive analysis of prognostic and genomic landscapes.

Pancreatic cancer is a globally recognized highly aggressive malignancy, posing a significant threat to human health and characterized by pronounced heterogeneity. In recent years, researchers have uncovered that the development and progression of cancer are often attributed to the accumulation of somatic mutations within cells. However, cancer somatic mutation data exhibit characteristics such as high dimensionality and sparsity, which pose new challenges in utilizing these data effectively. In this study, we propagated the discrete somatic mutation data of pancreatic cancer through a network propagation model based on protein-protein interaction networks. This resulted in smoothed somatic mutation profile data that incorporate protein network information. Based on this smoothed mutation profile data, we obtained the activity levels of different metabolic pathways in pancreatic cancer patients. Subsequently, using the activity levels of various metabolic pathways in cancer patients, we employed a deep clustering algorithm to establish biologically and clinically relevant metabolic subtypes of pancreatic cancer. Our study holds scientific significance in classifying pancreatic cancer based on somatic mutation data and may provide a crucial theoretical basis for the diagnosis and immunotherapy of pancreatic cancer patients.

Adaptive Deep Clustering Network for Retinal Blood Vessel and Foveal Avascular Zone Segmentation

Optical coherence tomography angiography (OCTA) is a new non-invasive imaging technology that provides detailed visual information on retinal biomarkers, such as the retinal vessel (RV) and the foveal avascular zone (FAZ). Ophthalmologists use these biomarkers to detect various retinal diseases, including diabetic retinopathy (DR) and hypertensive retinopathy (HR). However, only limited study is available on the parallel segmentation of RV and FAZ, due to multi-scale vessel complexity, inhomogeneous image quality, and non-perfusion, leading to erroneous segmentation. In this paper, we proposed a new adaptive segmented deep clustering (ASDC) approach that reduces features and boosts clustering performance by combining a deep encoder–decoder network with K-means clustering. This approach involves segmenting the image into RV and FAZ parts using separate encoder–decoder models and then employing K-means clustering on each part separated by the encoder–decoder models to obtain the final refined segmentation. To deal with the inefficiency of the encoder–decoder network during the down-sampling phase, we used separate encoding and decoding for each task instead of combining them into a single task. In summary, our method can segment RV and FAZ in parallel by reducing computational complexity, obtaining more accurate interpretable results, and providing an adaptive approach for a wide range of OCTA biomarkers. Our approach achieved 96% accuracy and can adapt to other biomarkers, unlike current segmentation methods that rely on complex networks for a single biomarker.

Optimal sensor placement for leak location in water distribution networks based on EGAE clustering algorithm

In the water supply network system, the placement of pressure sensors is crucial to the optimization of the monitoring effect of the system running condition. Most of the existing studies select the placement of sensors according to the hydraulic state of the water distribution network, but do not fully consider and analyze the connectivity between nodes and the topology of the water distribution network. Aiming at the shortcomings of the existing pressure monitoring point layout methods, this paper proposes a pressure sensor layout method based on the Embedding Graph Autoencoder (EGAE) clustering method. Firstly, the EGAE deep clustering algorithm is used to cluster the network nodes based on the topology structure and hydraulic characteristics of the network. According to the clustering results, the network nodes are divided into different regions. Then, by calculating the correlation between nodes in the region, the representative nodes of each region are selected and set as sensor layout nodes. By comparing and analyzing the performance of experiments with other schemes, the results show that the EGAE deep clustering network can effectively integrate the topological structure characteristics and hydraulic characteristics of the water distribution network. The sensor nodes laid out in the scheme proposed in this paper can effectively carry out accurate leakage positioning in subsequent positioning models, and have higher leakage pipe segment identification accuracy and smaller precise positioning errors compared with other schemes.

Autonomous Cross Domain Adaptation Under Extreme Label Scarcity.

A cross domain multistream classification is a challenging problem calling for fast domain adaptations to handle different but related streams in never-ending and rapidly changing environments. Notwithstanding that existing multistream classifiers assume no labeled samples in the target stream, they still incur expensive labeling costs since they require fully labeled samples of the source stream. This article aims to attack the problem of extreme label shortage in the cross domain multistream classification problems where only very few labeled samples of the source stream are provided before process runs. Our solution, namely, Learning Streaming Process from Partial Ground Truth (LEOPARD), is built upon a flexible deep clustering network where its hidden nodes, layers, and clusters are added and removed dynamically with respect to varying data distributions. A deep clustering strategy is underpinned by a simultaneous feature learning and clustering technique leading to clustering-friendly latent spaces. A domain adaptation strategy relies on the adversarial domain adaptation technique where a feature extractor is trained to fool a domain classifier by classifying source and target streams. Our numerical study demonstrates the efficacy of LEOPARD where it delivers improved performances compared to prominent algorithms in 15 of 24 cases. Source codes of LEOPARD are shared in https://github.com/wengweng001/LEOPARD.git to enable further study.

Structural deep clustering network for stratification of breast cancer patients through integration of somatic mutation profiles

Background and ObjectiveBreast cancer is among of the most malignant tumor that occurs in women and is one of the leading causes of death from gynecologic malignancy worldwide. The high degree of heterogeneity that characterizes breast cancer makes it challenging to devise effective therapeutic strategies. Accumulating evidence highlights the crucial role of stratifying breast cancer patients into clinically significant subtypes to achieve better prognoses and treatments. The structural deep clustering network is a graph convolutional network-based clustering algorithm that integrates structural information and has achieved state-of-the-art performance in various applications. MethodsIn this study, we employed structural deep clustering network to integrate somatic mutation profiles for stratifying 2526 breast cancer patients from the Memorial Sloan Kettering Cancer Center into two clinically differentiable subtypes. ResultsBreast cancer patients in cluster 1 exhibited better prognosis than breast cancer patients in cluster 2, and the difference between them was statistically significant. The immunogenomic landscape further demonstrated that cluster 1 was associated with remarkable infiltration of the tumor infiltrating lymphocytes. The clustering subtype could be used to evaluate the therapeutic benefit of immunotherapy and chemotherapy in breast cancer patients. Furthermore, our approach effectively classified patients from eight different cancer types, demonstrating its generalizability. ConclusionsOur study represents a step towards a generic methodology for classifying cancer patients using only somatic mutation data and structural deep clustering network approaches. Employing structural deep clustering network to identify breast cancer subtypes is promising and can inform the development of more accurate and personalized therapies.

Flood Disaster Assessment Method Based on a Stacked Denoising Autoencoder

In recent years, extreme weather has occurred frequently, and the risk of heavy rainfall and flooding faced by the people has risen. It is therefore an urgent requirement to carry out applied research on heavy rainfall and flooding risk assessment. We took Henan Province, where a major flood disaster occurred in 2021, as an example to analyze the impact factors of urban flooding and conduct a risk assessment. Indicators were first selected from population, housing, and the economy, and correlation analysis was used to optimize the indicator system. Then, a deep clustering network model based on a stacked denoising autoencoder (SDAE) was constructed, the feature information implied in the disaster indicators was abstracted into potential features through the coding and decoding of the network, and a small number of potential features were used to express the complex relationship between the disaster indicators. The results of the study show that the high-risk areas of flood damage in Henan Province in 2021 account for 2.3%, the medium-risk areas account for 9.4%, and the low-risk areas account for 80.3%. These evaluation results are in line with the actual situation in Henan Province, and the division of the grade in some areas is more reasonable compared with the entropy weighting method, which is a commonly used method of disaster assessment. The new model does not need to calculate weights to cope with changes in indicators and disaster conditions. The research results can provide scientific reference for urban flood risk management, disaster prevention and mitigation, and regional planning.

A novel deep clustering network using multi-representation autoencoder and adversarial learning for large cross-domain fault diagnosis of rolling bearings

A novel deep clustering network using multi-representation autoencoder and adversarial learning for large cross-domain fault diagnosis of rolling bearings

Marketing Insights from Reviews Using Topic Modeling with BERTopic and Deep Clustering Network

The feedback shared by consumers on e-commerce platforms holds immense value in marketing, as it offers insights into their opinions and preferences, which are readily accessible. However, analyzing a large volume of reviews manually is impractical. Therefore, automating the extraction of essential insights from these data can provide more comprehensive and efficient information. This research focuses on leveraging clustering algorithms to automate the extraction of consumer intentions, related products, and the pros and cons of products from review data. To achieve this, a review dataset was created by performing web crawling on the Naver Shopping platform. The findings are expected to contribute to a more precise understanding of consumer sentiments, enabling marketers to make informed decisions across a wide range of products and services.

Structure-aware deep clustering network based on contrastive learning

Recently, deep clustering has been extensively employed for various data mining tasks, and it can be divided into auto-encoder (AE)-based and graph neural networks (GNN)-based methods. However, existing AE-based methods fall short in effectively extracting structural information, while GNN suffer from smoothing and heterophily. Although methods that combine AE and GNN achieve impressive performance, there remains an inadequate balance between preserving the raw structure and exploring the underlying structure. Accordingly, we propose a novel network named Structure-Aware Deep Clustering network (SADC). Firstly, we compute the cumulative influence of non-adjacent nodes at multiple depths and, thus, enhance the adjacency matrix. Secondly, an enhanced graph auto-encoder is designed. Thirdly, the latent space of AE is endowed with the ability to perceive the raw structure during the learning process. Besides, we design self-supervised mechanisms to achieve co-optimization of node representation learning and topology learning. A new loss function is designed to preserve the inherent structure while also allowing for exploration of latent data structure. Extensive experiments on six benchmark datasets validate that our method outperforms state-of-the-art methods.

Unsupervised deep learning framework for data-driven gating in positron emission tomography.

Physiological motion, such as respiratory motion, has become a limiting factor in the spatial resolution of positron emission tomography (PET) imaging as the resolution of PET detectors continue to improve. Motion-induced misregistration between PET and CT images can also cause attenuation correction artifacts. Respiratory gating can be used to freeze the motion and to reduce motion induced artifacts. In this study, we propose a robust data-driven approach using an unsupervised deep clustering network that employs an autoencoder (AE) to extract latent features for respiratory gating. We first divide list-mode PET data into short-time frames. The short-time frame images are reconstructed without attenuation, scatter, or randoms correction to avoid attenuation mismatch artifacts and to reduce image reconstruction time. The deep AE is then trained using reconstructed short-time frame images to extract latent features for respiratory gating. No additional data are required for the AE training. K-means clustering is subsequently used to perform respiratory gating based on the latent features extracted by the deep AE. The effectiveness of our proposed Deep Clustering method was evaluated using physical phantom and real patient datasets. The performance was compared against phase gating based on an external signal (External) and image based principal component analysis (PCA) with K-means clustering (Image PCA). The proposed method produced gated images with higher contrast and sharper myocardium boundaries than those obtained using the External gating method and Image PCA. Quantitatively, the gated images generated by the proposed Deep Clustering method showed larger center of mass (COM) displacement and higher lesion contrast than those obtained using the other two methods. The effectiveness of our proposed method was validated using physical phantom and real patient data. The results showed our proposed framework could provide superior gating than the conventional External method and Image PCA.

Flight risk evaluation based on flight state deep clustering network

Flight risk evaluation based on data-driven approach is an essential topic of aviation safety management. Existing risk analysis methods ignore the coupling and time-variant characteristics of flight parameters, and cannot accurately establish the mapping relationship between flight state and loss-of-control risk. To deal with the problem, a flight state deep clustering network (FSDCN) model was proposed to mine latent loss-of-control risk information implicating in raw flight parameters. FSDCN integrates the feature extraction and clustering into an end-to-end deep hybrid network to extract latent risk features from multivariate time-series flight parameters and cluster them. In the FSDCN model, a sequential multi-attention encoder–decoder network is designed to extract embedded risk features, and the feature clustering layer is designed to iteratively refine clustering effects and feature extraction. Besides, a loss-of-control classifier is added to optimize the risk feature vector expression and ensure sufficient dividing feature for facilitate clustering. The multi-task joint learning strategy is adopted to improve the clustering performance of the model further. According to extracted risk features and similarity metrics, the optimal clusters number of flight states is set as 5. Comparative experiments show that FSDCN significantly performs better than other clustering models with performance percentage error below 6%. Through statistical analysis of clustering results, the risk level is quantified for each cluster. Three high-difficulty maneuver cases are presented to demonstrate FSDCN for flight risk evaluation. The flight parameter sequences of the maneuver cases are input into the well-trained FSDCN to obtain the risk prediction results. The spatiotemporal distribution characteristics of the risk-quantized results are consistent with flight parameters over-limit situations, which demonstrates the effectiveness of FSDCN on clustering flight states. The experimental results on flight maneuver cases show that FSDCN can find potential loss-of-control risk features according to multivariate time-series flight data and provide support for in-flight risk warnings.

Role of land use in China’s urban energy consumption: based on a deep clustering network and decomposition analysis

Role of land use in China’s urban energy consumption: based on a deep clustering network and decomposition analysis

Incomplete multi-view clustering network via nonlinear manifold embedding and probability-induced loss

Incomplete multi-view clustering, which included missing data in different views, is more challenging than multi-view clustering. For the purpose of eliminating the negative influence of incomplete data, researchers have proposed a series of solutions. However, the present incomplete multi-view clustering methods still confront three major issues: (1) The interference of redundant features hinders these methods to learn the most discriminative features. (2) The importance role of local structure is not considered during clustering. (3) These methods fail to utilize data distribution information to guide models update to decrease the effects of outliers and noise. To address above issues, a novel deep clustering network which exerted on incomplete multi-view data was proposed in this paper. We combine multi-view autoencoders with nonlinear manifold embedding method UMAP to extract latent consistent features of incomplete multi-view data. In the clustering method, we introduce Gaussian Mixture Model (GMM) to fit the complex distribution of data and deal with the interference of outliers. In addition, we reasonably utilize the probability distribution information generated by GMM, using probability-induced loss function to integrate feature learning and clustering as a joint framework. In experiments conducted on multiple benchmark datasets, our method captures incomplete multi-view data features effectively and perform excellent.

Consensus graph learning-based self-supervised ensemble clustering

目的 随着实际应用场景中海量数据采集技术的发展和数据标注成本的不断增加，自监督学习成为海量数据分析的一个重要策略。然而，如何从海量数据中抽取有用的监督信息，并该监督信息下开展有效的学习仍然是制约该方向发展的研究难点。为此，提出了一个基于共识图学习的自监督集成聚类框架。方法 框架主要包括3个功能模块。首先，利用集成学习中多个基学习器构建共识图；其次，利用图神经网络分析共识图，捕获节点优化表示和节点的聚类结构，并从聚类中挑选高置信度的节点子集及对应的类标签生成监督信息；再次，在此标签监督下，联合其他无标注样本更新集成成员基学习器。交替迭代上述功能块，最终提高无监督聚类的性能。结果 为验证该框架的有效性，在标准数据集（包括图像和文本数据）上设计了一系列实验。实验结果表明，所提方法在性能上一致优于现有聚类方法。尤其是在MNIST-Test （modified national institute of standards and technology database）上，本文方法实现了97.78%的准确率，比已有最佳方法高出3.85%。结论 该方法旨在利用图表示学习提升自监督学习中监督信息捕获的能力，监督信息的有效获取进一步强化了集成学习中成员构建的能力，最终提升了无监督海量数据本质结构的挖掘性能。;Objective Clustering is focused on machine learning-related data segmentation for multiple datasets. Its applications are in relevant to such domains like image segmentation and anomaly detection. In addition，to simplify complex tasks optimize its performance，clustering is used in data preprocessing tasks of those are data sub-blocks segmentation， pseudo-labels generation，and abnormal points-removal. Self-supervised learning has become an essential technique for massive data analysis. However，it is challenged to extract effective supervision information and analyze the input data. Method A consensus graph learning based self-supervised ensemble clustering（CGL-SEC）framework is developed. It consists of three main modules：1）to construct the consensus graph based on several ensemble components（i. e. ，the basic clustering methods）. 2）to extract the supervision information by learning the consensus graph representation，and 3）its node clustering results，where the subset of nodes with the high-confidence are selected as labeled samples. To optimize the ensemble components and the corresponding consensus graph，t basic clustering methods are re-trained in related the option of samples-labeled and other samples-unlabeled. The final clustering results can be optimized iteratively until the learning process converges. Result A series of experiments are carried out on benchmarks，including both image and textual datasets. Especially，CGL-SEC is 3. 85% over baseline in terms of clustering evaluation metric on themodified national institute of standards and technology database（MNIST-Test）. First，to optimize data representation and cluster assignment at the same time，deep embedding clustering can be focused on data itself as the supervision information and auto-encoder with the reconstruction loss is pre-trained. The soft cluster assignment of features-embedded is then calculated，and the KL（Kullback-Leibler）divergence is minimized between the soft cluster assignment and the auxiliary target distribution. To improve the performance of the model further，following deep clustering network（DCN）can use hard clustering instead of soft allocation，and local constraints are applied by improved deep embedding clustering（IDEC）. The pseudo-label strategy is implemented as a self-supervised learning method that uses the prediction results of the neural network as the label to simulate the supervision information compared to using data itself as the supervision information. Deepcluster-based K-means clustering is used to generate pseudo-labels to guide the training of convolutional networks. However，the generated pseudo-labels have lower confidence and are prone to trivial solutions in the initial stage of network training. Deep embedding clustering with data augmentation（DEC-DA）and MixMatch-based prediction of data-enhanced samples are used as the supervision information of the original data，which improves the accuracy of the supervision information to a certain extent，but this method is difficult to extend to text and other fields. Deep adaptive clustering-based high-confidence pseudo-label subsets-selected are iteratively trained the network in the prediction results，but lowconfidence samples-involved data distribution information is ignored. Pseudo-semi-supervised clustering votes are used to select a subset of high-confidence pseudo-labels，and all samples are used to train semi-supervised neural network. Although the ensemble strategy can improve the confidence of the pseudo-label，the voting strategy is concerned of category representation only without the feature representation of the sample itself，which can reduce the clustering performance in some cases. The ensemble learning is regarded as a representative machine learning method that reflects the ability of group intelligence，whereas a learning method can improve the overall prediction performance via multiple base learners training and their coordinated prediction results. In pseudo-label-based clustering tasks，it can coordinate multiple base learners to obtain high-confidence pseudo-labels. However，the effectiveness of the supervision information acquisition is still to be resolved. The category information of the sample is considered for current pseudo-label-based ensemble clustering method only when the label is captured and some effective information are ignored like the feature representation of the sample itself and the clustering structure between samples. Conclusion Graph neural network is composed of content information of nodes and the structural information between nodes at the same time. To design a self-supervised ensemble clustering method based on consensus graph representation learning，it is required to make full use of sample features and relationships between samples in ensemble learning. To obtain higher confidence pseudo-labels as supervised information and improve the performance of self-supervised clustering，it is necessary to mine global and local information at the same time. We illustrate a learnable data ensemble representation through graph neural network. The confidence of pseudo-labels is improved，and the entire model is trained in self-supervision iteratively. To be summarized：1）Commonly-used consensus graph learning-integrated clustering framework is developed，which can use multi-level information like clusteringintegrated sample characteristics and category structure. 2）Self-supervision method is proposed，which uses graph neural network to mine the global and local information of the consensus graph，and high-confidence pseudo-labels are obtained as supervised information. 3）Experiments are demonstrated that the consensus graph learning ensemble clustering method has its potentials on image and text datasets.