Self-supervised Clustering Research Articles

Integrating single-cell multi-omics data through self-supervised clustering

Advances in single-cell sequencing techniques enable individual cells to be sequenced across multiple modalities simultaneously, such as transcriptomics, epigenomics, and proteomics. Integrating multi-omics single-cell data provides a deeper and more comprehensive vision of genomic mechanisms. Due to the huge distribution shift between modalities, current integration methods mostly align the modality through domain adaption or similar strategies. They achieved limited performance likely because the modalities are over-divergent. Here, we propose a novel single-cell multimodal fusion method, scFPN, to improve the learned embedding by a clustering strategy. Specifically, scFPN first embeds each modality data through a feature pyramid network with a modality-specific variational auto-encoder. The learned hierarchical embeddings are then fused and input into a dual self-supervision optimizing module for attracting similar cells and separating dissimilar cells. We conducted comprehensive experiments on recently produced six datasets from different sequencing platforms and demonstrated the superiority of scFPN over a variety of state-of-the-art methods. More importantly, scFPN showed bio-interpretability by marker enrichment analysis from de-noising and imputing raw profiles.

Self-supervised based clustering for retinal optical coherence tomography images.

In response to the inadequacy of manual analysis in meeting the rising demand for retinal optical coherence tomography (OCT) images, a self-supervised learning-based clustering model was implemented. A public dataset was utilized, with 83,484 OCT images with categories of choroidal neovascularization (CNV), diabetic macular edema (DME), drusen, and normal fundus. This study employed the Semantic Pseudo Labeling for Image Clustering (SPICE) framework, a self-supervised learning-based method, to cluster unlabeled OCT images into binary and four categories, and the performances were compared with baseline models. We also analysed feature distribution using t-SNE, and explored the cluster centers, attention maps, and misclassified images. In addition, DME and CNV subsets were clustered binarily, and the results were interpreted by two retinal specialists. SPICE demonstrated superior performance in binary and four categories classification tasks, achieving the accuracy of 0.886 and 0.846, respectively. In t-SNE analysis, the four types exhibited significant clustering into distinct groups. The cluster centers corresponded to the human labels, and the heat map revealed that the model focused on important biomarkers. The misclassified images exposed similar features to the inaccurate classes. The model also grouped DME and CNV into two distinct categories respectively. Self-supervised clustering effectively distinguished disease variances and revealed common features, with a notable capability to detect disease heterogeneity through biomarkers.

Dual Information Enhanced Multiview Attributed Graph Clustering.

Multiview attributed graph clustering is an important approach to partition multiview data based on the attribute characteristics and adjacent matrices from different views. Some attempts have been made in using graph neural network (GNN), which have achieved promising clustering performance. Despite this, few of them pay attention to the inherent specific information embedded in multiple views. Meanwhile, they are incapable of recovering the latent high-level representation from the low-level ones, greatly limiting the downstream clustering performance. To fill these gaps, a novel dual information enhanced multiview attributed graph clustering (DIAGC) method is proposed in this article. Specifically, the proposed method introduces the specific information reconstruction (SIR) module to disentangle the explorations of the consensus and specific information from multiple views, which enables graph convolutional network (GCN) to capture the more essential low-level representations. Besides, the contrastive learning (CL) module maximizes the agreement between the latent high-level representation and low-level ones and enables the high-level representation to satisfy the desired clustering structure with the help of the self-supervised clustering (SC) module. Extensive experiments on several real-world benchmarks demonstrate the effectiveness of the proposed DIAGC method compared with the state-of-the-art baselines.

Multi-task hierarchical convolutional network for visual-semantic cross-modal retrieval

Bridging visual and textual representations plays a central role in delving into multimedia data understanding. The main challenge arises from that images and texts exist in heterogeneous spaces, leading to the difficulty to preserve the semantic consistency between both modalities. To narrow the modality gap, most recent methods resort to extra object detectors or parsers to obtain the hierarchical representations. In this work, we address this problem by introducing our Multi-Task Hierarchical Convolutional Neural Network (MT-HCN). It is characterized by mining the hierarchical semantic information without the aid of any extra supervisions. Firstly, from the perspective of representing architecture, we leverage the intrinsic hierarchical structure of Convolutional Neural Networks (CNNs) to decompose the representations of both modalities into two semantically complementary levels, i.e., exterior representations and concept representations. The former focuses on discovering the fine-grained low-level associations between both modalities, meanwhile the latter underlines capturing more high-level abstract semantics. Specifically, we present a Self-Supervised Clustering (SSC) loss to preserve more fine-grained semantic clues in exterior representations. It is constituted on the basis of viewing multiple image/text pairs with similar exterior as a category. In addition, a novel harmonious bidirectional triplet ranking (HBTR) loss is proposed, which mitigate the adverse effects brought about by the biased and noisy negative samples. Besides hardest negatives, it also imposes the constraints on the distance between the positive pairs and the centroid of negative pairs. Extensive experimental results on two popular cross-modal retrieval benchmarks demonstrate our proposed MT-HCN can achieve the competitive results compared with the state-of-the-art methods.

Trustworthy multi-view clustering via alternating generative adversarial representation learning and fusion

Multi-view clustering (MVC) has attached extensive attention as it provides an effective approach to deal with the unlabeled data in real-world applications. To enjoy the strong feature extraction capacity of deep learning, traditional shallow MVC methods are further extended to deep version. Though achieve superiorities in many fields, most existing deep MVC models are still limited by their lacking assurance of the trustworthiness and robustness. This drawback also leads to the possible poor clustering performance. To tackle this drawback, we propose an alternating generative adversarial learning strategy based network, termed as Alternating Generative Adversarial Representation Learning (AGARL) network for multi-view clustering, aiming to drive different views to fall into the same semantic space for the further alignment and information fusion. Various from the traditional generative adversarial learning strategy, the proposed alternating adversarial representation learning strategy successes in further aligning the distributions of the view-specific representations and exploring the cross-view consistency credited by its setting each view as the reference alternately. Then a consensus representative latent representation can be learned through an attention-like sub-network for all views. Finally, a self-supervised clustering module is employed in the proposed network to guide the learning of cluster assignment. In addition, comprehensive experiments on numerous benchmark MVC datasets have been performed to demonstrate the effectiveness of the proposed network compared to other state-of-the-art methods.

CLOINet: ocean state reconstructions through remote-sensing, in-situ sparse observations and deep learning

Combining remote-sensing data with in-situ observations to achieve a comprehensive 3D reconstruction of the ocean state presents significant challenges for traditional interpolation techniques. To address this, we developed the CLuster Optimal Interpolation Neural Network (CLOINet), which combines the robust mathematical framework of the Optimal Interpolation (OI) scheme with a self-supervised clustering approach. CLOINet efficiently segments remote sensing images into clusters to reveal non-local correlations, thereby enhancing fine-scale oceanic reconstructions. We trained our network using outputs from an Ocean General Circulation Model (OGCM), which also facilitated various testing scenarios. Our Observing System Simulation Experiments aimed to reconstruct deep salinity fields using Sea Surface Temperature (SST) or Sea Surface Height (SSH), alongside sparse in-situ salinity observations. The results showcased a significant reduction in reconstruction error up to 40% and the ability to resolve scales 50% smaller compared to baseline OI techniques. Remarkably, even though CLOINet was trained exclusively on simulated data, it accurately reconstructed an unseen SST field using only glider temperature observations and satellite chlorophyll concentration data. This demonstrates how deep learning networks like CLOINet can potentially lead the integration of modeling and observational efforts in developing an ocean digital twin.

Combining core points and cluster-level semantic similarity for self-supervised clustering

Combining core points and cluster-level semantic similarity for self-supervised clustering

ScAMAC: self-supervised clustering of scRNA-seq data based on adaptive multi-scale autoencoder.

Cluster assignment is vital to analyzing single-cell RNA sequencing (scRNA-seq) data to understand high-level biological processes. Deep learning-based clustering methods have recently been widely used in scRNA-seq data analysis. However, existing deep models often overlook the interconnections and interactions among network layers, leading to the loss of structural information within the network layers. Herein, we develop a new self-supervised clustering method based on an adaptive multi-scale autoencoder, called scAMAC. The self-supervised clustering network utilizes the Multi-Scale Attention mechanism to fuse the feature information from the encoder, hidden and decoder layers of the multi-scale autoencoder, which enables the exploration of cellular correlations within the same scale and captures deep features across different scales. The self-supervised clustering network calculates the membership matrix using the fused latent features and optimizes the clustering network based on the membership matrix. scAMAC employs an adaptive feedback mechanism to supervise the parameter updates of the multi-scale autoencoder, obtaining a more effective representation of cell features. scAMAC not only enables cell clustering but also performs data reconstruction through the decoding layer. Through extensive experiments, we demonstrate that scAMAC is superior to several advanced clustering and imputation methods in both data clustering and reconstruction. In addition, scAMAC is beneficial for downstream analysis, such as cell trajectory inference. Our scAMAC model codes are freely available at https://github.com/yancy2024/scAMAC.

Soft-orthogonal constrained dual-stream encoder with self-supervised clustering network for brain functional connectivity data

In many brain network studies, brain functional connectivity data is extracted from neuroimaging data and then used for disease prediction. For now, brain disease data not only has a small sample but also has the problem of high dimensional and nonlinear. Therefore, deep clustering on brain functional connectivity data is very challenging. To solve these problems, we propose a Soft-orthogonal Constrained Dual-stream Encoder with Self-supervised clustering network (SSCDE), which consists of a pretext task and downstream task, which can fully mine the effective information in brain disease data. In the pretext task, we use two brain disease data under the same category to do cross-domain learning to obtain effective information from the same dataset. In the downstream task, to reduce redundancy and avoid negative coding, we propose a soft-orthogonal constrained dual-stream encoder to encode features separately. At the same time, we use the pseudo labels given by the pretext task as prior information for self-supervised learning. We conduct validation on different brain disease recognition tasks, and the result have proved that the proposed framework has achieved good performance compared with the unsupervised clustering analysis algorithms. To our knowledge, this is the first cross-domain assisted recognition study on brain functional connectivity data. The code is available at https://github.com/hulu88/SSCDE.

Contrastive Self-Supervised Clustering for Specific Emitter Identification

Specific Emitter Identification (SEI) is crucial for attacking and defending Internet of Things (IoT) devices in untrusted scenarios or battlefield environments. However, existing SEI methods usually require annotation information, which is often unavailable in non-cooperative communications and untrusted scenarios. In this paper, we propose a signal contrastive self-supervised clustering (SCSC) method for unsupervised SEI applications. First, we propose SCSC with 1D fingerprint pyramid feature extractor (1D-FPFE) for obtaining hierarchical subtle features of emitter signals. Then, we propose a bit-pulse selection (BPS) strategy and several signal data augmentation methods. By constructing signal positive and negative instance pairs through data augmentation, our approach generates cluster preference representations in a contrastive self-supervised learning manner. Extensive experimental results based on communication burst emitter dataset show that SCSC achieves an accuracy improvement of about 26% over the current best communication signal clustering algorithm. Moreover, SCSC also exhibits good performance and generalization for 30 emitter clustering and few-shot unlabeled signal clustering.

Self-Supervised Clustering Models Based on BYOL Network Structure

Contrastive-based clustering models usually rely on a large number of negative pairs to capture uniform representations, which requires a large batch size and high computational complexity. In contrast, some self-supervised methods perform non-contrastive learning to capture discriminative representations only with positive pairs, but suffer from the collapse of clustering. To solve these issues, a novel end-to-end self-supervised clustering model is proposed in this paper. The basic self-supervised learning network is first modified, followed by the incorporation of a Softmax layer to obtain cluster assignments as data representation. Then, adversarial learning on the cluster assignments is integrated into the methods to further enhance discrimination across different clusters and mitigate the collapse between clusters. To further encourage clustering-oriented guidance, a new cluster-level discrimination is assembled to promote clustering performance by measuring the self-correlation between the learned cluster assignments. Experimental results on real-world datasets exhibit better performance of the proposed model compared with the existing deep clustering methods.

Self-supervised clustering analysis of colorectal cancer biomarkers based on multi-scale whole slides image and mass spectrometry imaging fused images

Mass spectrometry imaging (MSI) is widely used for unlabeled molecular co-localization in biological samples and is also commonly used for screening cancer biomarkers. The main issues affecting the screening of cancer biomarkers are: 1) low-resolution MSI and pathological slices cannot be accurately matched; 2) a large amount of MSI data cannot be directly analyzed without manual annotation. This paper proposes a self-supervised cluster analysis method for colorectal cancer biomarkers based on multi-scale whole slide images (WSI) and MSI fusion images without manual annotation, which can accurately determine the correlation between molecules and lesion areas. This paper uses the combination of WSI multi-scale high-resolution and MSI high-dimensional data to obtain high-resolution fusion images. This method can observe the spatial distribution of molecules in pathological slices and use this method as an evaluation index for self-supervised screening of cancer biomarkers. The experimental results show that the method proposed in this chapter can train the image fusion model with a small amount of MSI and WSI data, and the mean Pixel Accuracy (mPA) and mean Intersection over Union (mIoU) evaluation metrics of the fused images can reach 0.9587 and 0.8745. And self-supervised clustering using MSI features and fused image features can obtain good classification results, and the precision, recall, and F1-score values of the self-supervised model reach 0.9074, 0.9065, and 0.9069, respectively. This method effectively combines the advantages of WSI and MSI, which will significantly expand the application scenarios of MSI and facilitate the screening of disease markers.

Deep Latent Space Clustering for Detection of Stealthy False Data Injection Attacks Against AC State Estimation in Power Systems

This paper proposes a self-supervised latent space clustering algorithm, called the Deep Latent Space Clustering, for the detection of stealthy false data injection attacks (FDIAs) in smart grids against state estimation algorithms. The stealthy FDI model considered is based on the accurate AC state estimation and is able to bypass conventional bad data detection (BDD) algorithms with ease. The key element of the detection model is a stacked autoencoder network that first undergoes a carefully designed two-step finetuning process, following which a trainable clustering head is stacked on top of the finetuned encoder and the final network is further trained to achieve a clean clustering of the data into benign and compromised samples without labelled supervision. To test the efficacy and scalability of the detection model, it is tested on the standard IEEE 14 bus, 118 bus and 300 bus test systems. The self-supervised clustering model is compared to several supervised, semi-supervised and unsupervised algorithms proposed in the literature for detection of FDI on the aforementioned test cases and has been found to perform at par with the state of the art among them.

A Sorting Method of SAR Emitter Signal Sorting Based on Self-Supervised Clustering

Most existing methods for sorting synthetic aperture radar (SAR) emitter signals rely on either unsupervised clustering or supervised classification methods. However, unsupervised clustering can consume a significant amount of computational and storage space and is sensitive to the setting of hyperparameters, while supervised classification requires a considerable number of labeled samples. To address these limitations, we propose a self-supervised clustering-based method for sorting SAR radiation source signals. The method uses a constructed affinity propagation-convolutional neural network (AP-CNN) to perform self-supervised clustering of a large number of unlabeled signal time-frequency images into multiple clusters in the first stage. Subsequently, it uses a self-organizing map (SOM) network combined with inter-pulse parameters for further sorting in the second stage. The simulation results demonstrate that the proposed method outperforms other depth models and conventional methods in the environment where Gaussian white noise affects the signal. The experiments conducted using measured data also show the superiority of the proposed method in this paper.

Self-supervised clustering with assistance from off-the-shelf classifier

Deep clustering outperforms conventional clustering by mutually promoting representation learning and cluster assignment. However, most existing deep clustering methods suffer from two major drawbacks. Firstly, most cluster assignment methods are highly dependent on the intermediate target distribution generated by a handcrafted nonlinear mapping function. Secondly, the clustering results can be easily guided towards wrong direction by the misassigned samples in each cluster. The existing deep clustering methods are incapable of discriminating such samples. These facts largely limit the possible performance that deep clustering methods can reach. To address these issues, a novel Self-Supervised Clustering (SSC) framework is constructed, which boosts the clustering performance by classification in an unsupervised manner. Fuzzy theory is used to score the membership of each sample to the clusters in terms of probability in each training epoch, which evaluates the intermediate clustering result certainty of each sample. The most reliable samples can be selected with the help of a sample selection method according to the membership and enhanced by data augmentation method. These augmented data are employed to fine-tune an off-the-shelf deep network classifier with the labels provided by the clustering in a self-supervised way. The classification results of the original dataset are used as the target distribution to guide the training process of the deep clustering model. The proposed framework can efficiently discriminate sample outliers and generate better target distribution with the assistance of the powerful classifier. Extensive experiments indicate that the proposed framework remarkably outperforms state-of-the-art deep clustering methods on four benchmark datasets.

Consensus graph learning-based self-supervised ensemble clustering

目的 随着实际应用场景中海量数据采集技术的发展和数据标注成本的不断增加，自监督学习成为海量数据分析的一个重要策略。然而，如何从海量数据中抽取有用的监督信息，并该监督信息下开展有效的学习仍然是制约该方向发展的研究难点。为此，提出了一个基于共识图学习的自监督集成聚类框架。方法 框架主要包括3个功能模块。首先，利用集成学习中多个基学习器构建共识图；其次，利用图神经网络分析共识图，捕获节点优化表示和节点的聚类结构，并从聚类中挑选高置信度的节点子集及对应的类标签生成监督信息；再次，在此标签监督下，联合其他无标注样本更新集成成员基学习器。交替迭代上述功能块，最终提高无监督聚类的性能。结果 为验证该框架的有效性，在标准数据集（包括图像和文本数据）上设计了一系列实验。实验结果表明，所提方法在性能上一致优于现有聚类方法。尤其是在MNIST-Test （modified national institute of standards and technology database）上，本文方法实现了97.78%的准确率，比已有最佳方法高出3.85%。结论 该方法旨在利用图表示学习提升自监督学习中监督信息捕获的能力，监督信息的有效获取进一步强化了集成学习中成员构建的能力，最终提升了无监督海量数据本质结构的挖掘性能。;Objective Clustering is focused on machine learning-related data segmentation for multiple datasets. Its applications are in relevant to such domains like image segmentation and anomaly detection. In addition，to simplify complex tasks optimize its performance，clustering is used in data preprocessing tasks of those are data sub-blocks segmentation， pseudo-labels generation，and abnormal points-removal. Self-supervised learning has become an essential technique for massive data analysis. However，it is challenged to extract effective supervision information and analyze the input data. Method A consensus graph learning based self-supervised ensemble clustering（CGL-SEC）framework is developed. It consists of three main modules：1）to construct the consensus graph based on several ensemble components（i. e. ，the basic clustering methods）. 2）to extract the supervision information by learning the consensus graph representation，and 3）its node clustering results，where the subset of nodes with the high-confidence are selected as labeled samples. To optimize the ensemble components and the corresponding consensus graph，t basic clustering methods are re-trained in related the option of samples-labeled and other samples-unlabeled. The final clustering results can be optimized iteratively until the learning process converges. Result A series of experiments are carried out on benchmarks，including both image and textual datasets. Especially，CGL-SEC is 3. 85% over baseline in terms of clustering evaluation metric on themodified national institute of standards and technology database（MNIST-Test）. First，to optimize data representation and cluster assignment at the same time，deep embedding clustering can be focused on data itself as the supervision information and auto-encoder with the reconstruction loss is pre-trained. The soft cluster assignment of features-embedded is then calculated，and the KL（Kullback-Leibler）divergence is minimized between the soft cluster assignment and the auxiliary target distribution. To improve the performance of the model further，following deep clustering network（DCN）can use hard clustering instead of soft allocation，and local constraints are applied by improved deep embedding clustering（IDEC）. The pseudo-label strategy is implemented as a self-supervised learning method that uses the prediction results of the neural network as the label to simulate the supervision information compared to using data itself as the supervision information. Deepcluster-based K-means clustering is used to generate pseudo-labels to guide the training of convolutional networks. However，the generated pseudo-labels have lower confidence and are prone to trivial solutions in the initial stage of network training. Deep embedding clustering with data augmentation（DEC-DA）and MixMatch-based prediction of data-enhanced samples are used as the supervision information of the original data，which improves the accuracy of the supervision information to a certain extent，but this method is difficult to extend to text and other fields. Deep adaptive clustering-based high-confidence pseudo-label subsets-selected are iteratively trained the network in the prediction results，but lowconfidence samples-involved data distribution information is ignored. Pseudo-semi-supervised clustering votes are used to select a subset of high-confidence pseudo-labels，and all samples are used to train semi-supervised neural network. Although the ensemble strategy can improve the confidence of the pseudo-label，the voting strategy is concerned of category representation only without the feature representation of the sample itself，which can reduce the clustering performance in some cases. The ensemble learning is regarded as a representative machine learning method that reflects the ability of group intelligence，whereas a learning method can improve the overall prediction performance via multiple base learners training and their coordinated prediction results. In pseudo-label-based clustering tasks，it can coordinate multiple base learners to obtain high-confidence pseudo-labels. However，the effectiveness of the supervision information acquisition is still to be resolved. The category information of the sample is considered for current pseudo-label-based ensemble clustering method only when the label is captured and some effective information are ignored like the feature representation of the sample itself and the clustering structure between samples. Conclusion Graph neural network is composed of content information of nodes and the structural information between nodes at the same time. To design a self-supervised ensemble clustering method based on consensus graph representation learning，it is required to make full use of sample features and relationships between samples in ensemble learning. To obtain higher confidence pseudo-labels as supervised information and improve the performance of self-supervised clustering，it is necessary to mine global and local information at the same time. We illustrate a learnable data ensemble representation through graph neural network. The confidence of pseudo-labels is improved，and the entire model is trained in self-supervision iteratively. To be summarized：1）Commonly-used consensus graph learning-integrated clustering framework is developed，which can use multi-level information like clusteringintegrated sample characteristics and category structure. 2）Self-supervision method is proposed，which uses graph neural network to mine the global and local information of the consensus graph，and high-confidence pseudo-labels are obtained as supervised information. 3）Experiments are demonstrated that the consensus graph learning ensemble clustering method has its potentials on image and text datasets.

Teacher–Student Mutual Learning for efficient source-free unsupervised domain adaptation

Unsupervised domain adaptation (UDA) aims to alleviate domain shifts by transferring relevant domain information from a fully labeled source domain to an unknown target domain. Although significant progress has been made recently, the use of UDA in real-world applications is still limited owing to low-resource computers and privacy issues. To further extend the flexibility of the target model, this study attempts to address a new setting of UDA, where only a source model is provided, and target models with various network architectures can be learned according to the deployment environment. Therefore, we propose a novel two-stage Cross-domain Knowledge Distillation method via Teacher–Student Mutual Learning, termed CdKD-TSML, which enables the peer networks to employ pseudo labels assigned by one another as supplemental supervision. In the first stage, to depress the inevitable noise in hard pseudo labels generated by the self-supervised clustering procedure, we further propose to softly refine the pseudo labels with mutual learning, where networks and label predictions are online optimized cooperatively by distilling knowledge from each other. In the second stage, we jointly fine-tune the distilled models via a cooperative consistency learning strategy, which selects pseudo-labeled samples from the two peers to update the networks, thereby enhancing the generalization of the models. We theoretically demonstrate that the target error of CdKD-TSML is minimized by simultaneously decreasing the pseudo-label noise and alleviating the sample selection bias. Experiments on three challenging UDA datasets demonstrate that CdKD-TSML yields superior results compared to other state-of-the-art methods, proving its effectiveness in this novel setting.

Unsupervised domain adaptation through adversarial enhancement and gradient discrepancy minimization

Although the research involving Unsupervised Domain Adaptation (UDA) has been greatly emerging, many proposed UDA algorithms assume that the domain discrepancy is directly minimized. However, this is impractical in practice. Some methods reduce domain discrepancy by obtaining domain invariant and domain private features, but the disadvantage of this approach is that the domain invariant features obtained contain domain private attributes. So we propose a novel UDA algorithm that introduces an adversarial enhancement and gradient discrepancy minimization (AEGDM) to promote domain alignment. Specifically, the AEGDM method, on the one hand, restrains domain private attributes through enhanced adversation losses to reduce the difficulty of overall transfer and improve the degree of domain alignment, on the other hand, it uses gradient feedback signals to update model classifiers. Furthermore, this paper considers that using a classifier trained in the source domain to assign pseudo-labels to target samples may not be accurate enough, so, we adopt a self-supervised clustering algorithm to obtain pseudo-labels of target samples. The average classification accuracy of our method has achieved 89.5% on Office-31, 89.9% on ImageCLEF-DA and 68.8% on Office-Home, which have 3.3%, 0.2% and 1.5% improved to the ETD, respectively.

Self-supervised clustering on image-subtracted data with deep-embedded self-organizing map

ABSTRACT Developing an effective automatic classifier to separate genuine sources from artifacts is essential for transient follow-ups in wide-field optical surveys. The identification of transient detections from the subtraction artifacts after the image differencing process is a key step in such classifiers, known as real-bogus classification problem. We apply a self-supervised machine learning model, the deep-embedded self-organizing map (DESOM) to this ‘real-bogus’ classification problem. DESOM combines an autoencoder and a self-organizing map to perform clustering in order to distinguish between real and bogus detections, based on their dimensionality-reduced representations. We use 32 × 32 normalized detection thumbnails as the input of DESOM. We demonstrate different model training approaches, and find that our best DESOM classifier shows a missed detection rate of $6.6{{\ \rm per\,cent}}$ with a false-positive rate of $1.5{{\ \rm per\,cent}}$. DESOM offers a more nuanced way to fine-tune the decision boundary identifying likely real detections when used in combination with other types of classifiers, e.g. built on neural networks or decision trees. We also discuss other potential usages of DESOM and its limitations.

SAM-X: sorting algorithm for musculoskeletal x-ray radiography

ObjectiveTo develop a two-phased deep learning sorting algorithm for post-X-ray image acquisition in order to facilitate large musculoskeletal image datasets according to their anatomical entity.MethodsIn total, 42,608 unstructured and pseudonymized radiographs were retrieved from the PACS of a musculoskeletal tumor center. In phase 1, imaging data were sorted into 1000 clusters by a self-supervised model. A human-in-the-loop radiologist assigned weak, semantic labels to all clusters and clusters with the same label were merged. Three hundred thirty-two non-musculoskeletal clusters were discarded. In phase 2, the initial model was modified by “injecting” the identified labels into the self-supervised model to train a classifier. To provide statistical significance, data split and cross-validation were applied. The hold-out test set consisted of 50% external data. To gain insight into the model’s predictions, Grad-CAMs were calculated.ResultsThe self-supervised clustering resulted in a high normalized mutual information of 0.930. The expert radiologist identified 28 musculoskeletal clusters. The modified model achieved a classification accuracy of 96.2% and 96.6% for validation and hold-out test data for predicting the top class, respectively. When considering the top two predicted class labels, an accuracy of 99.7% and 99.6% was accomplished. Grad-CAMs as well as final cluster results underlined the robustness of the proposed method by showing that it focused on similar image regions a human would have considered for categorizing images.ConclusionFor efficient dataset building, we propose an accurate deep learning sorting algorithm for classifying radiographs according to their anatomical entity in the assessment of musculoskeletal diseases.Key Points• Classification of large radiograph datasets according to their anatomical entity.• Paramount importance of structuring vast amounts of retrospective data for modern deep learning applications.• Optimization of the radiological workflow and increase in efficiency as well as decrease of time-consuming tasks for radiologists through deep learning.