Unsupervised Segmentation Research Articles

Two-stage adversarial learning based unsupervised domain adaptation for retinal OCT segmentation.

Deep learning based optical coherence tomography (OCT) segmentation methods have achieved excellent results, allowing quantitative analysis of large-scale data. However, OCT images are often acquired by different devices or under different imaging protocols, which leads to serious domain shift problem. This in turn results in performance degradation of segmentation models. Aiming at the domain shift problem, we propose a two-stage adversarial learning based network (TSANet) that accomplishes unsupervised cross-domain OCT segmentation. In the first stage, a Fourier transform based approach is adopted to reduce image style differences from the image level. Then, adversarial learning networks, including a segmenter and a discriminator, are designed to achieve inter-domain consistency in the segmentation output. In the second stage, pseudo labels of selected unlabeled target domain training data are used to fine-tune the segmenter, which further improves its generalization capability. The proposed method was tested on cross-domain datasets for choroid or retinoschisis segmentation tasks. For choroid segmentation, the model was trained on 400 images and validated on 100 images from the source domain, and then trained on 1320 unlabeled images and tested on 330 images from target domain I, and also trained on 400 unlabeled images and tested on 200 images from target domain II. For retinoschisis segmentation, the model was trained on 1284 images and validated on 312 images from the source domain, and then trained on 1024 unlabeled images and tested on 200 images from the target domain. The proposed method achieved significantly improved results over that without domain adaptation, with improvement of 8.34%, 55.82% and 3.53% in intersection over union (IoU) respectively for the three test sets. The performance is better than some state-of-the-art domain adaptation methods. The proposed TSANet, with image level adaptation, feature level adaptation and pseudo-label based fine-tuning, achieved excellent cross-domain generalization. This alleviates the burden of obtaining additional manual labels when adapting the deep learning model to new OCT data.

Nonparametric K-means clustering-based adaptive unsupervised colour image segmentation

Nonparametric K-means clustering-based adaptive unsupervised colour image segmentation

Ensemble CART surrogate-assisted automatic multi-objective rough fuzzy clustering algorithm for unsupervised image segmentation

Multi-objective clustering algorithms (MOCAs) are popular in unsupervised image segmentation due to their merit of meeting multiple segmentation requirements and the prospect of automatically estimating the number of clusters. However, most of them suffer from high time costs and are easily to be influenced by the uncertainty when handling real complex images. To address these issues, we propose an ensemble classification and regression tree (CART) surrogate-assisted automatic multi-objective rough fuzzy clustering (ECS-AMRFC) algorithm for unsupervised image segmentation. Firstly, a cluster medoid-based encoding scheme is employed to represent solutions with different number of clusters and meanwhile lessen the length of encoding. Then, we design an ensemble CART as the surrogate model to significantly reduce the computational burden. Moreover, a surrogate model management strategy is proposed to accelerate the optimization and enhance the quality of surrogate modeling. To handle the uncertainty in data, we extend the rough fuzzy clustering into MOCAs and construct three complementary objective functions to seek proper cluster medoids from multiple perspectives. In addition, the Gaussian kernel is introduced into the objective functions to handle image pixels that cannot separate linearly in the feature space. Finally, a kernelized rough fuzzy clustering validity index is defined to automatically select the optimal solution with no requirements of any prior knowledge. Experiments show that ECS-AMRFC not only identifies appropriate number of clusters on different kinds of images, but also obtains better segmentation results than state-of-the-art rough fuzzy clustering algorithms and automatic MOCAs.

Learning to Generalize over Subpartitions for Heterogeneity-Aware Domain Adaptive Nuclei Segmentation

Annotation scarcity and cross-modality/stain data distribution shifts are two major obstacles hindering the application of deep learning models for nuclei analysis, which holds a broad spectrum of potential applications in digital pathology. Recently, unsupervised domain adaptation (UDA) methods have been proposed to mitigate the distributional gap between different imaging modalities for unsupervised nuclei segmentation in histopathology images. However, existing UDA methods are built upon the assumption that data distributions within each domain should be uniform. Based on the over-simplified supposition, they propose to align the histopathology target domain with the source domain integrally, neglecting severe intra-domain discrepancy over subpartitions incurred by mixed cancer types and sampling organs. In this paper, for the first time, we propose to explicitly consider the heterogeneity within the histopathology domain and introduce open compound domain adaptation (OCDA) to resolve the crux. In specific, a two-stage disentanglement framework is proposed to acquire domain-invariant feature representations at both image and instance levels. The holistic design addresses the limitations of existing OCDA approaches which struggle to capture instance-wise variations. Two regularization strategies are specifically devised herein to leverage the rich subpartition-specific characteristics in histopathology images and facilitate subdomain decomposition. Moreover, we propose a dual-branch nucleus shape and structure preserving module to prevent nucleus over-generation and deformation in the synthesized images. Experimental results on both cross-modality and cross-stain scenarios over a broad range of diverse datasets demonstrate the superiority of our method compared with state-of-the-art UDA and OCDA methods.Graphical abstract

Exploring Semantic Prompts in the Segment Anything Model for Domain Adaptation

Robust segmentation in adverse weather conditions is crucial for autonomous driving. However, these scenes struggle with recognition and make annotations expensive, resulting in poor performance. As a result, the Segment Anything Model (SAM) was recently proposed to finely segment the spatial structure of scenes and to provide powerful prior spatial information, thus showing great promise in resolving these problems. However, SAM cannot be applied directly for different geographic scales and non-semantic outputs. To address these issues, we propose SAM-EDA, which integrates SAM into an unsupervised domain adaptation mean-teacher segmentation framework. In this method, we use a “teacher-assistant” model to provide semantic pseudo-labels, which will fill in the holes in the fine spatial structure given by SAM and generate pseudo-labels close to the ground truth, which then guide the student model for learning. Here, the “teacher-assistant” model helps to distill knowledge. During testing, only the student model is used, thus greatly improving efficiency. We tested SAM-EDA on mainstream segmentation benchmarks in adverse weather conditions and obtained a more-robust segmentation model.

A guide to unsupervised image segmentation of mCT-scanned cellular metals with mixture modelling and Markov random fields

Characterising the structure of cellular metals is a difficult task. The internal structure of cellular metals can be determined using micro-computed tomography (mCT). However, mCT scanning provides digital images in greyscale with various problematic artefacts. In addition, the grey intensity of cellular metals usually varies greatly due to the internal porosity of the material. Therefore, binary image segmentation to extract material segments from digital images is quite difficult. Our contribution can be summarised as follows. A comprehensive evaluation of various mixture models that have been shown in the literature to be useful for tomography, but for the purpose of binary image segmentation of cellular metals and internal porosity assessment. We propose a novel merging technique to merge different components of the mixture model for the purpose of binary image segmentation of cellular metals. Finally, to enforce spatial regularisation and further improve the binary image segmentation, we combine the obtained two-segment mixture model (material-void mixture model) with Markov random fields and evaluate the effects of different strengths of spatial regularisation. Our proposals are thoroughly investigated using five different types of cellular metals. The reported results are promising and competitive and speak in favour of the relevance of our proposals.

HVS-Unsup: Unsupervised cervical cell instance segmentation method based on human visual simulation

Instance segmentation plays an important role in the automatic diagnosis of cervical cancer. Although deep learning-based instance segmentation methods can achieve outstanding performance, they need large amounts of labeled data. This results in a huge consumption of manpower and material resources. To solve this problem, we propose an unsupervised cervical cell instance segmentation method based on human visual simulation, named HVS-Unsup. Our method simulates the process of human cell recognition and incorporates prior knowledge of cervical cells. Specifically, firstly, we utilize prior knowledge to generate three types of pseudo labels for cervical cells. In this way, the unsupervised instance segmentation is transformed to a supervised task. Secondly, we design a Nucleus Enhanced Module (NEM) and a Mask-Assisted Segmentation module (MAS) to address problems of cell overlapping, adhesion, and even scenarios involving visually indistinguishable cases. NEM can accurately locate the nuclei by the nuclei attention feature maps generated by point-level pseudo labels, and MAS can reduce the interference from impurities by updating the weight of the shallow network through the dice loss. Next, we propose a Category-Wise droploss (CW-droploss) to reduce cell omissions in lower-contrast images. Finally, we employ an iterative self-training strategy to rectify mislabeled instances. Experimental results on our dataset MS-cellSeg, the public datasets Cx22 and ISBI2015 demonstrate that HVS-Unsup outperforms existing mainstream unsupervised cervical cell segmentation methods.

Source-free unsupervised adaptive segmentation for knee joint MRI

Knee osteoarthritis is a prevalent disease worldwide. The automatic segmentation of knee tissues in magnetic resonance (MR) images has important clinical utility in assessing knee osteoarthritis. Deep learning-based methods show great potential in this application, but they often require a large amount of labeled training data, which is challenging and expensive to acquire. Unsupervised domain adaptation that transfers the learned knowledge from a source labeled dataset to a target unlabeled dataset can be used to address this problem. However, in medical scenarios, domain adaption techniques are often limited by access to the source data due to concerns about patient privacy. In this work, we proposed a novel and effective source-free unsupervised domain adaptation method for knee joint multi-tissue segmentation that does not require a source dataset. The proposed framework is split into two stages. In the first stage, matching batch normalization statistics guides the first segmentation network to realize model adaptation and is combined with augmented entropy minimization to obtain pseudo segmentation labels for the target MR images. The pseudo labels generated by the first segmentation network are then refined using a voting strategy to supervise the training of the models in the second stage. In the second stage, uncertainty-aware cross pseudo supervision is used to further boost the performance of the desired segmentation network, which comprises an encoder and a primary decoder. Our experiments demonstrated that the proposed method outperforms the current state-of-the-art source-free unsupervised domain adaptation techniques for segmenting knee tissues in MR images. Our study also demonstrated that the proposed approach is effective in reverse-directional adaptation.

CycleSeg: Simultaneous synthetic CT generation and unsupervised segmentation for MR-only radiotherapy treatment planning of prostate cancer.

MR-only radiotherapy treatment planning is an attractive alternative to conventional workflow, reducing scan time and ionizing radiation. It is crucial to derive the electron density map or synthetic CT (sCT) from MR data to perform dose calculations to enable MR-only treatment planning. Automatic segmentation of relevant organs in MR images can accelerate the process by preventing the time-consuming manual contouring step. However, the segmentation label is available only for CT data in many cases. We propose CycleSeg, a unified framework that generates sCT and corresponding segmentation from MR images without access to MR segmentation labels METHODS: CycleSeg utilizes the CycleGAN formulation to perform unpaired synthesis of sCT and image alignment. To enable MR (sCT) segmentation, CycleSeg incorporates unsupervised domain adaptation by using a pseudo-labeling approach with feature alignment in semantic segmentation space. In contrast to previous approaches that perform segmentation on MR data, CycleSeg could perform segmentation on both MR and sCT. Experiments were performed with data from prostate cancer patients, with 78/7/10 subjects in the training/validation/test sets, respectively. CycleSeg showed the best sCT generation results, with the lowest mean absolute error of 102.2 and the lowest Fréchet inception distance of 13.0. CycleSeg also performed best on MR segmentation, with the highest average dice score of 81.0 and 81.1 for MR and sCT segmentation, respectively. Ablation experiments confirmed the contribution of the proposed components of CycleSeg. CycleSeg effectively synthesized CT and performed segmentation on MR images of prostate cancer patients. Thus, CycleSeg has the potential to expedite MR-only radiotherapy treatment planning, reducing the prescribed scans and manual segmentation effort, and increasing throughput.

Adaptive Segmentation Algorithm for Subtle Defect Images on the Surface of Magnetic Ring Using 2D-Gabor Filter Bank

In order to realize the unsupervised segmentation of subtle defect images on the surface of small magnetic rings and improve the segmentation accuracy and computational efficiency, here, an adaptive threshold segmentation method is proposed based on the improved multi-scale and multi-directional 2D-Gabor filter bank. Firstly, the improved multi-scale and multi-directional 2D-Gabor filter bank was used to filter and reduce the noise on the defect image, suppress the noise pollution inside the target area and the background area, and enhance the difference between the magnetic ring defect and the background. Secondly, this study analyzed the grayscale statistical characteristics of the processed image; the segmentation threshold was constructed according to the gray statistical law of the image; and the adaptive segmentation of subtle defect images on the surface of small magnetic rings was realized. Finally, a classifier based on a BP neural network is designed to classify the scar images and crack images determined by different threshold segmentation methods. The classification accuracies of the iterative method, the OTSU method, the maximum entropy method, and the adaptive threshold segmentation method are, respectively, 85%, 87.5%, 95%, and 97.5%. The adaptive threshold segmentation method proposed in this paper has the highest classification accuracy. Through verification and comparison, the proposed algorithm can segment defects quickly and accurately and suppress noise interference effectively. It is better than other traditional image threshold segmentation methods, validated by both segmentation accuracy and computational efficiency. At the same time, the real-time performance of our algorithm was performed on the advanced SEED-DVS8168 platform.

Online Unsupervised Video Object Segmentation via Contrastive Motion Clustering

Online unsupervised video object segmentation (UVOS) uses the previous frames as its input to automatically separate the primary object(s) from a streaming video without using any further manual annotation. A major challenge is that the model has no access to the future and must rely solely on the history, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., the segmentation mask is predicted from the current frame as soon as it is captured. In this work, a novel contrastive motion clustering algorithm with an optical flow as its input is proposed for the online UVOS by exploiting the common fate principle that visual elements tend to be perceived as a group if they possess the same motion pattern. We build a simple and effective auto-encoder to iteratively summarize non-learnable prototypical bases for the motion pattern, while the bases in turn help learn the representation of the embedding network. Further, a contrastive learning strategy based on a boundary prior is developed to improve foreground and background feature discrimination in the representation learning stage. The proposed algorithm can be optimized on arbitrarily-scale data ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., frame, clip, dataset) and performed in an online fashion. Experiments on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DAVIS₁₆, FBMS</i> , and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SegTrackV2</i> datasets show that the accuracy of our method surpasses the previous state-of-the-art (SoTA) online UVOS method by a margin of 0.8%, 2.9%, and 1.1%, respectively. Furthermore, by using an online deep subspace clustering to tackle the motion grouping, our method is able to achieve higher accuracy at 3× faster inference time compared to SoTA online UVOS method, and making a good trade-off between effectiveness and efficiency. Our code is available at https://github.com/xilin1991/CluterNet.

Integrated Framework for Unsupervised Building Segmentation with Segment Anything Model-Based Pseudo-Labeling and Weakly Supervised Learning

The Segment Anything Model (SAM) has had a profound impact on deep learning applications in remote sensing. SAM, which serves as a prompt-based foundation model for segmentation, exhibits a remarkable capability to “segment anything,” including building objects on satellite or airborne images. To facilitate building segmentation without inducing supplementary prompts or labels, we applied a sequential approach of generating pseudo-labels and incorporating an edge-driven model. We first segmented the entire scene by SAM and masked out unwanted objects to generate pseudo-labels. Subsequently, we employed an edge-driven model designed to enhance the pseudo-label by using edge information to reconstruct the imperfect building features. Our model simultaneously utilizes spectral features from SAM-oriented building pseudo-labels and edge features from resultant images from the Canny edge detector and, thus, when combined with conditional random fields (CRFs), shows capability to extract and learn building features from imperfect pseudo-labels. By integrating the SAM-based pseudo-label with our edge-driven model, we establish an unsupervised framework for building segmentation that operates without explicit labels. Our model excels in extracting buildings compared with other state-of-the-art unsupervised segmentation models and even outperforms supervised models when trained in a fully supervised manner. This achievement demonstrates the potential of our model to address the lack of datasets in various remote sensing domains for building segmentation.

Parameter probabilistic prediction for satellite power system based on unsupervised multi-dimensional sequence segmentation

With the increasing operational autonomy and mission complexity of satellites, the power system, as one of the most critical subsystems in a satellite platform, needs to provide high-quality and fast-response power output under complex load requirements. State monitoring is valuable to ensure the power system's stable and safe operation. Monitoring and predicting the key parameters based on the satellite power system's telemetry data are meaningful in detecting potential anomalies. However, the multi-physical high-dimensional telemetry data of the satellite power system are oriented to the operating conditions. The dynamic operating conditions of the satellite power system make it difficult to establish the proper parameter prediction model with high adaptability. To address these challenging issues, this paper proposes a parameter probabilistic prediction based on unsupervised time series segmentation. Firstly, the telemetry data under dynamic operation conditions are divided based on the unsupervised dynamic factor model and K-Means algorithm. The dynamic factor model is used to extract the common factor sequences of the multidimensional telemetry data according to the dynamic relationship between different parameters, followed by the clustering operation using K-Means to classify the multidimensional telemetry data into different sub-sequences. Secondly, a probabilistic prediction method based on the spatio-temporal fusion of multi-physical parameters is proposed. The method utilizes a Bayesian neural network to learn the spatial relationship between parameters and applies Gaussian process regression to establish the temporal dependence of parameters. Afterward, a specific strategy is utilized to fuse the prediction results of the two, thus realizing the fusion of spatio-temporal information of satellite multi-dimensional telemetry data. The experimental results based on the real meteorological satellite power system telemetry data in 2010 show that the average absolute errors of the key parameter predictions after the division of the working conditions are all less than 0.25, which is improved by more than 37.08 % compared with the pre-division. This indicates that the proposed method can well recognize the dynamic operation modes and has a positive advancing effect on the parameter prediction task. Meanwhile, the average absolute error of the proposed prediction method is improved by at least 40.4 % compared with other single prediction models. It shows that the method proposed in this paper can realize high-precision prediction of parameters, and has good application prospects in condition monitoring and anomaly detection of satellite power systems.

An evolutionary vehicle scanning method for bridges based on time series segmentation and change point detection

In this paper, the problem of bridge health monitoring is formulated as an unsupervised semantic segmentation where the aim is to identify the onset of structural damage by detecting unexpected irregularities in the time series data. In particular, the main objective of this study is to approach this problem through an indirect sensing framework based on the concept of vehicle-mounted scanning. Thus, the sole source of information is the acceleration responses of the moving vehicle passing over the bridge. This is achieved through automatic feature learning by adopting the concept of matrix profile to conduct an all-pair-similarity search for a given time series, where the location of the contextual change is determined by the corresponding Corrected Arc Crossings (CAC) profile. The performance of the method is first evaluated through numerical investigations for different damage cases where the effect of operational uncertainties such as road roughness, and variability in vehicle properties are considered. Further, the method is experimentally evaluated using a lab-scale bridge model with different damage cases. Through these investigations, it is consistently observed that the location of the change point always coincides with the minimum point of the CAC profile. The proposed framework has three major advantages making it more appealing compared to the competing techniques. First, it has been fully formulated in the context of unsupervised learning where no access to labeled data is required. Second, it is capable of working with a limited amount of data yet providing promising results. Finally, it can be easily applied to online settings where data is streaming in real-time. These successful results demonstrate the potential of the proposed method as a cost-effective and rigorous bridge inspection tool and open up opportunities in the field of indirect bridge health monitoring.

Unsupervised PCB Anomaly Segmentation with Foundational Models

Unsupervised PCB Anomaly Segmentation with Foundational Models

Evaluating the Efficacy of Segment Anything Model for Delineating Agriculture and Urban Green Spaces in Multiresolution Aerial and Spaceborne Remote Sensing Images

Segmentation of Agricultural Remote Sensing Images (ARSIs) stands as a pivotal component within the intelligent development path of agricultural information technology. Similarly, quick and effective delineation of urban green spaces (UGSs) in high-resolution images is also increasingly needed as input in various urban simulation models. Numerous segmentation algorithms exist for ARSIs and UGSs; however, a model with exceptional generalization capabilities and accuracy remains elusive. Notably, the newly released Segment Anything Model (SAM) by META AI is gaining significant recognition in various domains for segmenting conventional images, yielding commendable results. Nevertheless, SAM’s application in ARSI and UGS segmentation has been relatively limited. ARSIs and UGSs exhibit distinct image characteristics, such as prominent boundaries, larger frame sizes, and extensive data types and volumes. Presently, there is a dearth of research on how SAM can effectively handle various ARSI and UGS image types and deliver superior segmentation outcomes. Thus, as a novel attempt in this paper, we aim to evaluate SAM’s compatibility with a wide array of ARSI and UGS image types. The data acquisition platform comprises both aerial and spaceborne sensors, and the study sites encompass most regions of the United States, with images of varying resolutions and frame sizes. It is noteworthy that the segmentation effect of SAM is significantly influenced by the content of the image, as well as the stability and accuracy across images of different resolutions and sizes. However, in general, our findings indicate that resolution has a minimal impact on the effectiveness of conditional SAM-based segmentation, maintaining an overall segmentation accuracy above 90%. In contrast, the unsupervised segmentation approach, SAM, exhibits performance issues, with around 55% of images (3 m and coarser resolutions) experiencing lower accuracy on low-resolution images. Whereas frame size exerts a more substantial influence, as the image size increases, the accuracy of unsupervised segmentation methods decreases extremely fast, and conditional segmentation methods also show some degree of degradation. Additionally, SAM’s segmentation efficacy diminishes considerably in the case of images featuring unclear edges and minimal color distinctions. Consequently, we propose enhancing SAM’s capabilities by augmenting the training dataset and fine-tuning hyperparameters to align with the demands of ARSI and UGS image segmentation. Leveraging the multispectral nature and extensive data volumes of remote sensing images, the secondary development of SAM can harness its formidable segmentation potential to elevate the overall standard of ARSI and UGS image segmentation.

Unsupervised unpaired multiple fusion adaptation aided with self-attention generative adversarial network for scar tissues segmentation framework

Late gadolinium enhancement (LGE) is a specialized imaging technique used in cardiovascular magnetic resonance (CMR) imaging to detect and characterize areas of scar tissue or fibrosis within the heart muscle. After a heart attack, the affected region of the heart muscle becomes scarred due to insufficient blood supply. LGE CMR enhances the infarcted myocardium to appear with distinctive brightness compared to healthy tissues. It is primarily used to assess various heart conditions and can identify the extent and location of scar tissue, helping in risk stratification and guiding treatment decisions. Automated myocardium segmentation in LGE CMR poses challenges due to intensity heterogeneity caused by the accumulation of contrast agents in infarcted areas. Furthermore, LGE CMR images with gold standard labels are particularly scarce compared to other sequences, making the task even more demanding. Besides, the heart’s motion involves complex non-rigid deformations, including regional wall thickening, circumferential, and longitudinal ventricular shortening, making it challenging to model. This work presents an unsupervised unpaired domain adaptive scar tissue segmentation framework aided with the self-attention generative adversarial network. Besides, we proposed a novel segmentation architecture based on a parallel transformer for scar segmentation and compared the results with and without style transfer GAN and unpaired style self-supervised attention GAN. Besides, to model longitudinal changes in scare, we used eight days’ time points as a source and 1 month and 12 as a target and applied our proposed attention-generative network to transfer style from one-time point to another time. Experiments on four different LGE MICCAI datasets from 2019 to 2023 achieved significantly better performance than state-of-the-art segmentation methods.

Unsupervised corneal contour extraction algorithm with shared model for dynamic deformation videos: improving accuracy and noise resistance

BackgroundIn this study, an automatic corneal contour extraction algorithm with a shared model is developed to extract contours from dynamic corneal videos containing noise, which improves the accuracy of corneal biomechanical evaluation and clinical diagnoses. The algorithm does not require manual labeling and completes the unsupervised semantic segmentation of each frame in corneal dynamic deformation videos based on a fully convolutional deep-learning network using corneal geometry and texture information.ResultsWe included 1027 corneal videos at Tianjin Eye Hospital (Nankai University Affiliated Eye Hospital) from May 2020 to November 2021. The videos were obtained by the ultra-high-speed Scheimpflug camera, and then we used the shared model mechanism to accelerate the segmentation of corneal regions in videos, effectively resist noise, determine corneal regions based on shape factors, and finally achieve automatic and accurate extraction of corneal region contours. The Intersection over Union (IoU) of the extracted and real corneal contours using this algorithm reached 95%, and the average overlap error was 0.05, implying that the extracted corneal contour overlapped almost completely with the real contour.ConclusionsCompared to other algorithms, the method introduced in this study does not require manual annotation of corneal contour data in advance and can still extract accurate corneal contours from noisy corneal videos with good repeatability.

Benchmarking and Analysis of Unsupervised Object Segmentation from Real-World Single Images

In this paper, we study the problem of unsupervised object segmentation from single images. We do not introduce a new algorithm, but systematically investigate the effectiveness of existing unsupervised models on challenging real-world images. We first introduce seven complexity factors to quantitatively measure the distributions of background and foreground object biases in appearance and geometry for datasets with human annotations. With the aid of these factors, we empirically find that, not surprisingly, existing unsupervised models fail to segment generic objects in real-world images, although they can easily achieve excellent performance on numerous simple synthetic datasets, due to the vast gap in objectness biases between synthetic and real images. By conducting extensive experiments on multiple groups of ablated real-world datasets, we ultimately find that the key factors underlying the failure of existing unsupervised models on real-world images are the challenging distributions of background and foreground object biases in appearance and geometry. Because of this, the inductive biases introduced in existing unsupervised models can hardly capture the diverse object distributions. Our research results suggest that future work should exploit more explicit objectness biases in the network design.

Contrastive Registration for Unsupervised Medical Image Segmentation.

Medical image segmentation is an important task in medical imaging, as it serves as the first step for clinical diagnosis and treatment planning. While major success has been reported using deep learning supervised techniques, they assume a large and well-representative labeled set. This is a strong assumption in the medical domain where annotations are expensive, time-consuming, and inherent to human bias. To address this problem, unsupervised segmentation techniques have been proposed in the literature. Yet, none of the existing unsupervised segmentation techniques reach accuracies that come even near to the state-of-the-art of supervised segmentation methods. In this work, we present a novel optimization model framed in a new convolutional neural network (CNN)-based contrastive registration architecture for unsupervised medical image segmentation called CLMorph. The core idea of our approach is to exploit image-level registration and feature-level contrastive learning, to perform registration-based segmentation. First, we propose an architecture to capture the image-to-image transformation mapping via registration for unsupervised medical image segmentation. Second, we embed a contrastive learning mechanism in the registration architecture to enhance the discriminative capacity of the network at the feature level. We show that our proposed CLMorph technique mitigates the major drawbacks of existing unsupervised techniques. We demonstrate, through numerical and visual experiments, that our technique substantially outperforms the current state-of-the-art unsupervised segmentation methods on two major medical image datasets.