Unsupervised Segmentation Method Research Articles

Unsupervised cervical cell instance segmentation method integrating cellular characteristics.

Cell instance segmentation is a key technology for cervical cancer auxiliary diagnosis systems. However, pixel-level annotation is time-consuming and labor-intensive, making it difficult to obtain a large amount of annotated data. This results in the model not being fully trained. In response to these problems, this paper proposes an unsupervised cervical cell instance segmentation method that integrates cell characteristics. Cervical cells have a clear corresponding structure between the nucleus and cytoplasm. This method fully takes this feature into account by building a dual-flow framework to locate the nucleus and cytoplasm and generate high-quality pseudo-labels. In the nucleus segmentation stage, the position and range of the nucleus are determined using the standard cell-restricted nucleus segmentation method. In the cytoplasm segmentation stage, a multi-angle collaborative segmentation method is used to achieve the positioning of the cytoplasm. First, taking advantage of the self-similarity characteristics of pixel blocks in cells, a cytoplasmic segmentation method based on self-similarity map iteration is proposed. The pixel blocks are mapped from the perspective of local details, and the iterative segmentation is repeated. Secondly, using low-level features such as cell color and shape, a self-supervised heatmap-aware cytoplasm segmentation method is proposed to obtain the activation map of the cytoplasm from the perspective of global attention. The two methods are fused to determine cytoplasmic regions, and combined with nuclear locations, high-quality pseudo-labels are generated. These pseudo-labels are used to train the model cyclically, and the loss strategy is used to encourage the model to discover new object masks, thereby obtaining a segmentation model with better performance. Experimental results show that this method achieves good results in cytoplasm segmentation. On the three datasets of ISBI, MS_CellSeg, and Cx22, 54.32%, 44.64%, and 66.52% AJI were obtained, respectively, which is better than other typical unsupervised methods selected in this article.

Automated identification of soil functional components based on NanoSIMS data

NanoSIMS technique allows to investigate the micro-spatial organization in complex structures in multiple scientific fields such as material science, cosmochemistry, and biogeochemistry. In soil biogeochemistry applications, NanoSIMS-based approaches aim to disentangle the interactions of organic matter (OM) and mineral phases in the heterogeneous soil microstructure. Investigating the spatial arrangement of distinct organic and mineral functional components is necessary to understand how these components interact and contribute to biogeochemical processes in soil systems. Identifying soil functional components within NanoSIMS measurements necessitates advanced and efficient data processing tools capable of accessibility and automation. We have developed a pre-processing tool to streamline NanoSIMS data preparation and handling. The tool is provided as an open-source software toolbox (NanoT). In addition, a two-step unsupervised segmentation method was developed to identify soil functional components based on NanoSIMS analyses. To illustrate the segmentation method, here we describe its application to two exemplary NanoSIMS measurements. This allows to distinguish mineral- and OM-dominated regions, as well as different mineral phases. To improve the detection of iron oxides and aluminosilicates, the 56Fe16O− channel was separately processed. The presented NanoSIMS-based processing workflow helps to disentangle functional components within a biogeochemically-diverse microstructure in soils and further warrants applications to a wide range of complex environmental samples.

LF-SynthSeg: Label-Free Brain Tissue-Assisted Tumor Synthesis and Segmentation.

Unsupervised brain tumor segmentation is pivotal in realms of disease diagnosis, surgical planning, and treatment response monitoring, with the distinct advantage of obviating the need for labeled data. Traditional methodologies in this domain, however, often fall short in fully capitalizing on the extensive prior knowledge of brain tissue, typically approaching the task merely as an anomaly detection challenge. In our research, we present an innovative strategy that effectively integrates brain tissues' prior knowledge into both the synthesis and segmentation of brain tumor from T2-weighted Magnetic Resonance Imaging scans. Central to our method is the tumor synthesis mechanism, employing randomly generated ellipsoids in conjunction with the intensity profiles of brain tissues. This methodology not only fosters a significant degree of variation in the tumor presentations within the synthesized images but also facilitates the creation of an essentially unlimited pool of abnormal T2-weighted images. These synthetic images closely replicate the characteristics of real tumor-bearing scans. Our training protocol extends beyond mere tumor segmentation; it also encompasses the segmentation of brain tissues, thereby directing the networkâs attention to the boundary relationship between brain tumor and brain tissue, thus improving the robustness of our method. We evaluate our approach across five widely recognized public datasets (BRATS 2019, BRATS 2020, BRATS 2021, PED and SSA), and the results show that our method outperforms state-of-the-art unsupervised tumor segmentation methods by large margins. Moreover, the proposed method achieves more than 92 % of the fully supervised performance on the same testing datasets.

Segment Anything Model Combined with Multi-Scale Segmentation for Extracting Complex Cultivated Land Parcels in High-Resolution Remote Sensing Images

Accurate cultivated land parcel data are an essential analytical unit for further agricultural monitoring, yield estimation, and precision agriculture management. However, the high degree of landscape fragmentation and the irregular shapes of cultivated land parcels, influenced by topography and human activities, limit the effectiveness of parcel extraction. The visual semantic segmentation model based on the Segment Anything Model (SAM) provides opportunities for extracting multi-form cultivated land parcels from high-resolution images; however, the performance of the SAM in extracting cultivated land parcels requires further exploration. To address the difficulty in obtaining parcel extraction that closely matches the true boundaries of complex large-area cultivated land parcels, this study used segmentation patches with cultivated land boundary information obtained from SAM unsupervised segmentation as constraints, which were then incorporated into the subsequent multi-scale segmentation. A combined method of SAM unsupervised segmentation and multi-scale segmentation was proposed, and it was evaluated in different cultivated land scenarios. In plain areas, the precision, recall, and IoU for cultivated land parcel extraction improved by 6.57%, 10.28%, and 9.82%, respectively, compared to basic SAM extraction, confirming the effectiveness of the proposed method. In comparison to basic SAM unsupervised segmentation and point-prompt SAM conditional segmentation, the SAM unsupervised segmentation combined with multi-scale segmentation achieved considerable improvements in extracting complex cultivated land parcels. This study confirms that, under zero-shot and unsupervised conditions, the SAM unsupervised segmentation combined with the multi-scale segmentation method demonstrates strong cross-region and cross-data source transferability and effectiveness for extracting complex cultivated land parcels across large areas.

UMInSe: An Unsupervised Method for Segmentation and Detection of Surgical Instruments based on K-means

UMInSe: An Unsupervised Method for Segmentation and Detection of Surgical Instruments based on K-means

Physics-guided interpretable CNN for SAR target recognition

Deep Learning (DL) model has been widely used in the field of Synthetic Aperture Radar Automatic Target Recognition (SAR-ATR) and has achieved excellent performance. However, the black-box nature of DL models has been the focus of criticism, especially in the application of SAR-ATR, which is closely associated with the national defense and security domain. To address these issues, a new interpretable recognition model Physics-Guided BagNet (PGBN) is proposed in this article. The model adopts an interpretable convolutional neural network framework and uses time–frequency analysis to extract physical scattering features in SAR images. Based on the physical scattering features, an unsupervised segmentation method is proposed to distinguish targets from the background in SAR images. On the basis of the segmentation result, a structure is designed, which constrains the model’s spatial attention to focus more on the targets themselves rather than the background, thereby making the model’s decision-making more in line with physical principles. In contrast to previous interpretable research methods, this model combines interpretable structure with physical interpretability, further reducing the model’s risk of error recognition. Experiments on the MSTAR dataset verify that the PGBN model exhibits excellent interpretability and recognition performance, and comparative experiments with heatmaps indicate that the physical feature guidance module presented in this article can constrain the model to focus more on the target itself rather than the background.

UNSEG: unsupervised segmentation of cells and their nuclei in complex tissue samples

Multiplexed imaging technologies have made it possible to interrogate complex tissue microenvironments at sub-cellular resolution within their native spatial context. However, proper quantification of this complexity requires the ability to easily and accurately segment cells into their sub-cellular compartments. Within the supervised learning paradigm, deep learning-based segmentation methods demonstrating human level performance have emerged. However, limited work has been done in developing such generalist methods within the unsupervised context. Here we present an easy-to-use unsupervised segmentation (UNSEG) method that achieves deep learning level performance without requiring any training data via leveraging a Bayesian-like framework, and nucleus and cell membrane markers. We show that UNSEG is internally consistent and better at generalizing to the complexity of tissue morphology than current deep learning methods, allowing it to unambiguously identify the cytoplasmic compartment of a cell, and localize molecules to their correct sub-cellular compartment. We also introduce a perturbed watershed algorithm for stably and automatically segmenting a cluster of cell nuclei into individual nuclei that increases the accuracy of classical watershed. Finally, we demonstrate the efficacy of UNSEG on a high-quality annotated gastrointestinal tissue dataset we have generated, on publicly available datasets, and in a range of practical scenarios.

DynaSeg: A deep dynamic fusion method for unsupervised image segmentation incorporating feature similarity and spatial continuity

Our work tackles the fundamental challenge of image segmentation in computer vision, which is crucial for diverse applications. While supervised methods demonstrate proficiency, their reliance on extensive pixel-level annotations limits scalability. We introduce DynaSeg, an innovative unsupervised image segmentation approach that overcomes the challenge of balancing feature similarity and spatial continuity without relying on extensive hyperparameter tuning. Unlike traditional methods, DynaSeg employs a dynamic weighting scheme that automates parameter tuning, adapts flexibly to image characteristics, and facilitates easy integration with other segmentation networks. By incorporating a Silhouette Score Phase, DynaSeg prevents undersegmentation failures where the number of predicted clusters might converge to one. DynaSeg uses CNN-based and pre-trained ResNet feature extraction, making it computationally efficient and more straightforward than other complex models. Experimental results showcase state-of-the-art performance, achieving a 12.2% and 14.12% mIOU improvement over current unsupervised segmentation approaches on COCO-All and COCO-Stuff datasets, respectively. We provide qualitative and quantitative results on five benchmark datasets, demonstrating the efficacy of the proposed approach. Code available at \\url{https://github.com/RyersonMultimediaLab/DynaSeg}

An unsupervised semantic segmentation method that combines the ImSE-Net model with SLICm superpixel optimization

ABSTRACT In the field of remote sensing, using a large amount of labeled image data to supervise the training of fully convolutional networks for the semantic segmentation of images is expensive. However, using a small amount of labeled data can lead to reduced network performance. This paper proposes an unsupervised semantic segmentation method that combines the ImSE-Net model with SLICm superpixel optimization. First, the ImSE-Net model is used to extract semantic features from the image to obtain rough semantic segmentation results. Then, the SLICm superpixel segmentation algorithm is used to segment the input image into superpixel images. Finally, an unsupervised semantic segmentation model (UGLS) is used to combine high-level abstract semantic features with detailed information on superpixels to obtain edge-optimized semantic segmentation results. Experimental results show that compared with other semantic segmentation algorithms, our method more effectively handles unbalanced areas, such as object boundaries, and achieves better segmentation results, with higher semantic consistency.

Automatic Marker Generation to Similarity Based Region Merging Algorithm using Edge Information

One of the most important tasks in computer vision is image segmentation. Several interactive strategies are utilized for picture segmentation because automated techniques are difficult for this kind of work. The outcome of interactive methods is mostly determined by user input. Obtaining excellent interactions for huge datasets is challenging. However, automated picture segmentation is starting to play a significant role in image analysis and computer vision. Effective and efficient segmentation outcomes are obtained using the interacting region merging method suggested by Maximal Similarity based region merging algorithm. Limitation of MSBRM is it requires some efforts on the part of users and is yet not a fully-automatic approach. Here, we suggest a completely fresh unsupervised image segmentation method that combines edge information with MSBRM. We propose an integrated framework to generate object markers for similarity based region algorithm using edge information. Long edges give rough distribution of objects in image. After retrieving edges using phase congruency, edge processing operations are employed to remove small edges and to group color similar long boundaries. Centroids of long boundaries are used as object markers to the MSBRM algorithm. The generation of object markers is done using edge segment grouping. These object markers guide the region merging process. The proposed method shows its effectiveness in segmenting natural real world color images.

Development and validation of a semi-automated and unsupervised method for femur segmentation from CT

Quantitative computed tomography (QCT)-based in silico models have demonstrated improved accuracy in predicting hip fractures with respect to the current gold standard, the areal bone mineral density. These models require that the femur bone is segmented as a first step. This task can be challenging, and in fact, it is often almost fully manual, which is time-consuming, operator-dependent, and hard to reproduce. This work proposes a semi-automated procedure for femur bone segmentation from CT images. The proposed procedure is based on the bone and joint enhancement filter and graph-cut algorithms. The semi-automated procedure performances were assessed on 10 subjects through comparison with the standard manual segmentation. Metrics based on the femur geometries and the risk of fracture assessed in silico resulting from the two segmentation procedures were considered. The average Hausdorff distance (0.03 ± 0.01 mm) and the difference union ratio (0.06 ± 0.02) metrics computed between the manual and semi-automated segmentations were significantly higher than those computed within the manual segmentations (0.01 ± 0.01 mm and 0.03 ± 0.02). Besides, a blind qualitative evaluation revealed that the semi-automated procedure was significantly superior (p < 0.001) to the manual one in terms of fidelity to the CT. As for the hip fracture risk assessed in silico starting from both segmentations, no significant difference emerged between the two (R2 = 0.99). The proposed semi-automated segmentation procedure overcomes the manual one, shortening the segmentation time and providing a better segmentation. The method could be employed within CT-based in silico methodologies and to segment large volumes of images to train and test fully automated and supervised segmentation methods.

Generalizable Fourier Augmentation for Unsupervised Video Object Segmentation

The performance of existing unsupervised video object segmentation methods typically suffers from severe performance degradation on test videos when tested in out-of-distribution scenarios. The primary reason is that the test data in real- world may not follow the independent and identically distribution (i.i.d.) assumption, leading to domain shift. In this paper, we propose a generalizable fourier augmentation method during training to improve the generalization ability of the model. To achieve this, we perform Fast Fourier Transform (FFT) over the intermediate spatial domain features in each layer to yield corresponding frequency representations, including amplitude components (encoding scene-aware styles such as texture, color, contrast of the scene) and phase components (encoding rich semantics). We produce a variety of style features via Gaussian sampling to augment the training data, thereby improving the generalization capability of the model. To further improve the cross-domain generalization performance of the model, we design a phase feature update strategy via exponential moving average using phase features from past frames in an online update manner, which could help the model to learn cross-domain-invariant features. Extensive experiments show that our proposed method achieves the state-of-the-art performance on popular benchmarks.

Feature Re-Representation and Reliable Pseudo Label Retraining for Cross-Domain Semantic Segmentation.

This paper presents a novel unsupervised domain adaptation method for semantic segmentation. We argue that a good representation of the target-domain data should keep both the knowledge from the source domain and the target-domain-specific information. To obtain the knowledge from the source domain, we first learn a set of bases to characterize the feature distribution of the source domain, then features from both the source and the target domain are re-represented as a weighted summation of the source bases. A discriminator is additionally introduced to make the re-representation responsibilities of both domain features under the same bases indistinguishable. In this way, the domain gap between the source re-representation and target re-representation is minimized, and the re-represented target domain features contain the source domain information. Then we combine the feature re-representation with the original domain-specific feature together for subsequent pixel-wise classification. To further make the re-represented target features semantically meaningful, a Reliable Pseudo Label Retraining (RPLR) strategy is proposed, which utilizes the consistency of the prediction by the networks trained with multi-view source images to select the clean pseudo labels on unlabeled target images for re-training. Extensive experiments demonstrate the competitive performance of our approach for unsupervised domain adaptation on the semantic segmentation benchmarks.

HVS-Unsup: Unsupervised cervical cell instance segmentation method based on human visual simulation

Instance segmentation plays an important role in the automatic diagnosis of cervical cancer. Although deep learning-based instance segmentation methods can achieve outstanding performance, they need large amounts of labeled data. This results in a huge consumption of manpower and material resources. To solve this problem, we propose an unsupervised cervical cell instance segmentation method based on human visual simulation, named HVS-Unsup. Our method simulates the process of human cell recognition and incorporates prior knowledge of cervical cells. Specifically, firstly, we utilize prior knowledge to generate three types of pseudo labels for cervical cells. In this way, the unsupervised instance segmentation is transformed to a supervised task. Secondly, we design a Nucleus Enhanced Module (NEM) and a Mask-Assisted Segmentation module (MAS) to address problems of cell overlapping, adhesion, and even scenarios involving visually indistinguishable cases. NEM can accurately locate the nuclei by the nuclei attention feature maps generated by point-level pseudo labels, and MAS can reduce the interference from impurities by updating the weight of the shallow network through the dice loss. Next, we propose a Category-Wise droploss (CW-droploss) to reduce cell omissions in lower-contrast images. Finally, we employ an iterative self-training strategy to rectify mislabeled instances. Experimental results on our dataset MS-cellSeg, the public datasets Cx22 and ISBI2015 demonstrate that HVS-Unsup outperforms existing mainstream unsupervised cervical cell segmentation methods.

Evaluating the Efficacy of Segment Anything Model for Delineating Agriculture and Urban Green Spaces in Multiresolution Aerial and Spaceborne Remote Sensing Images

Segmentation of Agricultural Remote Sensing Images (ARSIs) stands as a pivotal component within the intelligent development path of agricultural information technology. Similarly, quick and effective delineation of urban green spaces (UGSs) in high-resolution images is also increasingly needed as input in various urban simulation models. Numerous segmentation algorithms exist for ARSIs and UGSs; however, a model with exceptional generalization capabilities and accuracy remains elusive. Notably, the newly released Segment Anything Model (SAM) by META AI is gaining significant recognition in various domains for segmenting conventional images, yielding commendable results. Nevertheless, SAM’s application in ARSI and UGS segmentation has been relatively limited. ARSIs and UGSs exhibit distinct image characteristics, such as prominent boundaries, larger frame sizes, and extensive data types and volumes. Presently, there is a dearth of research on how SAM can effectively handle various ARSI and UGS image types and deliver superior segmentation outcomes. Thus, as a novel attempt in this paper, we aim to evaluate SAM’s compatibility with a wide array of ARSI and UGS image types. The data acquisition platform comprises both aerial and spaceborne sensors, and the study sites encompass most regions of the United States, with images of varying resolutions and frame sizes. It is noteworthy that the segmentation effect of SAM is significantly influenced by the content of the image, as well as the stability and accuracy across images of different resolutions and sizes. However, in general, our findings indicate that resolution has a minimal impact on the effectiveness of conditional SAM-based segmentation, maintaining an overall segmentation accuracy above 90%. In contrast, the unsupervised segmentation approach, SAM, exhibits performance issues, with around 55% of images (3 m and coarser resolutions) experiencing lower accuracy on low-resolution images. Whereas frame size exerts a more substantial influence, as the image size increases, the accuracy of unsupervised segmentation methods decreases extremely fast, and conditional segmentation methods also show some degree of degradation. Additionally, SAM’s segmentation efficacy diminishes considerably in the case of images featuring unclear edges and minimal color distinctions. Consequently, we propose enhancing SAM’s capabilities by augmenting the training dataset and fine-tuning hyperparameters to align with the demands of ARSI and UGS image segmentation. Leveraging the multispectral nature and extensive data volumes of remote sensing images, the secondary development of SAM can harness its formidable segmentation potential to elevate the overall standard of ARSI and UGS image segmentation.

Contrastive Registration for Unsupervised Medical Image Segmentation.

Medical image segmentation is an important task in medical imaging, as it serves as the first step for clinical diagnosis and treatment planning. While major success has been reported using deep learning supervised techniques, they assume a large and well-representative labeled set. This is a strong assumption in the medical domain where annotations are expensive, time-consuming, and inherent to human bias. To address this problem, unsupervised segmentation techniques have been proposed in the literature. Yet, none of the existing unsupervised segmentation techniques reach accuracies that come even near to the state-of-the-art of supervised segmentation methods. In this work, we present a novel optimization model framed in a new convolutional neural network (CNN)-based contrastive registration architecture for unsupervised medical image segmentation called CLMorph. The core idea of our approach is to exploit image-level registration and feature-level contrastive learning, to perform registration-based segmentation. First, we propose an architecture to capture the image-to-image transformation mapping via registration for unsupervised medical image segmentation. Second, we embed a contrastive learning mechanism in the registration architecture to enhance the discriminative capacity of the network at the feature level. We show that our proposed CLMorph technique mitigates the major drawbacks of existing unsupervised techniques. We demonstrate, through numerical and visual experiments, that our technique substantially outperforms the current state-of-the-art unsupervised segmentation methods on two major medical image datasets.

Q-Seg: Quantum Annealing-Based Unsupervised Image Segmentation.

We present Q-Seg, a novel unsupervised image segmentation method based on quantum annealing, tailored for existing quantum hardware. We formulate the pixel- wise segmentation problem, which assimilates spectral and spatial information of the image, as a graph-cut optimization task. Our method efficiently leverages the interconnected qubit topology of the D-Wave Advantage device, offering superior scalability over existing quantum approaches and outperforming several tested state-of-the-art classical methods. Empirical evaluations on synthetic datasets have shown that Q-Seg has better runtime performance than the state-of-the-art classical optimizer Gurobi. The method has also been tested on earth observation image segmentation, a critical area with noisy and unreliable annotations. In the era of noisy intermediate-scale quantum, Q-Seg emerges as a reliable contender for real-world applications in comparison to advanced techniques like Segment Anything. Consequently, Q-Seg offers a promising solution using available quantum hardware, especially in situations constrained by limited labeled data and the need for efficient computational runtime.

Crop Identification of UAV Images Based on an Unsupervised Semantic Segmentation Method

Crop Identification of UAV Images Based on an Unsupervised Semantic Segmentation Method

Self-ensembling with mask-boundary domain adaptation for optic disc and cup segmentation

Due to different retinal fundus image acquisition devices having various imaging principles, domain shift often occurs between different datasets. Hence, a segmentation network well-trained on one dataset (i.e., source domain) usually obtains very poor performance on another dataset (i.e., target domain), which results in us having to annotate the new dataset (target domain) to train the segmentation network again. However, annotating a new dataset is usually time-consuming and laborious. To address this problem, we proposed a novel unsupervised domain adaptation method for optic disc and cup segmentation. To be specific, we first utilized a domain adaptation method based on self-ensembling to effectively align the features of the source domain and target domain. Then, we designed a novel backbone network (MBU-Net) to make full use of the mask and boundary information to improve the segmentation performance of self-ensembling. Finally, we proposed an output-level adversarial domain adaptation (OADA) to address the domain shift problem of the structured output space in self-ensembling. In experiments, we test our proposed method on three different target domain datasets including Target Domain 1 (RIM-ONE_r3 dataset), Target Domain 2 (Drishti-GS dataset) and Target Domain 3 (REFUGE dataset). The experimental results demonstrate that our proposed method outperforms the compared state-of-the-art methods in the optic disc and cup segmentation tasks.

Combining Color and Spatial Image Features for Unsupervised Image Segmentation with Mixture Modelling and Spectral Clustering

The demand for accurate and reliable unsupervised image segmentation methods is high. Regardless of whether we are faced with a problem for which we do not have a usable training dataset, or whether it is not possible to obtain one, we still need to be able to extract the desired information from images. In such cases, we are usually gently pushed towards the best possible clustering method, as it is often more robust than simple traditional image processing methods. We investigate the usefulness of combining two clustering methods for unsupervised image segmentation. We use the mixture models to extract the color and spatial image features based on the obtained output segments. Then we construct a similarity matrix (adjacency matrix) based on these features to perform spectral clustering. In between, we propose a label noise correction using Markov random fields. We investigate the usefulness of our method on many hand-crafted images of different objects with different shapes, colorization, and noise. Compared to other clustering methods, our proposal performs better, with 10% higher accuracy. Compared to state-of-the-art supervised image segmentation methods based on deep convolutional neural networks, our proposal proves to be competitive.