Unsupervised Segmentation Research Articles

Integrated Framework for Unsupervised Building Segmentation with Segment Anything Model-Based Pseudo-Labeling and Weakly Supervised Learning

The Segment Anything Model (SAM) has had a profound impact on deep learning applications in remote sensing. SAM, which serves as a prompt-based foundation model for segmentation, exhibits a remarkable capability to “segment anything,” including building objects on satellite or airborne images. To facilitate building segmentation without inducing supplementary prompts or labels, we applied a sequential approach of generating pseudo-labels and incorporating an edge-driven model. We first segmented the entire scene by SAM and masked out unwanted objects to generate pseudo-labels. Subsequently, we employed an edge-driven model designed to enhance the pseudo-label by using edge information to reconstruct the imperfect building features. Our model simultaneously utilizes spectral features from SAM-oriented building pseudo-labels and edge features from resultant images from the Canny edge detector and, thus, when combined with conditional random fields (CRFs), shows capability to extract and learn building features from imperfect pseudo-labels. By integrating the SAM-based pseudo-label with our edge-driven model, we establish an unsupervised framework for building segmentation that operates without explicit labels. Our model excels in extracting buildings compared with other state-of-the-art unsupervised segmentation models and even outperforms supervised models when trained in a fully supervised manner. This achievement demonstrates the potential of our model to address the lack of datasets in various remote sensing domains for building segmentation.

Parameter probabilistic prediction for satellite power system based on unsupervised multi-dimensional sequence segmentation

With the increasing operational autonomy and mission complexity of satellites, the power system, as one of the most critical subsystems in a satellite platform, needs to provide high-quality and fast-response power output under complex load requirements. State monitoring is valuable to ensure the power system's stable and safe operation. Monitoring and predicting the key parameters based on the satellite power system's telemetry data are meaningful in detecting potential anomalies. However, the multi-physical high-dimensional telemetry data of the satellite power system are oriented to the operating conditions. The dynamic operating conditions of the satellite power system make it difficult to establish the proper parameter prediction model with high adaptability. To address these challenging issues, this paper proposes a parameter probabilistic prediction based on unsupervised time series segmentation. Firstly, the telemetry data under dynamic operation conditions are divided based on the unsupervised dynamic factor model and K-Means algorithm. The dynamic factor model is used to extract the common factor sequences of the multidimensional telemetry data according to the dynamic relationship between different parameters, followed by the clustering operation using K-Means to classify the multidimensional telemetry data into different sub-sequences. Secondly, a probabilistic prediction method based on the spatio-temporal fusion of multi-physical parameters is proposed. The method utilizes a Bayesian neural network to learn the spatial relationship between parameters and applies Gaussian process regression to establish the temporal dependence of parameters. Afterward, a specific strategy is utilized to fuse the prediction results of the two, thus realizing the fusion of spatio-temporal information of satellite multi-dimensional telemetry data. The experimental results based on the real meteorological satellite power system telemetry data in 2010 show that the average absolute errors of the key parameter predictions after the division of the working conditions are all less than 0.25, which is improved by more than 37.08 % compared with the pre-division. This indicates that the proposed method can well recognize the dynamic operation modes and has a positive advancing effect on the parameter prediction task. Meanwhile, the average absolute error of the proposed prediction method is improved by at least 40.4 % compared with other single prediction models. It shows that the method proposed in this paper can realize high-precision prediction of parameters, and has good application prospects in condition monitoring and anomaly detection of satellite power systems.

An evolutionary vehicle scanning method for bridges based on time series segmentation and change point detection

In this paper, the problem of bridge health monitoring is formulated as an unsupervised semantic segmentation where the aim is to identify the onset of structural damage by detecting unexpected irregularities in the time series data. In particular, the main objective of this study is to approach this problem through an indirect sensing framework based on the concept of vehicle-mounted scanning. Thus, the sole source of information is the acceleration responses of the moving vehicle passing over the bridge. This is achieved through automatic feature learning by adopting the concept of matrix profile to conduct an all-pair-similarity search for a given time series, where the location of the contextual change is determined by the corresponding Corrected Arc Crossings (CAC) profile. The performance of the method is first evaluated through numerical investigations for different damage cases where the effect of operational uncertainties such as road roughness, and variability in vehicle properties are considered. Further, the method is experimentally evaluated using a lab-scale bridge model with different damage cases. Through these investigations, it is consistently observed that the location of the change point always coincides with the minimum point of the CAC profile. The proposed framework has three major advantages making it more appealing compared to the competing techniques. First, it has been fully formulated in the context of unsupervised learning where no access to labeled data is required. Second, it is capable of working with a limited amount of data yet providing promising results. Finally, it can be easily applied to online settings where data is streaming in real-time. These successful results demonstrate the potential of the proposed method as a cost-effective and rigorous bridge inspection tool and open up opportunities in the field of indirect bridge health monitoring.

Unsupervised PCB Anomaly Segmentation with Foundational Models

Unsupervised PCB Anomaly Segmentation with Foundational Models

Evaluating the Efficacy of Segment Anything Model for Delineating Agriculture and Urban Green Spaces in Multiresolution Aerial and Spaceborne Remote Sensing Images

Segmentation of Agricultural Remote Sensing Images (ARSIs) stands as a pivotal component within the intelligent development path of agricultural information technology. Similarly, quick and effective delineation of urban green spaces (UGSs) in high-resolution images is also increasingly needed as input in various urban simulation models. Numerous segmentation algorithms exist for ARSIs and UGSs; however, a model with exceptional generalization capabilities and accuracy remains elusive. Notably, the newly released Segment Anything Model (SAM) by META AI is gaining significant recognition in various domains for segmenting conventional images, yielding commendable results. Nevertheless, SAM’s application in ARSI and UGS segmentation has been relatively limited. ARSIs and UGSs exhibit distinct image characteristics, such as prominent boundaries, larger frame sizes, and extensive data types and volumes. Presently, there is a dearth of research on how SAM can effectively handle various ARSI and UGS image types and deliver superior segmentation outcomes. Thus, as a novel attempt in this paper, we aim to evaluate SAM’s compatibility with a wide array of ARSI and UGS image types. The data acquisition platform comprises both aerial and spaceborne sensors, and the study sites encompass most regions of the United States, with images of varying resolutions and frame sizes. It is noteworthy that the segmentation effect of SAM is significantly influenced by the content of the image, as well as the stability and accuracy across images of different resolutions and sizes. However, in general, our findings indicate that resolution has a minimal impact on the effectiveness of conditional SAM-based segmentation, maintaining an overall segmentation accuracy above 90%. In contrast, the unsupervised segmentation approach, SAM, exhibits performance issues, with around 55% of images (3 m and coarser resolutions) experiencing lower accuracy on low-resolution images. Whereas frame size exerts a more substantial influence, as the image size increases, the accuracy of unsupervised segmentation methods decreases extremely fast, and conditional segmentation methods also show some degree of degradation. Additionally, SAM’s segmentation efficacy diminishes considerably in the case of images featuring unclear edges and minimal color distinctions. Consequently, we propose enhancing SAM’s capabilities by augmenting the training dataset and fine-tuning hyperparameters to align with the demands of ARSI and UGS image segmentation. Leveraging the multispectral nature and extensive data volumes of remote sensing images, the secondary development of SAM can harness its formidable segmentation potential to elevate the overall standard of ARSI and UGS image segmentation.

Unsupervised unpaired multiple fusion adaptation aided with self-attention generative adversarial network for scar tissues segmentation framework

Late gadolinium enhancement (LGE) is a specialized imaging technique used in cardiovascular magnetic resonance (CMR) imaging to detect and characterize areas of scar tissue or fibrosis within the heart muscle. After a heart attack, the affected region of the heart muscle becomes scarred due to insufficient blood supply. LGE CMR enhances the infarcted myocardium to appear with distinctive brightness compared to healthy tissues. It is primarily used to assess various heart conditions and can identify the extent and location of scar tissue, helping in risk stratification and guiding treatment decisions. Automated myocardium segmentation in LGE CMR poses challenges due to intensity heterogeneity caused by the accumulation of contrast agents in infarcted areas. Furthermore, LGE CMR images with gold standard labels are particularly scarce compared to other sequences, making the task even more demanding. Besides, the heart’s motion involves complex non-rigid deformations, including regional wall thickening, circumferential, and longitudinal ventricular shortening, making it challenging to model. This work presents an unsupervised unpaired domain adaptive scar tissue segmentation framework aided with the self-attention generative adversarial network. Besides, we proposed a novel segmentation architecture based on a parallel transformer for scar segmentation and compared the results with and without style transfer GAN and unpaired style self-supervised attention GAN. Besides, to model longitudinal changes in scare, we used eight days’ time points as a source and 1 month and 12 as a target and applied our proposed attention-generative network to transfer style from one-time point to another time. Experiments on four different LGE MICCAI datasets from 2019 to 2023 achieved significantly better performance than state-of-the-art segmentation methods.

Unsupervised corneal contour extraction algorithm with shared model for dynamic deformation videos: improving accuracy and noise resistance

BackgroundIn this study, an automatic corneal contour extraction algorithm with a shared model is developed to extract contours from dynamic corneal videos containing noise, which improves the accuracy of corneal biomechanical evaluation and clinical diagnoses. The algorithm does not require manual labeling and completes the unsupervised semantic segmentation of each frame in corneal dynamic deformation videos based on a fully convolutional deep-learning network using corneal geometry and texture information.ResultsWe included 1027 corneal videos at Tianjin Eye Hospital (Nankai University Affiliated Eye Hospital) from May 2020 to November 2021. The videos were obtained by the ultra-high-speed Scheimpflug camera, and then we used the shared model mechanism to accelerate the segmentation of corneal regions in videos, effectively resist noise, determine corneal regions based on shape factors, and finally achieve automatic and accurate extraction of corneal region contours. The Intersection over Union (IoU) of the extracted and real corneal contours using this algorithm reached 95%, and the average overlap error was 0.05, implying that the extracted corneal contour overlapped almost completely with the real contour.ConclusionsCompared to other algorithms, the method introduced in this study does not require manual annotation of corneal contour data in advance and can still extract accurate corneal contours from noisy corneal videos with good repeatability.

Benchmarking and Analysis of Unsupervised Object Segmentation from Real-World Single Images

In this paper, we study the problem of unsupervised object segmentation from single images. We do not introduce a new algorithm, but systematically investigate the effectiveness of existing unsupervised models on challenging real-world images. We first introduce seven complexity factors to quantitatively measure the distributions of background and foreground object biases in appearance and geometry for datasets with human annotations. With the aid of these factors, we empirically find that, not surprisingly, existing unsupervised models fail to segment generic objects in real-world images, although they can easily achieve excellent performance on numerous simple synthetic datasets, due to the vast gap in objectness biases between synthetic and real images. By conducting extensive experiments on multiple groups of ablated real-world datasets, we ultimately find that the key factors underlying the failure of existing unsupervised models on real-world images are the challenging distributions of background and foreground object biases in appearance and geometry. Because of this, the inductive biases introduced in existing unsupervised models can hardly capture the diverse object distributions. Our research results suggest that future work should exploit more explicit objectness biases in the network design.

Contrastive Registration for Unsupervised Medical Image Segmentation.

Medical image segmentation is an important task in medical imaging, as it serves as the first step for clinical diagnosis and treatment planning. While major success has been reported using deep learning supervised techniques, they assume a large and well-representative labeled set. This is a strong assumption in the medical domain where annotations are expensive, time-consuming, and inherent to human bias. To address this problem, unsupervised segmentation techniques have been proposed in the literature. Yet, none of the existing unsupervised segmentation techniques reach accuracies that come even near to the state-of-the-art of supervised segmentation methods. In this work, we present a novel optimization model framed in a new convolutional neural network (CNN)-based contrastive registration architecture for unsupervised medical image segmentation called CLMorph. The core idea of our approach is to exploit image-level registration and feature-level contrastive learning, to perform registration-based segmentation. First, we propose an architecture to capture the image-to-image transformation mapping via registration for unsupervised medical image segmentation. Second, we embed a contrastive learning mechanism in the registration architecture to enhance the discriminative capacity of the network at the feature level. We show that our proposed CLMorph technique mitigates the major drawbacks of existing unsupervised techniques. We demonstrate, through numerical and visual experiments, that our technique substantially outperforms the current state-of-the-art unsupervised segmentation methods on two major medical image datasets.

Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation.

In this article, we address the challenges in unsupervised video object segmentation (UVOS) by proposing an efficient algorithm, termed MTNet, which concurrently exploits motion and temporal cues. Unlike previous methods that focus solely on integrating appearance with motion or on modeling temporal relations, our method combines both aspects by integrating them within a unified framework. MTNet is devised by effectively merging appearance and motion features during the feature extraction process within encoders, promoting a more complementary representation. To capture the intricate long-range contextual dynamics and information embedded within videos, a temporal transformer module is introduced, facilitating efficacious interframe interactions throughout a video clip. Furthermore, we employ a cascade of decoders all feature levels across all feature levels to optimally exploit the derived features, aiming to generate increasingly precise segmentation masks. As a result, MTNet provides a strong and compact framework that explores both temporal and cross-modality knowledge to robustly localize and track the primary object accurately in various challenging scenarios efficiently. Extensive experiments across diverse benchmarks conclusively show that our method not only attains state-of-the-art performance in UVOS but also delivers competitive results in video salient object detection (VSOD). These findings highlight the method's robust versatility and its adeptness in adapting to a range of segmentation tasks. The source code is available at https://github.com/hy0523/MTNet.

LF-SynthSeg: Label-Free Brain Tissue-Assisted Tumor Synthesis and Segmentation.

Unsupervised brain tumor segmentation is pivotal in realms of disease diagnosis, surgical planning, and treatment response monitoring, with the distinct advantage of obviating the need for labeled data. Traditional methodologies in this domain, however, often fall short in fully capitalizing on the extensive prior knowledge of brain tissue, typically approaching the task merely as an anomaly detection challenge. In our research, we present an innovative strategy that effectively integrates brain tissues' prior knowledge into both the synthesis and segmentation of brain tumor from T2-weighted Magnetic Resonance Imaging scans. Central to our method is the tumor synthesis mechanism, employing randomly generated ellipsoids in conjunction with the intensity profiles of brain tissues. This methodology not only fosters a significant degree of variation in the tumor presentations within the synthesized images but also facilitates the creation of an essentially unlimited pool of abnormal T2-weighted images. These synthetic images closely replicate the characteristics of real tumor-bearing scans. Our training protocol extends beyond mere tumor segmentation; it also encompasses the segmentation of brain tissues, thereby directing the networkâs attention to the boundary relationship between brain tumor and brain tissue, thus improving the robustness of our method. We evaluate our approach across five widely recognized public datasets (BRATS 2019, BRATS 2020, BRATS 2021, PED and SSA), and the results show that our method outperforms state-of-the-art unsupervised tumor segmentation methods by large margins. Moreover, the proposed method achieves more than 92 % of the fully supervised performance on the same testing datasets.

Trajectory Time Series Compression Algorithm Based on Unsupervised Segmentation

Trajectory Time Series Compression Algorithm Based on Unsupervised Segmentation

Segmentation and Completion of Human Motion Sequence via Temporal Learning of Subspace Variety Model.

Subspace-based models have been extensively employed in unsupervised segmentation and completion of human motion sequence (HMS). However, existing approaches often neglect the incorporation of temporal priors embedded in HMS, resulting in suboptimal results. This paper presents a subspace variety model for HMS, along with an innovative Temporal Learning of Subspace Variety Model (TL-SVM) method for enhanced segmentation and completion in HMS. The key idea is to segment incomplete HMS into motion clusters and extracting the subspace features of each motion through the temporal learning of the subspace variety model. Subsequently, the HMS is completed based on the extracted subspace features. Thus, the main challenge is to learn the subspace variety model with temporal priors when confronted with missing entries. To tackle this, the paper develops a spatio-temporal assignment consistency (STAC) constraint for the subspace variety model, leveraging temporal priors embedded in HMS. In addition, a subspace clustering approach under the STAC constraint is proposed to learn the subspace variety model by extracting subspace features from HMS and segmenting HMS into motion clusters alternatively. The proposed subspace clustering model can also handle missing entries with theoretical guarantees. Furthermore, the missing entries of HMS are completed by minimizing the distance between each human motion frame and its corresponding subspace. Extensive experimental results, along with comparisons to state-of-the-art methods on four benchmark datasets, underscore the advantages of the proposed method.

Unsupervised Log Sequence Segmentation

Unsupervised Log Sequence Segmentation

Fine-Grained Self-Supervision for Generalizable Semantic Segmentation

Unsupervised domain adaptative semantic segmentation is a powerful solution for the distribution shift problem between the source and target domains. However, such methods need specified target domain data that may be unavailable in actual applications due to excess expensive collection. Generalizable semantic segmentation as a new paradigm appears in recent research, which aims to generalize well on distinct unseen domains only using source domain data. The existing methods focus on learning domain-invariant features by using global distribution alignment strategies, which may lead to a decreased discriminability of the model. To cope with this challenge, we propose a fine-grained self-supervision (FGSS) framework for generalizable semantic segmentation that takes into account both discriminability and generalizability from the perspective of the intra-class relationship. The FGSS framework contains single-view and multi-view versions. In the single-view version, we propose a fine-grained self-supervision strategy to distinguish the sub-parts of the semantic class for better class discriminability. In the multi-view version, we propose a class prototype feature enhancement strategy to generate another view (i.e. another representation of the original representation). Then, we propose a multi-view mutual supervision loss to enforce consistency between different views and further enhance the generalizability of the model. Experimental results on five widely-used datasets, i.e., GTAV, SYNTHIA, BDD100K, Cityscapes, and Mapillary, demonstrate that our FGSS framework achieves superior performance compared to state-of-the-art methods.

Unsupervised Time Series Segmentation: A Survey on Recent Advances

Unsupervised Time Series Segmentation: A Survey on Recent Advances

Cross-Domain Scene Unsupervised Learning Segmentation With Dynamic Subdomains

Cross-Domain Scene Unsupervised Learning Segmentation With Dynamic Subdomains

Causal Prototype-Inspired Contrast Adaptation for Unsupervised Domain Adaptive Semantic Segmentation of High-Resolution Remote Sensing Imagery

Causal Prototype-Inspired Contrast Adaptation for Unsupervised Domain Adaptive Semantic Segmentation of High-Resolution Remote Sensing Imagery

URS: An Unsupervised Radargram Segmentation Network based on Self-supervised ViT with Contrastive Feature Learning Framework

URS: An Unsupervised Radargram Segmentation Network based on Self-supervised ViT with Contrastive Feature Learning Framework

Crop Identification of UAV Images Based on an Unsupervised Semantic Segmentation Method

Crop Identification of UAV Images Based on an Unsupervised Semantic Segmentation Method