Unsupervised Segmentation Research Articles

A new commercial boundary dataset for metropolitan areas in the USA and Canada, built from open data

The purpose of this study is to define the geographic boundaries of commercial areas by creating a consistent definition, combining various commercial area types, including downtowns, retail centres, financial districts, and other employment subcentres. Our research involved the collection of office, retail and job density data from 69 metropolitan regions across USA and Canada. Using this data, we conducted an unsupervised image segmentation model and clustering methods to identify distinctive commercial geographic boundaries. As a result, we identified 23,751 commercial areas, providing a detailed perspective on the commercial landscape of metropolitan areas in the USA and Canada. In addition, the generated boundaries were successfully validated through comparison with previously established commerce-related boundaries. The output of this study has implications for urban and regional planning and economic development, delivering valuable insights into the overall commercial geography in the region. The commercial boundary and used codes are freely available on the School of Cities Github, and users can reuse, reproduce and modify the boundaries.

Joint learning framework of superpixel generation and fuzzy sparse subspace clustering for color image segmentation

Sparse subspace clustering (SSC) is an important image segmentation method that constructs a self-representation coefficient matrix to represent the relationships between pixels, and then uses spectral clustering to achieve clustering. Compared with other unsupervised segmentation algorithms, SSC has good segmentation performance. However, SSC has a high computational complexity when processing large-sized image. To improve computational efficiency, many researchers have proposed superpixel-based SSC algorithms, which process superpixels instead of pixels and improve efficiency through preprocessing. Due to the sensitivity of superpixel generation to noise, superpixel-based SSC algorithms still have poor robustness. Additionally, preprocessing increases the complexity of the algorithm. To address these issues, this paper proposes a robust superpixel-based fuzzy sparse subspace clustering algorithm. This algorithm combines fuzzy sparse subspace clustering with superpixel generation, and it constructs a unified optimization learning framework through fuzzy C-multiple-means clustering to improve segmentation performance and reduce complexity. Additionally, this paper introduces additional features of superpixel in sparse subspace clustering to further enhance the segmentation performance of the algorithm. Experiment results indicate that the proposed algorithm not only outperforms existing state-of-the-art robust segmentation algorithms independent of superpixels, but also is superior to the latest superpixel-based segmentation algorithms.

An unsupervised semantic segmentation method that combines the ImSE-Net model with SLICm superpixel optimization

ABSTRACT In the field of remote sensing, using a large amount of labeled image data to supervise the training of fully convolutional networks for the semantic segmentation of images is expensive. However, using a small amount of labeled data can lead to reduced network performance. This paper proposes an unsupervised semantic segmentation method that combines the ImSE-Net model with SLICm superpixel optimization. First, the ImSE-Net model is used to extract semantic features from the image to obtain rough semantic segmentation results. Then, the SLICm superpixel segmentation algorithm is used to segment the input image into superpixel images. Finally, an unsupervised semantic segmentation model (UGLS) is used to combine high-level abstract semantic features with detailed information on superpixels to obtain edge-optimized semantic segmentation results. Experimental results show that compared with other semantic segmentation algorithms, our method more effectively handles unbalanced areas, such as object boundaries, and achieves better segmentation results, with higher semantic consistency.

Differentiable double clustering with edge-aware superpixel fitting for unsupervised image segmentation

Unsupervised image segmentation is an essential topic in the field of computer vision, which broke the limitation of training data and expanded application scenarios. Off-the-shelf clustering methods simply rely on semantic concepts and incomplete boundary cues, resulting in incorrect segmentation in object boundaries. Therefore, this paper proposes an unsupervised image segmentation framework combining differentiable double clustering (DDC) and edge-aware superpixel (EA), which outperform prior work on the accuracy of the prior art. First, a multi-layer feature extraction network is introduced to combine low-level and high-level features for clustering. Then, the DDC module is designed to obtain initial labels of pixels from both local and global perspectives to improve the clustering accuracy. Pixel-wise feature similarity in different classes is pushed away, and one in the same class is brought closer. Finally, we use EA to provide well-fitting boundary cues for DDC label fusion to reduce incorrect segmentation. Extensive experiments on the benchmark datasets PASCAL VOC2012 [34] and BSD500 demonstrate that the proposed method provides competitive segmentation results.

Unsupervised cross domain semantic segmentation with mutual refinement and information distillation

Unsupervised cross domain semantic segmentation recently has gained much attention, due to its powerful ability of solving the segmentation problem on unlabeled domains. Traditional methods often employ an adversarial network to confuse the source and target inputs, so as to align them in a new feature space. However, these methods cannot well fuse the source and target domain information because the information of the two domains does not really interact with each other and are passed through separate network branches. To tackle this problem, we propose a real interactive learning framework, named Mutual Refinement and Information Distillation (MURID), to align the two domains. Concretely, MURID introduces a Mutual Refinement module in shallow network layers to enhance information sharing and integration between the source and target domains, which can effectively transfer knowledge from source domain to target domain. In addition, in order to avoid using the same structure for testing as for training, which would result in huge computational requirements, we exploit an Information Distillation module to simplify the testing network while maintaining the powerful inference capability of the training. Moreover, we incorporate Curriculum Learning, a self-training mechanism that iteratively trains the network using pseudo-labels obtained from the target domain, to further improve performance. Extensive experiments were conducted on three popular datasets, i.e., GTA5→Cityscapes and Synthia→Cityscapes, and the results demonstrate the state-of-the-art performance of our method. Additionally, detailed analysis and ablation studies are also carried out to validate the effectiveness of each designed module.

Automatic Marker Generation to Similarity Based Region Merging Algorithm using Edge Information

One of the most important tasks in computer vision is image segmentation. Several interactive strategies are utilized for picture segmentation because automated techniques are difficult for this kind of work. The outcome of interactive methods is mostly determined by user input. Obtaining excellent interactions for huge datasets is challenging. However, automated picture segmentation is starting to play a significant role in image analysis and computer vision. Effective and efficient segmentation outcomes are obtained using the interacting region merging method suggested by Maximal Similarity based region merging algorithm. Limitation of MSBRM is it requires some efforts on the part of users and is yet not a fully-automatic approach. Here, we suggest a completely fresh unsupervised image segmentation method that combines edge information with MSBRM. We propose an integrated framework to generate object markers for similarity based region algorithm using edge information. Long edges give rough distribution of objects in image. After retrieving edges using phase congruency, edge processing operations are employed to remove small edges and to group color similar long boundaries. Centroids of long boundaries are used as object markers to the MSBRM algorithm. The generation of object markers is done using edge segment grouping. These object markers guide the region merging process. The proposed method shows its effectiveness in segmenting natural real world color images.

Exploration of Scientific Documents through Unsupervised Learning-Based Segmentation Techniques

Navigating the extensive landscape of scientific literature presents a significant challenge, prompting the development of innovative methodologies for efficient exploration. Our study introduces a pioneering approach for unsupervised segmentation, aimed at revealing thematic trends within articles and enhancing the accessibility of scientific knowledge. Leveraging three prominent clustering algorithms—K-Means, Hierarchical Agglomerative, and DBSCAN—we demonstrate their proficiency in generating meaningful clusters, validated through assessment metrics including Silhouette Score, Calinski-Harabasz Index, and Davies-Bouldin Index. Methodologically, comprehensive web scraping of scientific databases, coupled with thorough data cleaning and preprocessing, forms the foundation of our approach. The efficacy of our methodology in accurately identifying scientific domains and uncovering interdisciplinary connections underscores its potential to revolutionize the exploration of scientific publications. Future endeavors will further explore alternative unsupervised algorithms and extend the methodology to diverse data sources, fostering continuous innovation in scientific knowledge organization.

Expand-and-Quantize: Unsupervised Semantic Segmentation Using High-Dimensional Space and Product Quantization

Unsupervised semantic segmentation (USS) aims to discover and recognize meaningful categories without any labels. For a successful USS, two key abilities are required: 1) information compression and 2) clustering capability. Previous methods have relied on feature dimension reduction for information compression, however, this approach may hinder the process of clustering. In this paper, we propose a novel USS framework called Expand-and-Quantize Unsupervised Semantic Segmentation (EQUSS), which combines the benefits of high-dimensional spaces for better clustering and product quantization for effective information compression. Our extensive experiments demonstrate that EQUSS achieves state-of-the-art results on three standard benchmarks. In addition, we analyze the entropy of USS features, which is the first step towards understanding USS from the perspective of information theory.

Unsupervised Continual Anomaly Detection with Contrastively-Learned Prompt

Unsupervised Anomaly Detection (UAD) with incremental training is crucial in industrial manufacturing, as unpredictable defects make obtaining sufficient labeled data infeasible. However, continual learning methods primarily rely on supervised annotations, while the application in UAD is limited due to the absence of supervision. Current UAD methods train separate models for different classes sequentially, leading to catastrophic forgetting and a heavy computational burden. To address this issue, we introduce a novel Unsupervised Continual Anomaly Detection framework called UCAD, which equips the UAD with continual learning capability through contrastively-learned prompts. In the proposed UCAD, we design a Continual Prompting Module (CPM) by utilizing a concise key-prompt-knowledge memory bank to guide task-invariant 'anomaly' model predictions using task-specific 'normal' knowledge. Moreover, Structure-based Contrastive Learning (SCL) is designed with the Segment Anything Model (SAM) to improve prompt learning and anomaly segmentation results. Specifically, by treating SAM's masks as structure, we draw features within the same mask closer and push others apart for general feature representations. We conduct comprehensive experiments and set the benchmark on unsupervised continual anomaly detection and segmentation, demonstrating that our method is significantly better than anomaly detection methods, even with rehearsal training. The code will be available at https://github.com/shirowalker/UCAD.

Generalizable Fourier Augmentation for Unsupervised Video Object Segmentation

The performance of existing unsupervised video object segmentation methods typically suffers from severe performance degradation on test videos when tested in out-of-distribution scenarios. The primary reason is that the test data in real- world may not follow the independent and identically distribution (i.i.d.) assumption, leading to domain shift. In this paper, we propose a generalizable fourier augmentation method during training to improve the generalization ability of the model. To achieve this, we perform Fast Fourier Transform (FFT) over the intermediate spatial domain features in each layer to yield corresponding frequency representations, including amplitude components (encoding scene-aware styles such as texture, color, contrast of the scene) and phase components (encoding rich semantics). We produce a variety of style features via Gaussian sampling to augment the training data, thereby improving the generalization capability of the model. To further improve the cross-domain generalization performance of the model, we design a phase feature update strategy via exponential moving average using phase features from past frames in an online update manner, which could help the model to learn cross-domain-invariant features. Extensive experiments show that our proposed method achieves the state-of-the-art performance on popular benchmarks.

Few Shot Part Segmentation Reveals Compositional Logic for Industrial Anomaly Detection

Logical anomalies (LA) refer to data violating underlying logical constraints e.g., the quantity, arrangement, or composition of components within an image. Detecting accurately such anomalies requires models to reason about various component types through segmentation. However, curation of pixel-level annotations for semantic segmentation is both time-consuming and expensive. Although there are some prior few-shot or unsupervised co-part segmentation algorithms, they often fail on images with industrial object. These images have components with similar textures and shapes, and a precise differentiation proves challenging. In this study, we introduce a novel component segmentation model for LA detection that leverages a few labeled samples and unlabeled images sharing logical constraints. To ensure consistent segmentation across unlabeled images, we employ a histogram matching loss in conjunction with an entropy loss. As segmentation predictions play a crucial role, we propose to enhance both local and global sample validity detection by capturing key aspects from visual semantics via three memory banks: class histograms, component composition embeddings and patch-level representations. For effective LA detection, we propose an adaptive scaling strategy to standardize anomaly scores from different memory banks in inference. Extensive experiments on the public benchmark MVTec LOCO AD reveal our method achieves 98.1% AUROC in LA detection vs. 89.6% from competing methods.

Abstract 4406: Single cell multiomic analysis reveals how genomic, epigenomic and transcriptomic events drive progression of multiple myeloma precursor states

Abstract In this study we have leveraged Moffitt Cancer Center’s cohort of over 1,000 bone marrow (BMBX) aspirates from normal donors, MM precursor diseases (MGUS, SMOL), active therapy naïve disease (NDMM), as well as early (post 1-3 lines of therapy, ERMM) and late-relapse (3+ lines of therapy, LRMM) MM patients, characterized clinically, cytogenetically (FISH), genetically (WES), and transcriptomically (RNA-seq).Transcriptomic analysis identified a subset of cancer hallmarks differentially expressed between MGUS and NDMM samples (e.g. cell identity, adhesion and inflammatory signaling, etc.).Dimensionality reduction analysis, followed by unsupervised segmentation, identified clusters of co-expressing genes across this dataset. Enrichment analysis of aforementioned gene clusters, differentially expressed between MGUS and NDMM states, identified a number of putative transcription factors (GATA1/2, SUZ12, RUNX1, SOX2, TP63, NFE2L2, FOXM1 and NANOG), as well as polycomb complex and H3K27me3 as main drivers. Single cell ATAC-seq in 11 BMBX (4 SMOL, 4 NDMM, 1 ERMM, 2 LRMM) and single cell multiomics (simultaneous scATAC-seq/scRNA-seq) in additional 10 BMBX (2 healthy, 3 MGUS, 3 SMOL and 2 NDMM) confirmed that the H3K27me3 putatively regulated gene clusters were the only ones with significant chromatin accessibility changes. Single cell analysis confirmed differential expression and accessibility of cancer hallmarks identified in larger cohort, as well as putative driver TFs. Correlation between TF expression and motif accessibility produced an additional list of pioneer TFs putatively involved in progression from MM precursor states (ZEB1, EBF1, SPI1, PAX5, etc.).Investigation of scMultiomic data demonstrated that NDMM and SMOL samples contain subpopulations of cells in MGUS and normal “states” (cell clusters in scMultiomics UMAP), confirming intra-sample heterogeneity, and that divergence within the multiomic plane occurs in the transition between MGUS and SMOL. Through pseudotime and single cell copy number analysis (epiAneufinder/scATAC-seq and inferCNV/scRNA-seq) we were able to reconstruct how MM FISH cytogenetic abnormalities alter the multiome of these cells, creating aforementioned intra-sample heterogeneity. Despite different initiating events, all samples shared a common biology putatively regulated by epigenetic dysregulation, suggesting polycomb complex hijacking as a therapeutic opportunity for precursor MM. Citation Format: Rafael Canevarolo, Praneeth Sudalagunta, Maria Silva, Mark Meads, Rachid Baz, Melissa Alsina, Kenneth H. Shain, Ariosto Siqueira Silva. Single cell multiomic analysis reveals how genomic, epigenomic and transcriptomic events drive progression of multiple myeloma precursor states [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4406.

Key point generation as an instrument for generating core statements of a political debate on Twitter.

Identifying key statements in large volumes of short, user-generated texts is essential for decision-makers to quickly grasp their key content. To address this need, this research introduces a novel abstractive key point generation (KPG) approach applicable to unlabeled text corpora, using an unsupervised approach, a feature not yet seen in existing abstractive KPG methods. The proposed method uniquely combines topic modeling for unsupervised data space segmentation with abstractive summarization techniques to efficiently generate semantically representative key points from text collections. This is further enhanced by hyperparameter tuning to optimize both the topic modeling and abstractive summarization processes. The hyperparameter tuning of the topic modeling aims at making the cluster assignment more deterministic as the probabilistic nature of the process would otherwise lead to high variability in the output. The abstractive summarization process is optimized using a Davies-Bouldin Index specifically adapted to this use case, so that the generated key points more accurately reflect the characteristic properties of this cluster. In addition, our research recommends an automated evaluation that provides a quantitative complement to the traditional qualitative analysis of KPG. This method regards KPG as a specialized form of Multidocument summarization (MDS) and employs both word-based and word-embedding-based metrics for evaluation. These criteria allow for a comprehensive and nuanced analysis of the KPG output. Demonstrated through application to a political debate on Twitter, the versatility of this approach extends to various domains, such as product review analysis and survey evaluation. This research not only paves the way for innovative development in abstractive KPG methods but also sets a benchmark for their evaluation.

Unsupervised Color-Based Nuclei Segmentation in Histopathology Images with Various Color Spaces and K Values Selection

The development of digital pathology offers a significant opportunity to evaluate and analyze the whole slides of disease tissue effectively. In particular, the segmentation of nuclei from histopathology images plays an important role in quantitatively measuring and evaluating the acquired diseased tissue. There are many automatic methods to segment cell nuclei in histopathology images. One widely used unsupervised segmentation approach is based on standard k-means or fuzzy c-means (FCM) to process the color histopathology images to segment cell nuclei. Compared with the supervised learning method, this approach can obtain segmented nuclei without annotated nuclei labels for training, which saves a lot of labeling and training time. The color space and [Formula: see text] value among this method plays a crucial role in determining the nuclei segmentation performance. However, few works have investigated various color spaces and [Formula: see text] value selection simultaneously in unsupervised color-based nuclei segmentation with [Formula: see text]-means or FCM algorithms. In this study, we will present color-based nuclei segmentation methods with standard [Formula: see text]-means and FCM algorithms for histopathology images. Several color spaces of Haematoxylin and Eosin (H&E) stained histopathology data and various [Formula: see text] values among [Formula: see text]-means and FCM are investigated correspondingly to explore the suitable selection for nuclei segmentation. A comprehensive nuclei dataset with 7 different organs is used to validate our proposed method. Related experimental results indicate that [Formula: see text] and the YCbCr color spaces with a [Formula: see text] of 4 are more reasonable for nuclei segmentation via [Formula: see text]-means, while the [Formula: see text] color space with [Formula: see text] of 4 is useful via FCM.

PKU-GoodsAD: A Supermarket Goods Dataset for Unsupervised Anomaly Detection and Segmentation

PKU-GoodsAD: A Supermarket Goods Dataset for Unsupervised Anomaly Detection and Segmentation

Two-stage adversarial learning based unsupervised domain adaptation for retinal OCT segmentation.

Deep learning based optical coherence tomography (OCT) segmentation methods have achieved excellent results, allowing quantitative analysis of large-scale data. However, OCT images are often acquired by different devices or under different imaging protocols, which leads to serious domain shift problem. This in turn results in performance degradation of segmentation models. Aiming at the domain shift problem, we propose a two-stage adversarial learning based network (TSANet) that accomplishes unsupervised cross-domain OCT segmentation. In the first stage, a Fourier transform based approach is adopted to reduce image style differences from the image level. Then, adversarial learning networks, including a segmenter and a discriminator, are designed to achieve inter-domain consistency in the segmentation output. In the second stage, pseudo labels of selected unlabeled target domain training data are used to fine-tune the segmenter, which further improves its generalization capability. The proposed method was tested on cross-domain datasets for choroid or retinoschisis segmentation tasks. For choroid segmentation, the model was trained on 400 images and validated on 100 images from the source domain, and then trained on 1320 unlabeled images and tested on 330 images from target domain I, and also trained on 400 unlabeled images and tested on 200 images from target domain II. For retinoschisis segmentation, the model was trained on 1284 images and validated on 312 images from the source domain, and then trained on 1024 unlabeled images and tested on 200 images from the target domain. The proposed method achieved significantly improved results over that without domain adaptation, with improvement of 8.34%, 55.82% and 3.53% in intersection over union (IoU) respectively for the three test sets. The performance is better than some state-of-the-art domain adaptation methods. The proposed TSANet, with image level adaptation, feature level adaptation and pseudo-label based fine-tuning, achieved excellent cross-domain generalization. This alleviates the burden of obtaining additional manual labels when adapting the deep learning model to new OCT data.

Nonparametric K-means clustering-based adaptive unsupervised colour image segmentation

Nonparametric K-means clustering-based adaptive unsupervised colour image segmentation

Ensemble CART surrogate-assisted automatic multi-objective rough fuzzy clustering algorithm for unsupervised image segmentation

Multi-objective clustering algorithms (MOCAs) are popular in unsupervised image segmentation due to their merit of meeting multiple segmentation requirements and the prospect of automatically estimating the number of clusters. However, most of them suffer from high time costs and are easily to be influenced by the uncertainty when handling real complex images. To address these issues, we propose an ensemble classification and regression tree (CART) surrogate-assisted automatic multi-objective rough fuzzy clustering (ECS-AMRFC) algorithm for unsupervised image segmentation. Firstly, a cluster medoid-based encoding scheme is employed to represent solutions with different number of clusters and meanwhile lessen the length of encoding. Then, we design an ensemble CART as the surrogate model to significantly reduce the computational burden. Moreover, a surrogate model management strategy is proposed to accelerate the optimization and enhance the quality of surrogate modeling. To handle the uncertainty in data, we extend the rough fuzzy clustering into MOCAs and construct three complementary objective functions to seek proper cluster medoids from multiple perspectives. In addition, the Gaussian kernel is introduced into the objective functions to handle image pixels that cannot separate linearly in the feature space. Finally, a kernelized rough fuzzy clustering validity index is defined to automatically select the optimal solution with no requirements of any prior knowledge. Experiments show that ECS-AMRFC not only identifies appropriate number of clusters on different kinds of images, but also obtains better segmentation results than state-of-the-art rough fuzzy clustering algorithms and automatic MOCAs.

Learning to Generalize over Subpartitions for Heterogeneity-Aware Domain Adaptive Nuclei Segmentation

Annotation scarcity and cross-modality/stain data distribution shifts are two major obstacles hindering the application of deep learning models for nuclei analysis, which holds a broad spectrum of potential applications in digital pathology. Recently, unsupervised domain adaptation (UDA) methods have been proposed to mitigate the distributional gap between different imaging modalities for unsupervised nuclei segmentation in histopathology images. However, existing UDA methods are built upon the assumption that data distributions within each domain should be uniform. Based on the over-simplified supposition, they propose to align the histopathology target domain with the source domain integrally, neglecting severe intra-domain discrepancy over subpartitions incurred by mixed cancer types and sampling organs. In this paper, for the first time, we propose to explicitly consider the heterogeneity within the histopathology domain and introduce open compound domain adaptation (OCDA) to resolve the crux. In specific, a two-stage disentanglement framework is proposed to acquire domain-invariant feature representations at both image and instance levels. The holistic design addresses the limitations of existing OCDA approaches which struggle to capture instance-wise variations. Two regularization strategies are specifically devised herein to leverage the rich subpartition-specific characteristics in histopathology images and facilitate subdomain decomposition. Moreover, we propose a dual-branch nucleus shape and structure preserving module to prevent nucleus over-generation and deformation in the synthesized images. Experimental results on both cross-modality and cross-stain scenarios over a broad range of diverse datasets demonstrate the superiority of our method compared with state-of-the-art UDA and OCDA methods.Graphical abstract

Exploring Semantic Prompts in the Segment Anything Model for Domain Adaptation

Robust segmentation in adverse weather conditions is crucial for autonomous driving. However, these scenes struggle with recognition and make annotations expensive, resulting in poor performance. As a result, the Segment Anything Model (SAM) was recently proposed to finely segment the spatial structure of scenes and to provide powerful prior spatial information, thus showing great promise in resolving these problems. However, SAM cannot be applied directly for different geographic scales and non-semantic outputs. To address these issues, we propose SAM-EDA, which integrates SAM into an unsupervised domain adaptation mean-teacher segmentation framework. In this method, we use a “teacher-assistant” model to provide semantic pseudo-labels, which will fill in the holes in the fine spatial structure given by SAM and generate pseudo-labels close to the ground truth, which then guide the student model for learning. Here, the “teacher-assistant” model helps to distill knowledge. During testing, only the student model is used, thus greatly improving efficiency. We tested SAM-EDA on mainstream segmentation benchmarks in adverse weather conditions and obtained a more-robust segmentation model.