Unsupervised Segmentation Research Articles

S&GDA: An Unsupervised Domain Adaptive Semantic Segmentation Framework Considering Both Imaging Scene and Geometric Domain Shifts

Unsupervised domain adaptation uses labeled data from a source domain to help learn a target domain without any labeled data. Previous studies have not systematically analyzed the causes of remote sensing (RS) domain shifts, making it difficult to effectively model domain shifts caused by differences in geographic scene and platform imaging positions and attitudes. Therefore, this study conducts detailed analysis of the causes of domain shifts in RS images, and an unsupervised domain adaptive semantic segmentation framework, called S&GDA, that considers both imaging scene and geometric domain shifts is proposed. S&GDA comprised two modules: imaging scene simulation and imaging geometric simulation modules. The imaging scene simulation module is instrumental in mitigating domain shifts in geographical scenes due to variations in natural and human factors, thereby achieving cross-domain imaging scene consistency. Meanwhile, the imaging geometric simulation module allows for accurate simulation of domain shifts caused by changes in the position and attitude of a platform, ensuring cross-domain imaging geometry consistency. Note that none of these modules add additional parameters or computational complexity to the model as they only work on the input side of the data. Comprehensive experiments are conducted on the LoveDA and ISPRS datasets to evaluate S&GDA. Results indicate that S&GDA outperforms the state-of-the-art (SOTA) unsupervised domain adaptive semantic segmentation method by 3.12% of mIoU and can achieve 85% of the performance of the fully supervised method.

Wildfire aerial thermal image segmentation using unsupervised methods: a multilayer level set approach

Background and aims Infrared thermal images of a propagating wildfire taken by manned or unmanned aerial vehicles can help firefighting authorities with combat planning. Segmenting these images into regions of different fire temperatures is a necessary step to measure the fire perimeter and determine the location of the fire front. Methods This work proposes a multilayer segmentation method based on level sets, which have the property of handling topology, making them suitable to segment images that contain scattered fire areas. The experimental results were compared using hand-drawn labels over a set of images provided by the Portuguese Air Force as ground truth. These labels were carefully drawn by the authors to ensure that they complied with the requirements indicated by the Portuguese National Authority for Emergency and Civil Protection. The proposed method was optimised to ensure contour smoothness and reliability, as well as reduce computation time. Key results The proposed method can surpass other common unsupervised methods in terms of intersection over union, although it has not yet been able to perform real-time segmentation. Conclusions Although falling out of use in relation to supervised and deep learning methods, unsupervised segmentation can still be very useful when annotated datasets are unavailable.

A One-Stage Domain Adaptation Network With Image Alignment for Unsupervised Nighttime Semantic Segmentation.

In this paper, we tackle the problem of semantic segmentation for nighttime images that plays an equally important role as that for daytime images in autonomous driving, but is also much more challenging due to very poor illuminations and scarce annotated datasets. It can be treated as an unsupervised domain adaptation (UDA) problem, i.e., applying other labeled dataset taken in the daytime to guide the network training meanwhile reducing the domain shift, so that the trained model can generalize well to the desired domain of nighttime images. However, current general-purpose UDA approaches are insufficient to address the significant appearance difference between the day and night domains. To overcome such a large domain gap, we propose a novel domain adaptation network "DANIA" for nighttime semantic image segmentation by leveraging a labeled daytime dataset (the source domain) and an unlabeled dataset that contains coarsely aligned day-night image pairs (the target daytime and nighttime domains). These three domains are used to perform a multi-target adaptation via adversarial training in the network. Specifically, for the unlabeled day-night image pairs, we use the pixel-level predictions of static object categories on a daytime image as a pseudo supervision to segment its counterpart nighttime image. We also include a step of image alignment to relieve the inaccuracy caused by the misalignment between day-night image pairs by estimating a flow to refine the pseudo supervision produced by daytime images. Finally, a re-weighting strategy is applied to further improve the predictions, especially boosting the prediction accuracy of small objects. The proposed DANIA is a one-stage adaptation framework for nighttime semantic segmentation, which does not train additional day-night image transfer models as a separate pre-processing stage. Extensive experiments on Dark Zurich and Nighttime Driving datasets show that our DANIA achieves state-of-the-art performance for nighttime semantic segmentation.

An Unsupervised Snow Segmentation Approach Based on Dual-Polarized Scattering Mechanism and Deep Neural Network

Distribution of snow and its melting is a critical factor affecting local weather, avalanche and flood forecasting, livelihood of people residing, and hydropower production. Most of the existing dry and wet snow identification methods were based on expensive quad-pol SAR with finite generalizability, while dual-pol SAR with larger coverage, longer time series and open availability has more advantages. In this study, an unsupervised algorithm for dry and wet snow discrimination, NSAE-WFCM, is proposed based on a variety of polarimetric features derived from H-α decomposition in dual-pol mode using C-band Sentinel-1 SAR data. NSAE-WFCM constructs a deep training network using the pixel neighborhood-based sparse autoencoder (NSAE) to optimize polarimetric parameters, and inputs reconstructed features with different weights into feature-weighted fuzzy C-means clustering (WFCM) to distinguish dry and wet snow for each underlying surface. Ground observation was carried out during the snow melting period of March 2021 in Altay, China, to validate dual-pol NSAE-WFCM method with an overall accuracy and kappa coefficient of 88.8% and 0.68, respectively. The results show that NSAE-WFCM’s accuracy is similar to that of the quad-pol SAR-based dry and wet snow result (90.0%), and significantly better than that of previously published approaches extended to dual-pol SAR, such as SVM (76.7%), H-α-Wishart (65.5%), SPAN-based threshold method (51.7%), and wet snow-based method (43.1%). Therefore, the NSAE-WFCM algorithm improves the ability to classify wet and dry snow based on dual-pol polarimetric features, overcomes the high dependence of existing methods on quad-pol SAR data, and reduces manual interpretation by using unsupervised clustering.

An 3D MRI Unsupervised Graph-based Skull Stripping Algorithm

Brain tumours and strokes are increasingly common, even from a young age. For this reason preliminary screenings, such as MRI scans, are crucial steps in detecting the abnormalities and finding a treatment. Given the increasing number of people developing different brain problems, there is a need in developing automated segmentation systems. Regardless the various methods presented in the literature, the lack of ground truth images determines the researchers to move towards unsupervised techniques.This paper presents an modified graph-based unsupervised brain segmentation method that uses a minimal spanning tree to segment brain from non-brain tissue. The original method uses human intervention for node selection. The aim is to eliminate human intervention and reduce the computational time required for brain segmentation.The presented method was compared with the original one in two ways: by resizing and scaling the images, and using images without any pre-processing step. Experimental results were obtained on the NFBS dataset and in both experiments the proposed approach delivers better results, both visually and numerically. Performance measures have improved, leading to better overall segmentation. The new method increased the precision and dice coefficient by 20%, resulting in a more accurate segmentation.

Unsupervised Domain Adaptation Semantic Segmentation of High-Resolution Remote Sensing Imagery With Invariant Domain-Level Prototype Memory

Semantic segmentation is a key technique involved in automatic interpretation of high-resolution remote sensing (HRS) imagery and has drawn much attention in the remote sensing community. Deep convolutional neural networks (DCNNs) have been successfully applied to the HRS imagery semantic segmentation task due to their hierarchical representation ability. However, the heavy dependency on a large number of training data with dense annotation and the sensitiveness to the variation of data distribution severely restrict the potential application of DCNNs for the semantic segmentation of HRS imagery. This study proposes a novel unsupervised domain adaptation semantic segmentation network (MemoryAdaptNet) for the semantic segmentation of HRS imagery. MemoryAdaptNet constructs an output space adversarial learning scheme to bridge the domain distribution discrepancy between source domain and target domain and to narrow the influence of domain shift. Specifically, we embed an invariant feature memory module to store invariant domain-level context information because the features obtained from adversarial learning only tend to represent the variant feature of current limited inputs. This module is integrated by a category attention-driven invariant domain-level context aggregation module to current pseudo invariant feature for further augmenting the pixel representations. An entropy-based pseudo label filtering strategy is used to update the memory module with high-confident pseudo invariant feature of current target images. Extensive experiments under three cross-domain tasks indicate that our proposed MemoryAdaptNet is remarkably superior to the state-of-the-art methods.

Word Segmentation on Discovered Phone Units With Dynamic Programming and Self-Supervised Scoring

Recent work on unsupervised speech segmentation has used self-supervised models with phone and word segmentation modules that are trained jointly. This paper instead revisits an older approach to word segmentation: bottom-up phone-like unit discovery is performed first, and symbolic word segmentation is then performed on top of the discovered units (without influencing the lower level). To do this, I propose a new unit discovery model, a new symbolic word segmentation model, and then chain the two models to segment speech. Both models use dynamic programming to minimize segment costs from a self-supervised network with an additional duration penalty that encourages longer units. Concretely, for acoustic unit discovery, duration-penalized dynamic programming (DPDP) is used with a contrastive predictive coding model as the scoring network. For word segmentation, DPDP is applied with an autoencoding recurrent neural as the scoring network. The two models are chained in order to segment speech. This approach gives comparable word segmentation results to state-of-the-art joint self-supervised segmentation models on an English benchmark. On French, Mandarin, German and Wolof data, it outperforms previous systems on the ZeroSpeech benchmarks. Analysis shows that the chained DPDP system segments shorter filler words well, but longer words might require some external top-down signal.

Self-Sufficient Feature Enhancing Networks for Video Salient Object Detection

Detecting salient objects in videos is a very challenging task. Current state-of-the-art methods are dominated by motion based deep neural networks, among which optical flow is often leveraged as motion representation. Though with robust performance, these optical flow-based video salient object detection methods face at least two problems that may hinder their generalization and application. First, computing optical flow as a pre-processing step does not support direct end-to-end learning; second, little attention has been given to the quality of visual features due to high computational cost of spatiotemporal feature encoding. In this paper we propose a novel self-sufficient feature enhancing network (SFENet) for video salient object detection, which leverages optical flow estimation as an auxiliary task while being end-to-end trainable. With a joint training scheme of both salient object detection and optical flow estimation, its multi-task architecture can be totally self-sufficient for achieving good performance without any pre-processing. Furthermore, for improving feature quality, we design four lightweight modules in spatial and temporal domains, including cross-layer fusion, multi-level warping, spatial-channel attention and boundary-aware refinement. The proposed method is evaluated through extensive experiments on five video salient object detection datasets. Experimental results show that our SFENet can be easily trained with fast convergence speed. It significantly outperforms previous methods in terms of various evaluation metrics. Moreover, with optical flow estimation and unsupervised video object segmentation as example applications, our method also yields state-of-the-art results on standard datasets.

Sequential Order-Aware Coding-Based Robust Subspace Clustering for Human Action Recognition in Untrimmed Videos.

Human action recognition (HAR) is one of most important tasks in video analysis. Since video clips distributed on networks are usually untrimmed, it is required to accurately segment a given untrimmed video into a set of action segments for HAR. As an unsupervised temporal segmentation technology, subspace clustering learns the codes from each video to construct an affinity graph, and then cuts the affinity graph to cluster the video into a set of action segments. However, most of the existing subspace clustering schemes not only ignore the sequential information of frames in code learning, but also the negative effects of noises when cutting the affinity graph, which lead to inferior performance. To address these issues, we propose a sequential order-aware coding-based robust subspace clustering (SOAC-RSC) scheme for HAR. By feeding the motion features of video frames into multi-layer neural networks, two expressive code matrices are learned in a sequential order-aware manner from unconstrained and constrained videos, respectively, to construct the corresponding affinity graphs. Then, with the consideration of the existence of noise effects, a simple yet robust cutting algorithm is proposed to cut the constructed affinity graphs to accurately obtain the action segments for HAR. The extensive experiments demonstrate the proposed SOAC-RSC scheme achieves the state-of-the-art performance on the datasets of Keck Gesture and Weizmann, and provides competitive performance on the other 6 public datasets such as UCF101 and URADL for HAR task, compared to the recent related approaches.

Deep Spectral Clustering With Regularized Linear Embedding for Hyperspectral Image Clustering

The past decade has witnessed the rapid development of deep learning techniques, especially for large-scale and complex data sets. However, it is still a noteworthy problem in dealing with unsupervised hyperspectral image segmentation since inefficiency and misleading result from the absence of supervised information. Generally, spectral clustering is one of the most powerful clustering algorithms, as it often outperforms other methods for image segmentation. Unfortunately, the poor scalability and generalization severely limit the use of spectral clustering, especially for large-scale and high-dimensional hyperspectral images processing. The major motivation of this work is to solve this problem, and we designed a novel algorithm, termed Deep Spectral Clustering with Regularized Linear Embedding (DSCRLE), to benefit from both spectral graph theory and deep learning techniques. The brief procedure is first to construct a fully connected neural network to extract latent feature representations, and then normalize the feature representations by the spectral orthonormal constraint. Lastly, by introducing low-dimensional embedding, we refined the final outputs of all given unlabeled hyperspectral pixels. Extensive experiments have demonstrated that the competitiveness of the proposed method, and it outperforms state-of-art clustering approaches in the task of hyperspectral image segmentation.

Building Extraction From High Spatial Resolution Remote Sensing Images of Complex Scenes by Combining Region-Line Feature Fusion and OCNN

Building extraction from remote sensing imagery has been a research hotspot for some time with the advancement of AI in remote sensing. However, the edges of buildings extracted using existing techniques are commonly broken and inaccurate for the complex scenes in suburban and rural areas. This study proposes a framework for extracting structures by combining region-line feature fusion with object-based convolutional neural networks to solve this problem. First, a building edge detection network known as the Multichannel Attention-based Dense Extreme Inception Network for Edge Detection (MA-DexiNed) is constructed, which is considered more accurate for building edge extraction in complicated image scenes. Second, the probability map of the building edges obtained by MA-DexiNed is refined. According to rule judgment, breakpoints are linked by an edge thinning connection algorithm to obtain single-pixel, contiguous building line features. Third, the geometric boundaries of buildings are obtained by combining region attributes derived by unsupervised image segmentation and line features obtained from deep learning supervised segmentation. Finally, the pretrained AlexNet is employed to identify the class characteristics of buildings. The suggested framework was used for two GF-2 images and one Google Earth image from various regions and with numerous types of complicated scenes. The experimental findings demonstrated that this approach could extract more precise and complete building edges for complex image scenes compared with several existing methods. This advancement results from constrained regional image segmentation using deep semantic edge features. This methodology can offer a benchmark for subsequent building extraction tasks from high resolution imagery.

A Topological Semantic Mapping Method Based on Text-Based Unsupervised Image Segmentation for Assistive Indoor Navigation

A Topological Semantic Mapping Method Based on Text-Based Unsupervised Image Segmentation for Assistive Indoor Navigation

Prototype and Context-Enhanced Learning for Unsupervised Domain Adaptation Semantic Segmentation of Remote Sensing Images

In unsupervised domain adaptation (UDA) of remote sensing images (RSIs), the huge inter-domain discrepancies and intra-domain variances lead to complicated class-level relations. Specifically, the instances of the same class differ greatly while instances of different classes are similar, whether across different RSIs domains or within the same RSIs domain. However, existing methods cannot fully consider these problems, limiting the performance of UDA semantic segmentation of RSIs. To this end, this paper proposes a novel cross-domain multi-prototypes learning method, the core idea of which is to abstract the cross-and intra-domain class-level relations into multiple prototypes. Specifically, the multiple prototypes belonging to different classes can detailedly describe complex inter-class relations, and the multiple prototypes within the same class can better model rich intra-class relations. Further, the source and target samples are jointly used for prototypes calculation, to fully fuse the feature information of different RSIs. In a nutshell, utilizing the samples from different RSIs domains to learn multiple prototypes for each class can achieve better domain alignment at the class level. In addition, considering that RSIs simultaneously contain large targets with wide coverage and important small targets, two masked consistency learning strategies are designed to better explore the contextual structure of target RSIs and improve the quality of pseudo labels for prototype updating. The global consistency strategy can strengthen the utilization of global context relations, while the local consistency strategy can further improve the learning of local context details. Therefore, the proposed method is actually a prototype and context enhanced learning method for UDA semantic segmentation of RSIs. Extensive experiments demonstrate that the proposed method can achieve better performance than existing state-of-the-art UDA methods.

Review of Unsupervised Segmentation Techniques on Long Wave Infrared Images

Review of Unsupervised Segmentation Techniques on Long Wave Infrared Images

Unsupervised Unmixing and Segmentation of Hyper Spectral Images Accounting for Soil Fertility

A crucial component of precision agriculture is the capability to assess the fertility of soil by looking at the precise distribution and composition of its different constituents. This study aims to investigate how different machine learning models may be used to assess soil fertility using hyperspectral pictures. The development of images using a random mixing of different soil components is the first phase, and the hyper spectral bands utilized to create the images are not used again during the analysis procedure. The resulting end members are then acquired by applying the NFINDR algorithm to the process of spectral unmixing this image. The comparison between these end members and the band values of the known elements is then quantified., i.e. it is represented as a graph of band values obtained through spectral unmixing. Finally we quantify the similarities between both graphs and proceed towards the classification of the hyper spectral image as fertile or infertile. In order to classify the hyper spectral image as fertile or infertile, we quantify the similarities between the two graphs. Clustering and picture segmentation algorithms have been devised to help with this process, and a comparison is then made to show which techniques are the most effective.

Convolutional autoencoder joint boundary and mask adversarial learning for fundus image segmentation.

The precise segmentation of the optic cup (OC) and the optic disc (OD) is important for glaucoma screening. In recent years, medical image segmentation based on convolutional neural networks (CNN) has achieved remarkable results. However, many traditional CNN methods do not consider the cross-domain problem, i.e., generalization on datasets of different domains. In this paper, we propose a novel unsupervised domain-adaptive segmentation architecture called CAE-BMAL. Firstly, we enhance the source domain with a convolutional autoencoder to improve the generalization ability of the model. Then, we introduce an adversarial learning-based boundary discrimination branch to reduce the impact of the complex environment during segmentation. Finally, we evaluate the proposed method on three datasets, Drishti-GS, RIM-ONE-r3, and REFUGE. The experimental evaluations outperform most state-of-the-art methods in accuracy and generalization. We further evaluate the cup-to-disk ratio performance in OD and OC segmentation, which indicates the effectiveness of glaucoma discrimination.

Fine-Grained Unsupervised Temporal Action Segmentation and Distributed Representation for Skeleton-Based Human Motion Analysis.

Understanding the fine-grained temporal structure of human actions and its semantic interpretation is beneficial to many real-world tasks, such as sports movements, rehabilitation exercises, and daily-life activities analysis. Current action segmentation methods mainly rely on deep neural networks to derive feature embedding of actions from motion data, while works on analyzing human actions in fine-granularity are still lacking due to the lack of clear and generic definitions of subactions and related datasets. On the other hand, the motion representations obtained in current action segmentation methods lack semantic or mathematical interpretability that can be used to evaluate action/subaction similarity in quantitative motion analysis. Toward the goal of fine-grained, interpretable, scalable, and efficient action segmentation, we propose a novel unsupervised action segmentation and distributed representation framework based on intuitive motion primitives defined on pose data. Metrics for comprehensive evaluation of the unsupervised fine-grained action segmentation task performance are proposed, and both public and self-constructed datasets are adopted in the experiments. The results show that the proposed method has good performance and generality across different subjects, datasets, and application scenarios.

Unsupervised Tissue Segmentation via Deep Constrained Gaussian Network.

Tissue segmentation is the mainstay of pathological examination, whereas the manual delineation is unduly burdensome. To assist this time-consuming and subjective manual step, researchers have devised methods to automatically segment structures in pathological images. Recently, automated machine and deep learning based methods dominate tissue segmentation research studies. However, most machine and deep learning based approaches are supervised and developed using a large number of training samples, in which the pixel-wise annotations are expensive and sometimes can be impossible to obtain. This paper introduces a novel unsupervised learning paradigm by integrating an end-to-end deep mixture model with a constrained indicator to acquire accurate semantic tissue segmentation. This constraint aims to centralise the components of deep mixture models during the calculation of the optimisation function. In so doing, the redundant or empty class issues, which are common in current unsupervised learning methods, can be greatly reduced. By validation on both public and in-house datasets, the proposed deep constrained Gaussian network achieves significantly (Wilcoxon signed-rank test) better performance (with the average Dice scores of 0.737 and 0.735, respectively) on tissue segmentation with improved stability and robustness, compared to other existing unsupervised segmentation approaches. Furthermore, the proposed method presents a similar performance (p-value >0.05) compared to the fully supervised U-Net.

Usable Amharic text corpus for natural language processing applications

In this paper, we describe the preparation of a usable Amharic text corpus for different Natural Language Processing (NLP) applications. Natural language applications, such as document classification, topic modeling, machine translation, speech recognition, and others, suffer greatly from a lack of digital resources. This is especially true for Amharic, a resource-constrained, morphologically rich, and complex language. In response to this, a total of 67,739 Amharic news documents consisting of 8 different categories from online sources are collected. The collected corpus passes through a number of pre-processing steps including; data cleaning, text normalization and punctuation correction. To validate the usability of the collected corpora from different domains, a baseline document classification experiment was conducted. Experimental results show that, 84.53% accuracy is registered using deep learning in the absence of linguistic information. Finding indicated that it is possible to use the prepared corpora for different natural language applications in the absence of linguistic resources such as stemmer and dictionary despite the complexity of Amharic language. We are further working towards Amharic news document classification by incorporating a linguistic independent stop-word detection, stemming and unsupervised morphological segmentation of Amharic documents.

Unsupervised Word Segmentation with Bi-directional Neural Language Model

We propose an unsupervised word segmentation model, in which for each unlabelled sentence sample, the learning objective is to maximize the generation probability of the sentence given its all possible segmentations. Such a generation probability can be factorized into the likelihood of each possible segment given the context in a recursive way. To capture both the long- and short-term dependencies, we propose to use a bi-directional neural language model to better extract the features of the segment’s context. Two decoding algorithms were also developed to combine the context features from both directions to generate the final segmentation at the inference time, which helps to reconcile word-boundary ambiguities. Experimental results show that our context-sensitive unsupervised segmentation model achieved state-of-the-art at different evaluation settings on various datasets for Chinese, and the comparable result for Thai.