Unsupervised Image Segmentation Research Articles

A Superpixel-Guided Unsupervised Fast Semantic Segmentation Method of Remote Sensing Images

Semantic segmentation is one of the fundamental tasks of pixel-level remote sensing image analysis. Currently, most high-performance semantic segmentation methods are trained in a supervised learning manner. These methods require a large number of image labels as support, but manual annotations are difficult to obtain. To address the problem, we propose an efficient unsupervised remote sensing image segmentation method based on superpixel segmentation and fully convolutional networks (FCNs) in this letter. Our method can achieve pixel-level images segmentation of various scales rapidly without any manual labels or prior knowledge. We use the superpixel segmentation results as synthetic ground truth to guide the gradient descent direction during FCN training. In experiments, our method achieved high performance compared to current unsupervised image segmentation methods on three public datasets. Specifically, our method achieves an Adjusted Mutual Information (AMI) score of 0.2955 on the Gaofen Image Dataset (GID) dataset, while processing each image of size 7200 × 6800 pixels in just 30 seconds.

Analyzing Cell-Scaffold Interaction through Unsupervised 3D Nuclei Segmentation.

Fibrous scaffolds have been extensively used in three-dimensional (3D) cell culture systems to establish in vitro models in cell biology, tissue engineering, and drug screening. It is a common practice to characterize cell behaviors on such scaffolds using confocal laser scanning microscopy (CLSM). As a noninvasive technology, CLSM images can be utilized to describe cell-scaffold interaction under varied morphological features, biomaterial composition, and internal structure. Unfortunately, such information has not been fully translated and delivered to researchers due to the lack of effective cell segmentation methods. We developed herein an end-to-end model called Aligned Disentangled Generative Adversarial Network (AD-GAN) for 3D unsupervised nuclei segmentation of CLSM images. AD-GAN utilizes representation disentanglement to separate content representation (the underlying nuclei spatial structure) from style representation (the rendering of the structure) and align the disentangled content in the latent space. The CLSM images collected from fibrous scaffold-based culturing A549, 3T3, and HeLa cells were utilized for nuclei segmentation study. Compared with existing commercial methods such as Squassh and CellProfiler, our AD-GAN can effectively and efficiently distinguish nuclei with the preserved shape and location information. Building on such information, we can rapidly screen cell-scaffold interaction in terms of adhesion, migration and proliferation, so as to improve scaffold design.

Unsupervised hyperspectral stimulated Raman microscopy image enhancement: denoising and segmentation via one-shot deep learning.

Hyperspectral stimulated Raman scattering (SRS) microscopy is a label-free technique for biomedical and mineralogical imaging which can suffer from low signal-to-noise ratios. Here we demonstrate the use of an unsupervised deep learning neural network for rapid and automatic denoising of SRS images: UHRED (Unsupervised Hyperspectral Resolution Enhancement and Denoising). UHRED is capable of "one-shot" learning; only one hyperspectral image is needed, with no requirements for training on previously labelled datasets or images. Furthermore, by applying a k-means clustering algorithm to the processed data, we demonstrate automatic, unsupervised image segmentation, yielding, without prior knowledge of the sample, intuitive chemical species maps, as shown here for a lithium ore sample.

Unsupervised Image Segmentation by Mutual Information Maximization and Adversarial Regularization

Semantic segmentation is one of the basic, yet essential scene understanding tasks for an autonomous agent. The recent developments in supervised machine learning and neural networks have enjoyed great success in enhancing the performance of the state-of-the-art techniques for this task. However, their superior performance is highly reliant on the availability of a large-scale annotated dataset. In this letter, we propose a novel fully unsupervised semantic segmentation method, the so-called Information Maximization and Adversarial Regularization Segmentation (InMARS). Inspired by human perception which parses a scene into perceptual groups, rather than analyzing each pixel individually, our proposed approach first partitions an input image into meaningful regions (also known as superpixels). Next, it utilizes Mutual-Information-Maximization followed by an adversarial training strategy to cluster these regions into semantically meaningful classes. To customize an adversarial training scheme for the problem, we incorporate adversarial pixel noise along with spatial perturbations to impose photometrical and geometrical invariance on the deep neural network. Our experiments demonstrate that our method achieves the state-of-the-art performance on two commonly used unsupervised semantic segmentation datasets, COCO-Stuff, and Potsdam.

Unsupervised segmentation of microstructural images of steel using data mining methods

A novel and efficient data mining method for the segmentation of microstructural images of low-carbon steel is presented. Microstructural characterization has been the focus of many works in the field of materials science because microstructure is the fundamental element in understanding the link between process and property. Recently, deep-learning-based methods have been actively employed for microstructural classification since it has shown outstanding performance for solving image classification problems. However, previous applications of deep learning models to microstructural classification revealed limitations in that not only do they require the time-consuming labeling process but it is also still difficult to obtain a satisfactory result, especially for steel microstructures containing substances developed by displacive transformation. In this study, we propose a rule-based segmentation method that not only works without labeled images but also requires no prior knowledge of the number of microstructural constituents in each image. This unsupervised inference algorithm captures the morphological features of each microstructure and automatically finds the optimal number of microstructures having similar characteristics using a Bayesian Gaussian mixture model. The viability of our method is demonstrated by qualitative and quantitative evaluations with optical microscopy images of steel composed of different microstructures taken under different imaging conditions.

Unsupervised Object Segmentation Based on Bi-Partitioning Image Model Integrated with Classification

The development of convolutional neural networks for deep learning has significantly contributed to image classification and segmentation areas. For high performance in supervised image segmentation, we need many ground-truth data. However, high costs are required to make these data, so unsupervised manners are actively being studied. The Mumford–Shah and Chan–Vese models are well-known unsupervised image segmentation models. However, the Mumford–Shah model and the Chan–Vese model cannot separate the foreground and background of the image because they are based on pixel intensities. In this paper, we propose a weakly supervised model for image segmentation based on the segmentation models (Mumford–Shah model and Chan–Vese model) and classification. The segmentation model (i.e., Mumford–Shah model or Chan–Vese model) is to find a base image mask for classification, and the classification network uses the mask from the segmentation models. With the classifcation network, the output mask of the segmentation model changes in the direction of increasing the performance of the classification network. In addition, the mask can distinguish the foreground and background of images naturally. Our experiment shows that our segmentation model, integrated with a classifier, can segment the input image to the foreground and the background only with the image’s class label, which is the image-level label.

Automatic Segmentation of Digital Pathology Slides Based on Unsupervised Learning

To segment images through an unsupervised method as an alternative to manual labeling. A total of 100 whole slide image (WSI) data of HE stained and Pap stained slides were selected as the research and test objects, including 70 breast slides, 20 lung slides and 10 thyroid slides. In order to ensure the diversity of data, the breast slides included those of normal tissue, inflammation and tumor, the lung slides were mainly neoplasms in the lower lobe, including those of inflammation and tumor, and the thyroid slides were of cells, all benign, obtained through fine needle aspiration. The maximum total magnification (original magnification) of each image was 400 times, and the file format was NDPI. Each WSI was manually labeled, and the labeled area of each WSI was more than 10 fields of vision. The labeled information was to be used for validity verification. An unsupervised image segmentation technique based on superpixel and fully convolution neural network algorithms was constructed and used to segment any region of interest (ROI) of unlabeled WSI. In comparison with the region adjacency graph merging method, the segmentation effect of the two methods was assessed with the under segmentation error, the boundary recall and the mean Intersection-over-Union, and the efficiency of the two methods was also compared. In the comparison of execution efficiency, the test process included the preprocessing time of superpixel, and excluded the time of loading the deep learning engine. Unsupervised automatic segmentation was implemented for any ROI region of WSI according to the texture and color. The results of the breast slides, lung slides and thyroid slides showed slight differences, and multiple tests yielded stable results. However, the performance of this method in differentiating inflammation and tumor was average. The under-segmentation error, the boundary recall and the mean Intersection-over-Union were 19.10%, 82.06% and 45.06%, respectively. The under segmentation error, the boundary recall and the mean Intersection-over-Union for the region adjacency graph merging method were 21.52%, 78.39% and 44.81%, respectively. The average time consumption of the whole process was 0.27 s in GPU mode and 1.30 s in CPU mode. The average time consumption of the region adjacency graph merging method was 10.5 s in CPU mode because the method of region adjacency graph merging was not realized in the GPU mode. This method produced ideal pixel level labeling results through simple human-computer interaction, which could effectively reduce the cost of digital pathology slide data labeling. Compared with the region adjacency graph merging method, this method had better performance in processing image texture and had faster processing speed.

Benthic Habitat Map of the Southern Adriatic Sea (Mediterranean Sea) from Object-Based Image Analysis of Multi-Source Acoustic Backscatter Data

A huge amount of seabed acoustic reflectivity data has been acquired from the east to the west side of the southern Adriatic Sea (Mediterranean Sea) in the last 18 years by CNR-ISMAR. These data have been used for geological, biological and habitat mapping purposes, but a single and consistent interpretation of them has never been carried out. Here, we aimed at coherently interpreting acoustic data images of the seafloor to produce a benthic habitat map of the southern Adriatic Sea showing the spatial distribution of substrates and biological communities within the basin. The methodology here applied consists of a semi-automated classification of acoustic reflectivity, bathymetry and bathymetric derivatives images through object-based image analysis (OBIA) performed by using the ArcGIS tool RSOBIA (Remote Sensing OBIA). This unsupervised image segmentation was carried out on each cruise dataset separately, then classified and validated through comparison with bottom samples, images, and prior knowledge of the study areas.

Image Segmentation Using the EM/MPM Algorithm

In this article we propose to place our work in a Markovian framework for unsupervised image segmentation. We give one of the procedures for estimating the parameters of a Markov field, we limit the work to the EM estimation method and the Posterior Marginal Maximization (MPM) segmentation method. Estimating the number of regions who compones the image is relatively difficult, we try to solve this problem by the K-means Histogram method.

A novel CNN-LSTM-based approach to predict urban expansion

Time-series remote sensing data offer a rich source of information that can be used in a wide range of applications, from monitoring changes in land cover to surveillance of crops, coastal changes, flood risk assessment, and urban sprawl. In this paper, time-series satellite images are used to predict urban expansion. As the ground truth is not available in time-series satellite images, an unsupervised image segmentation method based on deep learning is used to generate the ground truth for training and validation. The automated annotated images are then manually validated using Google Maps to generate the ground truth. The remaining data were then manually annotated. Prediction of urban expansion is achieved by using a ConvLSTM network, which can learn the global spatio-temporal information without shrinking the size of spatial feature maps. The ConvLSTM based model is applied on the time-series satellite images and the results of prediction are compared with Pix2pix and Dual GAN networks. In this paper, experimental results are conducted using several multi-date satellite images representing the three largest cities in Saudi Arabia, namely: Riyadh, Jeddah, and Dammam. The evaluation results show that the proposed ConvLSTM based model produced better prediction results in terms of Mean Square Error, Root Mean Square Error, Peak Signal to Noise Ratio, Structural Similarity Index, and overall classification accuracy as compared to Pix2pix and Dual GAN. Moreover, the training time of the proposed architecture is less than the Dual GAN architecture.

Deep self-representative subspace clustering network

Deep learning based subspace clustering networks have been a significant technique for motion segmentation, unsupervised image segmentation, image representation and compression, and face clustering by separating the high-dimensional data points into their representative low-dimensional linear subspaces. Effective feature selection is critical to remove redundant samples and select the representative feature subset from high-dimensional data space; hence deriving the number of subspaces, their dimensions, data segmentation, and a basis for each subspace. The effective self-representative feature selection and emphasis by scaling the feature map in the learned embedded space is required for deep learning based subspace clustering to reduce the number of parameters and dimension of the self-representative layer. In this paper, we propose a self-representative feature extraction deep neural network for unsupervised subspace clustering to improve representativeness and clustering ability. The extensive relevant results on various data demonstrate that deep subspace clustering employing self-representative features from high-dimensional data can effectively reduce the dimension of the self-representative layer while improving performance.

A scalable approach for the efficient segmentation of hyperspectral images

The number of applications of hyperspectral imaging (HSI) is steadily increasing, as technology evolves and cameras become more affordable. However, the volume of data in a hyperspectral image is large (order of Gigabytes) and standard off-the-shelf algorithms for multi-channel image analysis cannot be readily applied, due to the prohibitive computational time and large memory requirements. Therefore, new scalable approaches are required to perform hyperspectral image analysis. In this article we address an efficient methodology for conducting Unsupervised Image Segmentation – one of the basic and most fundamental image analysis operations. In the methodology proposed, unsupervised segmentation is conducted after transforming the spectral and spatial dimensions of the raw hyperspectral image into a more compact representation using multivariate and multiresolution techniques. The clusters identified in the compact image representation are then used to train a discriminative classifier. The classifier is then adapted and transferred for application to the raw image, where it will efficiently label all the original pixels. With the proposed methodology, the computational expensive operations (unsupervised clustering and classifier learning) are minimized, whereas the efficient implementation of the classifier guarantees the analysis at the native resolution. The effectiveness of the proposed methodology was tested on a real case study considering an industrial hyperspectral image capturing the reflectance spectrum for several objects made of different unknown materials. A significant reduction in the computational cost was achieved without compromising the quality of the unsupervised segmentation, demonstrating the potential of the proposed approach.

Unsupervised image segmentation with Gaussian Pairwise Markov Fields

Modeling strongly correlated random variables is a critical task in the context of latent variable models. A new probabilistic model, called Gaussian Pairwise Markov Field, is presented to generalize existing Markov Fields latent variables models, and to introduce more correlations between variables. This is done by considering the correlations within Gaussian Markov Random Fields models which are much richer than in the classical Markov Field models. The assets of the Gaussian Pairwise Markov Field model are explained. In particular, it offers a generalization of the classical Markov Field modelization that is highlighted. The new model is also considered in the practical case of unsupervised segmentation of images corrupted by long-range spatially-correlated noise, producing interesting new results.

Non-local spatially varying finite mixture models for image segmentation

In this work, we propose a new Bayesian model for unsupervised image segmentation based on a combination of the spatially varying finite mixture models (SVFMMs) and the non-local means (NLM) framework. The probabilistic NLM weighting function is successfully integrated into a varying Gauss–Markov random field, yielding a prior density that adaptively imposes a local regularization to simultaneously preserve edges and enforce smooth constraints in homogeneous regions of the image. Two versions of our model are proposed: a pixel-based model and a patch-based model, depending on the design of the probabilistic NLM weighting function. Contrary to previous methods proposed in the literature, our approximation does not introduce new parameters to be estimated into the model, because the NLM weighting function is completely known once the neighborhood of a pixel is fixed. The proposed model can be estimated in closed-form solution via a maximum a posteriori (MAP) estimation in an expectation–maximization scheme. We have compared our model with previously proposed SVFMMs using two public datasets: the Berkeley Segmentation dataset and the BRATS 2013 dataset. The proposed model performs favorably to previous approaches in the literature, achieving better results in terms of Rand Index and Dice metrics in our experiments.

Introduction to Basic and Deep Learning Based Methods for Unsupervised Image Segmentation

Introduction to Basic and Deep Learning Based Methods for Unsupervised Image Segmentation

Unsupervised Palm Vein Image Segmentation

In this article the new hybrid algorithm for palm vein image segmentation using convolutional neural network and principal curvatures is proposed. After palm vein image preprocessing vein structure is detected using unsupervised learning approach based on W-Net architecture, that ties together into a single autoencoder two fully convolutional neural network architectures, each simi-lar to the U-Net. Then segmentation results are improved using principal cur-vatures technique. Some vein points with highest maximum principal curva-ture values are selected, and the other vein points are found by moving from starting points along the direction of minimum principal curvature. To obtain the final vein image segmentation the result of intersection of the principal curvatures-based and neural network-based segmentations is taken. The evaluation of the proposed unsupervised image segmentation method based on palm vein recognition results using multilobe differential filters is given. Test results using CASIA multi-spectral palmprint image database show the effectiveness of the proposed segmentation approach.

High-Order Triplet CRF-PCANet for Unsupervised Segmentation of Nonstationary SAR Image

Conditional random fields (CRFs) model is suitable for image segmentation because it can capture the dependencies of observed data and incorporate the spatial correlations into the segmentation process. In this article, to deal with the segmentation of nonstationary synthetic aperture radar (SAR) image, we combine the modeling power of the CRF model with the representation-learning ability of principal component analysis network (PCANet), and thus propose a high-order triplet CRF model based on PCANet (HOTCRF-PCANet). HOTCRF-PCANet introduces an auxiliary field to explicitly regulate nonstationary label structure patterns. Under the guidance of this auxiliary field, HOTCRF-PCANet defines a discrete quadrilateral nonstationary Markov fields model, and thus considers both the nonstationary property of image and high-order label interactions. In addition, guided by the auxiliary field, HOTCRF-PCANet proposes to use a product-of-expert (POE) potential to enforce the regions’ labeling consistency for pixels within the weak-structured region. To automatically learn rich feature representations, HOTCRF-PCANet modifies PCANet into an unsupervised mode, i.e., unsupervised PCANet (UPCANet), and constructs an UPCANet-based unary potential to effectively predict the local class probability. The effectiveness of HOTCRF-PCANet is demonstrated by the application to the unsupervised segmentation of the simulated images and real SAR images.

Unsupervised color image segmentation: A case of RGB histogram based K-means clustering initialization.

Color-based image segmentation classifies pixels of digital images in numerous groups for further analysis in computer vision, pattern recognition, image understanding, and image processing applications. Various algorithms have been developed for image segmentation, but clustering algorithms play an important role in the segmentation of digital images. This paper presents a novel and adaptive initialization approach to determine the number of clusters and find the initial central points of clusters for the standard K-means algorithm to solve the segmentation problem of color images. The presented scheme uses a scanning procedure of the paired Red, Green, and Blue (RGB) color-channel histograms for determining the most salient modes in every histogram. Next, the histogram thresholding is applied and a search in every histogram mode is performed to accomplish RGB pairs. These RGB pairs are used as the initial cluster centers and cluster numbers that clustered each pixel into the appropriate region for generating the homogeneous regions. The proposed technique determines the best initialization parameters for the conventional K-means clustering technique. In this paper, the proposed approach was compared with various unsupervised image segmentation techniques on various image segmentation benchmarks. Furthermore, we made use of a ranking approach inspired by the Evaluation Based on Distance from Average Solution (EDAS) method to account for segmentation integrity. The experimental results show that the proposed technique outperforms the other existing clustering techniques by optimizing the segmentation quality and possibly reducing the classification error.

A novel approach for unsupervised image segmentation fusion of plant leaves based on G-mutual information

Plant leaf segmentation has a very important role in most plant identification methods. Tree leaves segmentation in images with complex background is very difficult when there is no prior information about the leaves and backgrounds. In practice, the parameters of unsupervised image segmentation algorithms must be set for each image to get the best results. In this paper, to overcome this problem, fusion of the results of five leaf segmentation algorithms (fuzzy c-means, SOM and k-means in various color spaces or different parameters) is applied. To fuse the results of these segmentations, new equations for mutual information (g-mutual information equations) based on the g-calculus are introduced to find the best consensus segmentation. The results of the mentioned primary clustering algorithms are considered as a new feature vector for each pixel. To reduce the time complexity, a fast method is employed using truth table containing different feature vectors. To evaluate this new approach, a leaf image database with natural scenes, taken from Pl@ntLeaves database, is generated to have different positions and orientations. In addition, a widely used database is used to compare the proposed method with other methods. The experimental results presented in this paper show that the use of g-calculus in fusion of image segmentations improves the evaluation parameters.

An unsupervised image segmentation method combining graph clustering and high-level feature representation

Image segmentation is one of the most important assignments in computer vision. In this paper, we present an unsupervised segmentation method that combines graph-based clustering and high-level semantic features. We over-segment the given image into a collection of superpixels. Various low-level features assemble a descriptor of each superpixel. Besides the intrinsic image features such as color, texture and gradient, we add image saliency into the low-level visual features as prior knowledge of human perception. Instead of using the low-level features directly, we design a graph-based method to segment the image by clustering the high-level semantic features learned from a neural network. We test the proposed method on two well-known datasets. The experimental evaluation validates that our approach can provide consistent and meaningful segmentation.