Superpixel-based Segmentation Research Articles

Joint learning framework of superpixel generation and fuzzy sparse subspace clustering for color image segmentation

Sparse subspace clustering (SSC) is an important image segmentation method that constructs a self-representation coefficient matrix to represent the relationships between pixels, and then uses spectral clustering to achieve clustering. Compared with other unsupervised segmentation algorithms, SSC has good segmentation performance. However, SSC has a high computational complexity when processing large-sized image. To improve computational efficiency, many researchers have proposed superpixel-based SSC algorithms, which process superpixels instead of pixels and improve efficiency through preprocessing. Due to the sensitivity of superpixel generation to noise, superpixel-based SSC algorithms still have poor robustness. Additionally, preprocessing increases the complexity of the algorithm. To address these issues, this paper proposes a robust superpixel-based fuzzy sparse subspace clustering algorithm. This algorithm combines fuzzy sparse subspace clustering with superpixel generation, and it constructs a unified optimization learning framework through fuzzy C-multiple-means clustering to improve segmentation performance and reduce complexity. Additionally, this paper introduces additional features of superpixel in sparse subspace clustering to further enhance the segmentation performance of the algorithm. Experiment results indicate that the proposed algorithm not only outperforms existing state-of-the-art robust segmentation algorithms independent of superpixels, but also is superior to the latest superpixel-based segmentation algorithms.

Superpixel linear independent preprocessing for endmember extraction

ABSTRACT One of the limitations of remote sensing is the low spatial resolution of the open-access multispectral sensors, generating a mixture of spatial information. The mixed pixels can be modelled as the linear combination of fundamental components, called endmember, with a weighted contribution or abundance. The development of linear unmixing algorithms considering spatial and spectral information has recently increased. Some unmixing methods have relied on segmentation to integrate spatial data, and one of the most used is superpixel-based segmentation. However, previous work in superpixel-based unmixing focuses on using superpixels as uniform regions. Commonly, linear unmixing is used on hyperspectral imagery, and limited literature is found with multispectral images. This paper aims to propose a new preprocessing approach for multispectral linear unmixing called Superpixel Linear Independent Preprocessing. The proposed approach generates a set of candidates to endmembers based on spatial-spectral information; these are the input of traditional endmember extraction methods for multispectral unmixing. Experimental results show that the proposed preprocessing improves the performance of endmember extraction.

SUShe: simple unsupervised shadow removal

Shadow removal is an important problem in computer vision, since the presence of shadows complicates core computer vision tasks, including image segmentation and object recognition. Most state-of-the-art shadow removal methods are based on complex deep learning architectures, which require training on a large amount of data. In this paper a novel and efficient methodology is proposed aiming to provide a simple solution to shadow removal, both in terms of implementation and computational cost. The proposed methodology is fully unsupervised, based solely on color image features. Initially, the shadow region is automatically extracted by a segmentation algorithm based on Electromagnetic-Like Optimization. Superpixel-based segmentation is performed and pairs of shadowed and non-shadowed regions, which are nearest neighbors in terms of their color content, are identified as parts of the same object. The shadowed part of each pair is relighted by means of histogram matching, using the content of its non-shadowed counterpart. Quantitative and qualitative experiments on well-recognized publicly available benchmark datasets are conducted to evaluate the performance of proposed methodology in comparison to state-of-the-art methods. The results validate both its efficiency and effectiveness, making evident that solving the shadow removal problem does not necessarily require complex deep learning-based solutions.

Efficient thermal face recognition method using optimized curvelet features for biometric authentication.

Biometric technology is becoming increasingly prevalent in several vital applications that substitute traditional password and token authentication mechanisms. Recognition accuracy and computational cost are two important aspects that are to be considered while designing biometric authentication systems. Thermal imaging is proven to capture a unique thermal signature for a person and thus has been used in thermal face recognition. However, the literature did not thoroughly analyse the impact of feature selection on the accuracy and computational cost of face recognition which is an important aspect for limited resources applications like IoT ones. Also, the literature did not thoroughly evaluate the performance metrics of the proposed methods/solutions which are needed for the optimal configuration of the biometric authentication systems. This paper proposes a thermal face-based biometric authentication system. The proposed system comprises five phases: a) capturing the user's face with a thermal camera, b) segmenting the face region and excluding the background by optimized superpixel-based segmentation technique to extract the region of interest (ROI) of the face, c) feature extraction using wavelet and curvelet transform, d) feature selection by employing bio-inspired optimization algorithms: grey wolf optimizer (GWO), particle swarm optimization (PSO) and genetic algorithm (GA), e) the classification (user identification) performed using classifiers: random forest (RF), k-nearest neighbour (KNN), and naive bayes (NB). Upon the public dataset, Terravic Facial IR, the proposed system was evaluated using the metrics: accuracy, precision, recall, F-measure, and receiver operating characteristic (ROC) area. The results showed that the curvelet features optimized using the GWO and classified with random forest could help in authenticating users through thermal images with performance up to 99.5% which is better than the results of wavelet features by 10% while the former used 5% fewer features. In addition, the statistical analysis showed the significance of our proposed model. Compared to the related works, our system showed to be a better thermal face authentication model with a minimum set of features, making it computational-friendly.

Sparse‐sensing and superpixel‐based segmentation model for concrete cracks

AbstractEfficient image‐recognition algorithms to classify the pixels accurately are required for the computer‐vision‐based inspection of concrete defects. This study proposes a deep learning‐based model called sparse‐sensing and superpixel‐based segmentation (SSSeg) for accurate and efficient crack segmentation. The model employed a sparse‐sensing‐based encoder and a superpixel‐based decoder and was compared with six state‐of‐the‐art models. An input pipeline of 1231 diverse crack images was specially designed to train and evaluate the models. The results indicated that the SSSeg achieved a good balance between the recognition correctness and completeness and outperformed other models in both accuracy and efficiency. The SSSeg also exhibited good resistance to the interference of surface roughness, dirty stains, and moisture. The increased depth and receptive field of sparse‐sensing units guaranteed the representability; meanwhile, structured sparse characteristics protected the network from overfitting. The lightweight superpixel‐based decoder omitted skip connections, which greatly reduced the computation and memory footprint and enlarged the input size in the inference.

Revisiting SLIC: Fast Superpixel Segmentation of Marine SAR Images Using Density Features

The simple linear iterative clustering (SLIC) has been shown as an efficient and widely used superpixel-based algorithm for segmenting marine synthetic aperture radar (SAR) images. However, SLIC does not consider the fact that the density of ship target pixels is significantly lower than that of sea clutter pixels, leading to a waste of computational cost and memory resources on lots of pure clutter areas and to the degradation of the compactness of superpixels. To address the aforementioned issues, we develop a new density-based SLIC (DSLIC) method for the superpixel-based segmentation of marine SAR images. In the initialization stage of our DSLIC, all the subimages in a large marine SAR image are rapidly prescreened via a new density-driven classifier, where most of the subimages only occupied by clutter pixels with comparatively high density are discarded and do not need to be segmented in the subsequent local clustering stage. The retained subimages contain both the clutter and potential target areas. This prescreening operation results in higher computation efficiency and memory savings. In the local clustering stage of DSLIC, besides the intensity proximity and the spatiality proximity (used in SLIC), the sparsity proximity (measured by density distances) is considered to reduce the coexistence of sparse target pixels with low density and nonsparse clutter pixels with high density within superpixels. Our theoretical and experimental results show that the proposed DSLIC method is faster and requires less memory than SLIC and other state-of-the-art superpixel-based segmentation methods for marine SAR images with similar or better segmentation accuracy.

Gradient convolutional neural network for classification of agricultural fields with contour levee

ABSTRACT To better understand how contour-levee irrigation practice impacts water resources for formulating effective water management policies, it is important to obtain its application on large-scale data sets, e.g. state-wide. Automatic classification of contour levee croplands from high-resolution aerial images is of great potential given the success of deep neural networks and the availability of high-resolution remote sensing imagery. This paper proposes a gradient CNN model to classify fields with contour levees from remote sensing images. Our model produces high-quality segmentation masks that are refined with superpixel-based segmentation post-processing. Our method is evaluated using images by the National Agriculture Imagery Program (NAIP) for the counties in Arkansas. A comparison with the state-of-the-art methods demonstrates the improved performance of our proposed method. Our method demonstrates superior performance in the classification of challenging cases and achieves an overall 3.08% of accuracy improvement and 28.57% BER error reduction, compared to the second-best method. The p-value with respect to the second-best method is 0.005, which indicates great statistical significance. In addition, the results for data of different counties demonstrate the exceptional generalization ability of our method.

Real-Time Plane Detection with Consistency from Point Cloud Sequences.

Real-time consistent plane detection (RCPD) from structured point cloud sequences facilitates various high-level computer vision and robotic tasks. However, it remains a challenge. Existing techniques for plane detection suffer from a long running time or the problem that the plane detection result is not precise. Meanwhile, labels of planes are not consistent over the whole image sequence due to plane loss in the detection stage. In order to resolve these issues, we propose a novel superpixel-based real-time plane detection approach, while keeping their consistencies over frames simultaneously. In summary, our method has the following key contributions: (i) a real-time plane detection algorithm to extract planes from raw structured three-dimensional (3D) point clouds collected by depth sensors; (ii) a superpixel-based segmentation method to make the detected plane exactly match its actual boundary; and, (iii) a robust strategy to recover the missing planes by utilizing the contextual correspondences information in adjacent frames. Extensive visual and numerical experiments demonstrate that our method outperforms state-of-the-art methods in terms of efficiency and accuracy.

Superpixel-based segmentation algorithm for mature citrus.

With the decrease of agricultural labors and the increase in production costs, harvesting robots have become a research hotspot in recent years. To guide harvesting robots to pick mature citrus more precisely under variable illumination conditions, an image segmentation algorithm based on superpixel was proposed. Efficient simple linear iterative clustering (SLIC) algorithm which takes similarity of adjacent pixels into account was adopted to segment the images captured under variable illumination conditions into superpixels. The color and texture features of these superpixels were extracted and fused into feature vectors as descriptors to train backpropagation neural networks (BPNN) classifier in the next step. The adjacency information of superpixels was considered by calculating the global-local binary pattern (LBP) in R component images when extracting texture features. To accelerate the classification process, the mean of Cr-Cb image was utilized to find superpixels of interest which were regarded as candidates of citrus superpixels. These candidates were then classified by a pre-trained BPNN model with superpixel-level accuracy of 98.77% and pixel-level accuracy of 94.96%, while the average time to segment one image was 0.4778 s. Therefore, the results indicated that a superpixel-based segmentation algorithm toward citrus images had decent light robustness as well as high accuracy that could guide harvesting robot to pick mature citrus efficiently. Keywords: superpixel, image segmentation, BPNN, variable illumination, mature citrus DOI: 10.25165/j.ijabe.20201304.5607 Citation: Yang Q H, Chen Y Q, Xun Y, Bao G J. Superpixel-based segmentation algorithm for mature citrus. Int J Agric & Biol Eng, 2020; 13(4): 166–171.

Multilabel Sample Augmentation-Based Hyperspectral Image Classification

The quantity and quality of training samples have a great influence on the performance of most hyperspectral image classification approaches. However, in a real scenario, manually annotating a large number of accurate training samples is extremely labor-intensive and time-consuming. In this article, a multilabel training sample augmentation method is proposed. Instead of giving an exact label to each pixel, we just precisely label a small number of pixels by giving them a single label (called single-label samples) and annotate a large number of pixels in certain regions together by giving them multiple labels (called multilabel samples). Furthermore, in order to make full use of the multilabel training samples, a superpixel segmentation and recursive filtering-based method is proposed. The proposed method consists of the following major steps: recursive filtering-based feature extraction, superpixel-based segmentation, and spectral–spatial similarity-based mislabeled sample removal. Experimental results demonstrate that the proposed method can significantly improve the classification accuracy of multiple classifiers by using the multilabel training samples.

Superpixel-based segmentation algorithm for mature citrus

With the decrease of agricultural labors and the increase in production costs, harvesting robots have become a research hotspot in recent years. To guide harvesting robots to pick mature citrus more precisely under variable illumination conditions, an image segmentation algorithm based on superpixel was proposed. Efficient simple linear iterative clustering (SLIC) algorithm which takes similarity of adjacent pixels into account was adopted to segment the images captured under variable illumination conditions into superpixels. The color and texture features of these superpixels were extracted and fused into feature vectors as descriptors to train backpropagation neural networks (BPNN) classifier in the next step. The adjacency information of superpixels was considered by calculating the global-local binary pattern (LBP) in R component images when extracting texture features. To accelerate the classification process, the mean of Cr-Cb image was utilized to find superpixels of interest which were regarded as candidates of citrus superpixels. These candidates were then classified by a pre-trained BPNN model with superpixel-level accuracy of 98.77% and pixel-level accuracy of 94.96%, while the average time to segment one image was 0.4778 s. Therefore, the results indicated that a superpixel-based segmentation algorithm toward citrus images had decent light robustness as well as high accuracy that could guide harvesting robot to pick mature citrus efficiently. Keywords: superpixel, image segmentation, BPNN, variable illumination, mature citrus DOI: 10.25165/j.ijabe.20201304.5607 Citation: Yang Q H, Chen Y Q, Xun Y, Bao G J. Superpixel-based segmentation algorithm for mature citrus. Int J Agric & Biol Eng, 2020; 13(4): 166–171.

Multiple Kernel-Based SVM Classification of Hyperspectral Images by Combining Spectral, Spatial, and Semantic Information

In this study, we present a hyperspectral image classification method by combining spectral, spatial, and semantic information. The main steps of the proposed method are summarized as follows: First, principal component analysis transform is conducted on an original image to produce its extended morphological profile, Gabor features, and superpixel-based segmentation map. To model spatial information, the extended morphological profile and Gabor features are used to represent structure and texture features, respectively. Moreover, the mean filtering is performed within each superpixel to maintain the homogeneity of the spatial features. Then, the k-means clustering and the entropy rate superpixel segmentation are combined to produce semantic feature vectors by using a bag of visual-words model for each superpixel. Next, three kernel functions are constructed to describe the spectral, spatial, and semantic information, respectively. Finally, the composite kernel technique is used to fuse all the features into a multiple kernel function that is fed into a support vector machine classifier to produce a final classification map. Experiments demonstrate that the proposed method is superior to the most popular kernel-based classification methods in terms of both visual inspection and quantitative analysis, even if only very limited training samples are available.

Superpixel-Based Segmentation of Polarimetric SAR Images through Two-Stage Merging

Image segmentation plays a fundamental role in image understanding and region-based applications. This paper presents a superpixel-based segmentation method for Polarimetric SAR (PolSAR) data, in which a two-stage merging strategy is proposed. First, based on the initial superpixel partition, the Wishart-merging stage (WMS) simultaneously merges the regions in homogeneous areas. The edge penalty is combined with the Wishart energy loss to ensure that the superpixels to be merged are from the same land cover. The second stage follows the iterative merging procedure, and applies the doubly flexible KummerU distribution to better characterize the resultant regions from WMS, which are usually located in heterogeneous areas. Moreover, the edge penalty and the proposed homogeneity penalty are adopted in the KummerU-merging stage (KUMS) to further improve the segmentation accuracy. The two-stage merging strategy applies the general statistical model for the superpixels without ambiguity, and more advanced model for the regions with ambiguity. Therefore, the implementing efficiency can be improved based on the WMS, and the accuracy can be increased through the KUMS. Experimental results on two real PolSAR datasets show that the proposed method can effectively improve the computation efficiency and segmentation accuracy compared with the classical merging-based methods.

Spatial-spectral neighbour graph for dimensionality reduction of hyperspectral image classification

ABSTRACTRecently, graph embedding-based methods have drawn increasing attention for dimensionality reduction (DR) of hyperspectral image (HSI) classification. Graph construction is a critical step for those DR methods. Pairwise similarity graph is generally employed to reflect the geometric structure in the original data. However, it ignores the similarity of neighbouring pixels. In order to further improve the classification performance, both spectral and spatial-contextual information should be taken into account in HSI classification. In this paper, a novel spatial-spectral neighbour graph (SSNG) is proposed for DR of HSI classification, which consists of the following four steps. First, a superpixel-based segmentation algorithm is adopted to divide HSI into many superpixels. Second, a novel distance metric is utilized to reflect the similarity of two spectral pixels in each superpixel. In the third step, a spatial-spectral neighbour graph is constructed according to the above distance metric. At last, support vector machine with a composite kernel (SVM-CK) is adopted to classify the dimensionality-reduced HSI. Experimental results on three real hyperspectral datasets demonstrate that our method can achieve higher classification accuracy with relatively less consumed time than other graph embedding-based methods.

Automatic superpixel-based segmentation method for breast ultrasound images

Automatic and accurate breast ultrasound (BUS) image segmentation is crucial to achieve effective ultrasound-based computer aided diagnosis (CAD) systems for breast cancer. However, segmenting the tumor in BUS images is often challenging due to several artifacts that degrade the quality of ultrasound images. In this study, a new two-phase method is proposed to enable automatic and accurate segmentation of BUS images by decomposing the image into superpixels with high boundary recall ratio and employing edge- and region-based information to outline the tumor. The first phase of the method obtains an initial outline of the tumor by decomposing the BUS image into coarse superpixels to enable effective estimation of their tumor likelihoods and employing a customized graph cuts algorithm to segment the superpixels. The segmentation of the superpixels is carried out using edge-based information that quantifies the image contour cue and region-based information that characterizes the texture content of the superpixels. In the second phase, the tumor outline is improved by decomposing the BUS image into fine superpixels that enable high boundary recall ratio and employing the customized graph cuts algorithm to segment the superpixel located around the initial tumor outline. Furthermore, an edge-based active contour model is used to smooth the tumor outline. The performance of the proposed method was evaluated using a database that includes 160 BUS images (86 benign and 74 malignant). The results indicate that the first phase of the proposed method was able to detect the tumor in all BUS images and obtain mean values of the true positive ratio (TPR), false positive ratio (FPR), false negative ratio (FNR), similarity ratio (SIR), Hausdorff error (HE), and mean absolute error (ME) equal to 91.68, 11.16, 8.32, 84.52, 17.59, and 4.67, respectively. In fact, the results obtained by the first phase of the proposed method outperform four existing BUS image segmentation algorithms. Moreover, the second phase of the proposed method was able to improve the tumor outlines of the first phase and achieve mean TPR, FPR, FNR, SIR, HE, and ME values of 96.04, 7.99, 3.96, 91.41, 11.66, and 3.65, respectively. These results suggest the feasibility of employing the proposed method, which enables automatic and accurate tumor segmentation in BUS images, to develop effective CAD systems for breast cancer.

Guided lazy snapping for long thin object selection

We show a novel way to select long thin objects in an image by enhancing the output of the existing foreground/background image segmentation methods. Most superpixel-based methods fail to select the long thin details, such as legs and whiskers, and extended curves from the main objects. We observe, however, the output without long thin details, can be used as the guided information to obtain the connected components. Based on this observation, our Guided Lazy Snapping method overcomes the limitation of the Lazy Snapping methods (or other alternatives superpixel-based segmentation method) to select long thin objects. The results show that connected components in the image can be selected without having a lot of user interactions (mouse clicks) on each extended parts of the object.

Epithelium and Stroma Identification in Histopathological Images Using Unsupervised and Semi-Supervised Superpixel-Based Segmentation

We present superpixel-based segmentation frameworks for unsupervised and semi-supervised epithelium-stroma identification in histopathological images or oropharyngeal tissue micro arrays. A superpixel segmentation algorithm is initially used to split-up the image into binary regions (superpixels) and their colour features are extracted and fed into several base clustering algorithms with various parameter initializations. Two Consensus Clustering (CC) formulations are then used: the Evidence Accumulation Clustering (EAC) and the voting-based consensus function. These combine the base clustering outcomes to obtain a more robust detection of tissue compartments than the base clustering methods on their own. For the voting-based function, a technique is introduced to generate consistent labellings across the base clustering results. The obtained CC result is then utilized to build a self-training Semi-Supervised Classification (SSC) model. Unlike supervised segmentations, which rely on large number of labelled training images, our SSC approach performs a quality segmentation while relying on few labelled samples. Experiments conducted on forty-five hand-annotated images of oropharyngeal cancer tissue microarrays show that (a) the CC algorithm generates more accurate and stable results than individual clustering algorithms; (b) the clustering performance of the voting-based function outperforms the existing EAC; and (c) the proposed SSC algorithm outperforms the supervised methods, which is trained with only a few labelled instances.

Supervised and unsupervised segmentation using superpixels, model estimation, and graph cut

Image segmentation is widely used as an initial phase of many image analysis tasks. It is often advantageous to first group pixels into compact, edge-respecting superpixels, because these reduce the size of the segmentation problem and thus the segmentation time by an order of magnitudes. In addition, features calculated from superpixel regions are more robust than features calculated from fixed pixel neighborhoods. We present a fast and general multiclass image segmentation method consisting of the following steps: (i) computation of superpixels; (ii) extraction of superpixel-based descriptors; (iii) calculating image-based class probabilities in a supervised or unsupervised manner; and (iv) regularized superpixel classification using graph cut. We apply this segmentation pipeline to five real-world medical imaging applications and compare the results with three baseline methods: pixelwise graph cut segmentation, supertexton-based segmentation, and classical superpixel-based segmentation. On all datasets, we outperform the baseline results. We also show that unsupervised segmentation is surprisingly efficient in many situations. Unsupervised segmentation provides similar results to the supervised method but does not require manually annotated training data, which is often expensive to obtain.

Object Segmentation Ensuring Consistency Across Multi-Viewpoint Images.

We present a hybrid approach that segments an object by using both color and depth information obtained from views captured from a low-cost RGBD camera and sparsely-located color cameras. Our system begins with generating dense depth information of each target image by using Structure from Motion and Joint Bilateral Upsampling. We formulate the multi-view object segmentation as the Markov Random Field energy optimization on the graph constructed from the superpixels. To ensure inter-view consistency of the segmentation results between color images that have too few color features, our local mapping method generates dense inter-view geometric correspondences by using the dense depth images. Finally, the pixel-based optimization step refines the boundaries of the results obtained from the superpixel-based binary segmentation. We evaluate the validity of our method under various capture conditions such as numbers of views, rotations, and distances between cameras. We compared our method with the state-of-the-art methods that use the standard multi-view datasets. The comparison verified that the proposed method works very efficiently especially in a sparse wide-baseline capture environment.

Superpixel-based segmentation of glottal area from videolaryngoscopy images

Segmentation of the glottal area with high accuracy is one of the major challenges for the development of systems for computer-aided diagnosis of vocal-fold disorders. We propose a hybrid model combining conventional methods with a superpixel-based segmentation approach. We first employed a superpixel algorithm to reveal the glottal area by eliminating the local variances of pixels caused by bleedings, blood vessels, and light reflections from mucosa. Then, the glottal area was detected by exploiting a seeded region-growing algorithm in a fully automatic manner. The experiments were conducted on videolaryngoscopy images obtained from both patients having pathologic vocal folds as well as healthy subjects. Finally, the proposed hybrid approach was compared with conventional region-growing and active-contour model-based glottal area segmentation algorithms. The performance of the proposed method was evaluated in terms of segmentation accuracy and elapsed time. The F-measure, true negative rate, and dice coefficients of the hybrid method were calculated as 82%, 93%, and 82%, respectively, which are superior to the state-of-art glottal-area segmentation methods. The proposed hybrid model achieved high success rates and robustness, making it suitable for developing a computer-aided diagnosis system that can be used in clinical routines.